vscode-languageserver-node: Is there some document about how language server receive semantic tokens request?

[imbant]

Hi there.

I'm developing a language server. It provides semantic highlight. By I'm not sure how to receive something like textDocument/semanticTokens/full from client.

My code like this:

import { createConnection, ProposedFeatures } from "vscode-languageserver/node";

const connection = createConnection(ProposedFeatures.all);

connection.onInitialize(() => {
  // ...
  return {
    semanticTokensProvider: {
      legend: {
        tokenTypes: ["function", "namespace"], // register legend like the same thing in VS Code example
        tokenModifiers: [],
      },
    },
  }
})

connection.languages.semanticTokens.on(() => {
  // DOESN'T called 
});

connection.languages.semanticTokens.on doesn't trigger my callback when client open document, change text. I add breakpoints and logs and prove it doesn't work.

I have read this guide and I have successfully implemented semantic highlighting through the vs code API. But this is not the same in languageserver-server.
I don't know how to use connection. I got languages.semanticTokens.on just by intelligent completion. Is there some document about this? Additional, can language server manually update or push semantic tokens to client, for force update?

[theReynald]

Hi @imbant,

The connection.languages.semanticTokens.on event is not typically used for providing semantic tokens to the client in response to a request like textDocument/semanticTokens/full. Instead, you should implement the textDocument/semanticTokens request handler in your language server.

Here's how you can set up your language server to handle textDocument/semanticTokens/full requests:

import { createConnection, ProposedFeatures } from "vscode-languageserver/node";

const connection = createConnection(ProposedFeatures.all);

connection.onInitialize(() => {
  // ...
  return {
    semanticTokensProvider: {
      legend: {
        tokenTypes: ["function", "namespace"], // register legend like the same thing in VS Code example
        tokenModifiers: [],
      },
    },
  }
});

connection.onRequest("textDocument/semanticTokens/full", (params) => {
  // Implement your logic to provide semantic tokens for the given document here.
  // You should return the semantic tokens as a response.
  const semanticTokens = computeSemanticTokens(params.textDocument.uri);
  return semanticTokens;
});

// Start the language server
connection.listen();

In the code above:

1. We use the connection.onRequest method to define a request handler for the textDocument/semanticTokens/full request. When this request is received from the client, your handler function will be called.

2. Inside the handler function, you should implement the logic to compute semantic tokens for the given document specified by params.textDocument.uri. You will need to return the computed semantic tokens as a response.

3. Finally, the connection.listen() method is used to start the language server and listen for incoming requests.

Make sure that your handler function computes and returns the semantic tokens in the format expected by the client, following the Language Server Protocol (LSP) specification for semantic tokens.

Regarding pushing semantic tokens to the client, in most cases, the client (e.g., Visual Studio Code) sends requests to the language server to obtain semantic tokens. However, you can't push semantic tokens to the client at will. The client initiates the requests when it needs the tokens, such as when opening a document or making changes to the text. The language server responds to these requests with the corresponding semantic tokens. You can't force updates; the client controls when it requests updates.

Hope this helped.

[imbant]

Thank you very much for your detailed answer! I know how to receive the textDocument/semanticTokens/full request now. And it does work on my server.

Besides, I opened range config in semanticTokensProvider, then connection.languages.semanticTokens.onRange works.

connection.onInitialize(() => {
  // ...
  return {
    semanticTokensProvider: {
++    range: true,
      legend: {
        tokenTypes: ["function", "namespace"], // register legend like the same thing in VS Code example
        tokenModifiers: [],
      },
    },
  }

Now I know server should not force update semantic tokens. Originally, I was worried that when requesting semantic tokens, the server had not completed semantic analysis and might return the previous version of the token. But I circumvented this by delaying the response with a await sleep(200) Primise.

[deanmaster]

hi @imbant ,

Could you share with me Client capabilities config so that it will send for semantictokens/full ?

Thanks,
Tuan do

[imbant]

For the client(vscode), I use vscode-languageclient/node npm package and construct a LanguageClient instance. I just call registerProposedFeatures() on it and it works fine.

For the server, my config is like:

            semanticTokensProvider: {
                full: true,
                legend: {
                    tokenTypes: [...tokenTypes],
                    tokenModifiers: [...tokenModifiers],
                },
            },

Hope this can help you

[imbant]

BTW, I learned a lot from Volar, like this

[deanmaster]

yes my server has correct configuration. But the client couldn't send the request. I used genericLSPClient plugin so I don't configure anything from client side actually.

semanticTokensProvider: {
				legend: {
					tokenTypes: [...tokenTypesLegend],
					tokenModifiers: [...tokenModifiersLegend]
				},
				full: true,
				range: false
			}

BTW, do you know how to enable verbose in Language Server to see all requests are going on. These requests are avalable in Intelij but not in VSCode

[imbant]

Hi. I just checked some documents but it seems I can't provide more useful information right now.
This repo may help you enable verbose, but I'm not sure since I can't run this demo successfully