Update textDocument/documentSymbol parsing to LSP 3.10.0 spec

[ilohmar]

https://microsoft.github.io/language-server-protocol/specification#textDocument_documentSymbol

The code is correct, I hope, but possibly very inelegant, as it is the
first time I have worked with the eglot-* macros.

Also note: xref-backend-identifier-completion-table needs the same
parsing, but I have not had a closer look at the xref part yet. I think
we should parse in one place, to a structure keeping all information,
and then cast that into the necessary form for either xref or imenu.

---

[nemethf]

This seems useful, but I'm not familiar with this part of the code, so I'd rather not comment the code changes. However, there's a pending pull request (#213) also modifying imenu related code. There, João asked for writing accompanying tests. I think it's not a must, but can you do that for this PR?

[joaotavora]

@ilohmar As @nemethf mentions, I think tests are a good idea, if for no other reason, to better understand what this is about here (I have limited time to follow the spec nowadays).

I also see there's a recursive call to eglot--parse-DocumentSymbol. Can you explain why?

[joaotavora]

I also see there's a recursive call to eglot--parse-DocumentSymbol. Can you explain why?

Nevermind, I see it's a recursive data structure, so recursiveness makes sense.

[ilohmar]

Yeah, it's actually a tree (although I have no clue yet how to best use this for imenu).. :)

I will try to get around to writing tests, but preferably by mocking the behavior of the language server, instead of relying on actual servers being available.

In any case, that would probably only happen after I figure out how to extract the parsing from the xref part as well. Are there any comments explaining how that works? It seems to cache identifiers from the current buffer or sth like that..?

[joaotavora]

@ilohmar do you know of any other server that supports this? I was hoping I could avoid installing gopls.

[ilohmar]

Sorry, I have only used pyls, gopls and clangd myself. Have you ever worked with Go? I understand trying to avoid dependencies, however, the actual installation itself is easy and fast.

[galeo]

@ilohmar do you know of any other server that supports this? I was hoping I could avoid installing gopls.

Hi @joaotavora

AFAIK, the Microsoft‘s python-language-server and Apple's swift language server sourcekit-lsp support textDocument/documentSymbol feature.

I'm looking forward to seeing this feature supported by eglot.

[joaotavora]

AFAIK, the Microsoft‘s python-language-server and Apple's swift language server sourcekit-lsp support textDocument/documentSymbol feature.

I'm not in a position to install any of those right now. But this is indeed in my queue. I will clean it up and put it in a side repo where you can test with those servers

[joaotavora]

I just resolved some conflicts and now it applies cleanly on master. But I haven't tested in any way.

[muffinmad]

@ilohmar Please specify hierarchicalDocumentSymbolSupport in the TextDocumentClientCapabilities.documentSymbol as well. This way server will know that we support this feature.

[sstepashka]

Is there way to prioritize the PR? :)

[joaotavora]

Is there way to prioritize the PR? :)

I get the message. It helps if you can confirm that it applies cleanly to master, addresses @muffinmad 's comment and works correctly for at least one server where it wasn't working before. Post your results here, please. Thanks.

[joaotavora]

To be clear: doing any of those things helps, you don't have to do all of them to be useful.

[joaotavora]

I've pushed some changes to the implementation, in hopes of simplifying it, but I don't have any server to test with yet. Maybe someone can do that testing for me?

[sstepashka]

Sorry fo delay. It works! thank you very much!

[muffinmad]

Thanks @joaotavora !

I can't test this feature right now, but I notice that Eglot is now sending textDocument/documentSymbol even if server reported :documentSymbolProvider :json-false in textDocumentSync capabilities.

[joaotavora]

I can't test this feature right now, but I notice that Eglot is now sending
textDocument/documentSymbol even if server reported :documentSymbolProvider :json-false in
textDocumentSync capabilities.

Ahh, good catch. A simple if should do. Or maybe an eglot-error? Not sure if that's a good idea since other stuff will call this function non-interactively.

[muffinmad]

Example python file:

def some_function():
    def some_inline_function():
        pass
    pass


class SomeClassName:
    foo = 1

    def method_name(self):
        pass


class SomeOtherClass:
    foo = 2

    def method_name(self):
        pass

For that file the server responded to the documentSymbol request with the following hierarchical DocumentSymbol list:

(:id 2 :jsonrpc "2.0" :result
     [(:name "some_function" :kind 12 :range
	     (:start
	      (:line 0 :character 4)
	      :end
	      (:line 0 :character 20))
	     :selectionRange
	     (:start
	      (:line 0 :character 4)
	      :end
	      (:line 0 :character 20))
	     :detail nil :children
	     [(:name "some_inline_function" :kind 12 :range
		     (:start
		      (:line 1 :character 8)
		      :end
		      (:line 1 :character 31))
		     :selectionRange
		     (:start
		      (:line 1 :character 8)
		      :end
		      (:line 1 :character 31))
		     :detail nil :children nil :deprecated :json-false)]
	     :deprecated :json-false)
      (:name "SomeClassName" :kind 5 :range
	     (:start
	      (:line 6 :character 6)
	      :end
	      (:line 6 :character 20))
	     :selectionRange
	     (:start
	      (:line 6 :character 6)
	      :end
	      (:line 6 :character 20))
	     :detail nil :children
	     [(:name "foo" :kind 13 :range
		     (:start
		      (:line 7 :character 4)
		      :end
		      (:line 7 :character 11))
		     :selectionRange
		     (:start
		      (:line 7 :character 4)
		      :end
		      (:line 7 :character 11))
		     :detail nil :children nil :deprecated :json-false)
	      (:name "method_name" :kind 12 :range
		     (:start
		      (:line 9 :character 8)
		      :end
		      (:line 9 :character 26))
		     :selectionRange
		     (:start
		      (:line 9 :character 8)
		      :end
		      (:line 9 :character 26))
		     :detail nil :children nil :deprecated :json-false)]
	     :deprecated :json-false)
      (:name "SomeOtherClass" :kind 5 :range
	     (:start
	      (:line 13 :character 6)
	      :end
	      (:line 13 :character 21))
	     :selectionRange
	     (:start
	      (:line 13 :character 6)
	      :end
	      (:line 13 :character 21))
	     :detail nil :children
	     [(:name "foo" :kind 13 :range
		     (:start
		      (:line 14 :character 4)
		      :end
		      (:line 14 :character 11))
		     :selectionRange
		     (:start
		      (:line 14 :character 4)
		      :end
		      (:line 14 :character 11))
		     :detail nil :children nil :deprecated :json-false)
	      (:name "method_name" :kind 12 :range
		     (:start
		      (:line 16 :character 8)
		      :end
		      (:line 16 :character 26))
		     :selectionRange
		     (:start
		      (:line 16 :character 8)
		      :end
		      (:line 16 :character 26))
		     :detail nil :children nil :deprecated :json-false)]
	     :deprecated :json-false)])

The Imenu structure now looks like this:

+ Class
   SomeClassName
   SomeOtherClass
+ Variable
   + SomeClassName
     foo
   + SomeOtherClass
     foo
+ Function
   some_function
   + some_function
     some_inline_function
   + SomeClassName
     method_name
   + SomeOtherClass
     method_name

I used to Imenu structure provided by python-imenu-create-flat-index that looks like this (for the same file):

+ some_function (def)
   *function definition*
   some_inline_function (def)
+ SomeClassName (class)
   *class definition*
   method_name (def)
+ SomeOtherClass (class)
   *class definition*
   method_name (def)

I found the later Imenu structure more compact, intuitive and easy to use. No need to think about type of item to navigate to at the first place.

Maybe it would be better to not group Imenu items by item type but only by hierarchical structure?

What do you think?

P.S. The python language server is anakinls. The documentSymbol implementation is not published on pypi yet, so in case someone want to check it in action, please install it from master.

[joaotavora]

No need to think about type of item to navigate to at the first place.

Yes, I agree. imenu is doing needless restructuring: it shouldn't. The reason it does it is historical I think: there were no DocumentSymbol trees before. BUt now that there are, there's no point in doing this. I'll check this later.