Index files

[bentsherman]

The language server needs to access metadata for built-in definitions such as standard library constants and functions, both from Nextflow (e.g. script variables and functions, process directives, operators, channel factories) and Java/Groovy (e.g. list / map / string methods), so that it can provide types, function signatures, and descriptions for features like hover hints and code completion.

The current approach is to define some "placeholder classes" which mimic the true classes in the Nextflow runtime and include annotations to identify "user-facing" members and provide descriptions. If this approach is stabilized, it can eventually be incorporated into Nextflow itself.

There are a few limitations here:

• free-text descriptions would eventually be included into the Nextflow JAR, which adds some small amount of bloat
• the language server must use a particular version of Nextflow as a library dependency, which could create some challenges with backwards compatibility
• Javadoc/Groovydoc comments can't be used because they aren't included in the JAR (so no descriptions for Java/Groovy classes)

On the other hand, the current approach is mostly fine and is easy to maintain. However, eventually we might consider storing this information in an index file which the language server could query. I believe this is how other IDEs like IntelliJ solve this problem.

There is already a spec called LSIF which is designed to integrate seamlessly with LSP, and does not enforce any particular encoding or storage backend. So the main challenge for us would be to extract the following information from source code:

• annotated classes and members
• field types
• function signatures
• Javadoc/Groovydoc content

Basically we would need some kind of build tool to extract this information into an index file which can be queried via LSIF, then the language server could use it instead of library dependencies. Benefits:

• provide documentation via Groovydoc comments which won't bloat the Nextflow JAR
• easily support library definitions for multiple Nextflow versions
• include documentation for Java/Groovy classes (as long as their source code is available)

The build tool is essentially Javadoc but targeting an index file instead of HTML. In fact I wonder if Javadoc can be repurposed in this way, otherwise we might have to build it ourselves.

[bentsherman]

Looks like the Doclet API can be used to define custom output formats for javadoc: https://docs.oracle.com/javase/7/docs/technotes/guides/javadoc/doclet/overview.html

So we could write a custom "doclet" to target LSIF.

Groovydoc does not appear to support the doclet API. I suppose we could migrate our stdlib classes to Java...