Interoperability with Java language servers

[gabro]

Metals is a language server for Scala, but it's very common for a project to have either Java dependencies or Java sources.

When jumping to a definition and landing on a Java source file, you're on your own, Metals won't help you there.

Fortunately, there exist Java language servers, which can provide features for Java sources. These servers (such as https://github.com/eclipse/eclipse.jdt.ls) need to know the project's classpath in order to provide features like build import, navigation, find references, and so on.

Metals knows the project's classpath, so it could make this information available to the Java LS.

One possible approach is to generate a POM file, but let's discuss other options.

[liewhite]

Any progress?

[gabro]

@liewhite no, sorry.

For clarity: features that are being actively worked on are moved to the main issue tracker, so if you see something here then you can assume it is still not in progress.

[regadas]

Hi just stumbled upon this!

With vscode and latest java extension I'm doing a small trick (dirty) to allow better java go to definition. Here we can see that we can reference libraries through:

"java.project.referencedLibraries": [
    "library/**/*.jar",
    "/home/username/lib/foo.jar"
]

so I basically generate a list based on .bloop

jq 'reduce ([inputs.project.classpath[] | select(. | test(".jar"))] | sort | unique) as $jar ({}; ."java.project.referencedLibraries" += $jar)' .bloop/*.json

it's meh but helps a bit when inspecting some java libs.

[gdoenlen]

@gabro Instead of generating a pom file why not directly generate or modify the eclipse project files that the java lang server uses? Much like how https://github.com/sbt/sbteclipse does it. I've had good success with getting it to work in java projects, but it seems to miss the scala files when generating.

[gabro]

@gdoenlen I have very little experience with working on Java projects, so that may well be a solution.

@regadas that's interesting too!

Anyway, I won't like have time to work on this anytime soon, so it's up for grab for whoever wants it

[tgodzik]

Looks like it's possible to generate eclipse files via sbt-eclipse or similar tool in gradle or maven. Those files will be later used by Visual Studio Code Java plugin to provide language support for java code. The main problem with separate plugin is the fact that rename/references and anything that requires indexes will not work for all files. However, this will be a workaround until we provide proper support in Metals for Java.

In short for sbt:

• add eclipse plugin:
  addSbtPlugin("com.typesafe.sbteclipse" % "sbteclipse-plugin" % "5.2.4")
• run eclipse command:
  sbt eclipse
• open the project with java redhat plugin enabled

[vishalovercome]

While the suggested approach solves the original issue of code navigation, metals constantly complains about java methods being undefined (in Java source files) even when the code compiles are runs just fine. Is there any additional configuration that needs to be done to get rid of these spurious messages.

Separately, is there any ongoing work to improve interoperability with Java without using the eclipse plugin?

[tgodzik]

While the suggested approach solves the original issue of code navigation, metals constantly complains about java methods being undefined (in Java source files) even when the code compiles are runs just fine. Is there any additional configuration that needs to be done to get rid of these spurious messages.

I think it's not Metals complaining. Is it possible it's coming from the java language server?

Separately, is there any ongoing work to improve interoperability with Java without using the eclipse plugin?

There is some work, you can take a look at scalameta/metals#2520, which I plan to merge after the next Bloop release.

[spangaer]

TBH, I've had most success using java lightweight mode. It has java syntax highlighting an jumps from scala work. It would be great if it all works. But for the time being if you like error and warning free code that sort of helps.

[stumash]

the real killer feature that metals needs is:

1. Go to definition from a Scala file brings you to a Java one
2. All usages in that Java file shows you all usages of a given java entity - including both usages within the Java codebase you're now exploring, AND those in the Scala codebase you came from

no idea how to do it, just something for the wishlist

[tgodzik]

@stumash that is actually almost done here scalameta/metals#2520 and waiting mostly on me 😓

[stumash]

@tgodzik even the 2nd part!? very impressive, and very excited for that feature! no rush haha

[tgodzik]

They're connected, so yeah. Both parts should be implemented at the same time.

[andrescmasmas]

@tgodzik @stumash As far as I know the first part is already possible. When I'm on scala file, 'Go to definition' takes me to the java file where the method is implemented

[tgodzik]

Yes, but it's not ideal currently. It will only show you the line and it will not show any references from java files.

[rcano]

Would this cover showing documentation for methods defined in java during auto completion? This is a big problem right now.

[tgodzik]

@rcano eventually yes, we could potentially use the parser from JDT for that.

[tgodzik]

Currently Metals supports a number of features aside from a couple that we would need JDT or a separate language server for:

• Formatting on Type
• Code completion
• Code outline
• Organize imports
• Hover (type at point)

[tgodzik]

I found that there is an alternative Java language server which doesn't use OSGI https://github.com/georgewfraser/java-language-server

It is less maintained than the redhat one, but we might be able to reuse parts of the code for the missing Java support features,

[JYInMyHeart]

Any progress?

[ckipp01]

Hey @JYInMyHeart if this is still in the feature requests repo and not moved to the main repo, and there are no comments on here, it's safe to assume there is no progress being made. While there has been a lot of progress over the last 6 months in better Java support in Metals, there has been no movement in interoperability with actual Java language servers.

[JYInMyHeart]

All right. Thanks for your excellent work.

[tgodzik]

I've looked into interoperability between servers and it seems to be really hard to achieve without and additional formal protocol. We're aiming instead to provide basic Java capabilities to metals. If your project is focused on Java it might be better to use the Java language server explicitely.

[gravelld]

Maybe this should be in the discussions area of the project...

It certainly feels to me like trying to reimplement Java tooling is not the scalable way to go, and your efforts would be better focused on Scala.

That said, my finger-in-the-air also suggests that in the real world mixed Java/Scala multi-module projects are the norm rather than nice, clean pure Scala setups. Therefore Java support will always be required. My own interest is here - I have Java code dependent on Scala code and vice versa. I want to use RedHat LSP when I'm working just on the Java stuff because it's better than Metals' (Bloop's) Java support.

This talks about having LSPs being supplied to the GraalVM LSP - is this a standardised approach, or something custom GraalVM LSP is doing? It doesn't sound like quite what we want between RedHat LSP and Metals, where it would be more of a collaborative peer type relationship, but I thought I'd point it out.

[tgodzik]

I would love to have some kind of interaction between the different servers, but to make that work might actually require more time than adding some minimal java features. RedHat LSP is not able to compile Scala, so I've seen multiple issues with making the setup work, we would need to work closely with the RedHat team to refine the approach.

Also, I was thinking of maybe running Java LSP in the background, but that requires creating a bunch of files for the server and it doesn't seem like an entirely clean approach Eclipse JDT and it's not possible to just run parts of the code. The architecture there depends on those files to exists, it can't get build information via parameters. We could try creating those files from the information we have, but that might prove to be flaky.

https://docs.oracle.com/en/graalvm/enterprise/20/docs/tools/lsp/ seems interesting and this might fit well with the initial idea behind this issue. But just merging the results from different servers might not always give us the best possible info. We need one server with the full knowledge of the workspace. All the implementations/references etc. Otherwise we might always be getting partial information.

[tanishiking]

Unfortunately, GraalVM LSP won't help this issue, I read the paper and watched the conference talk before.

https://dl.acm.org/doi/abs/10.1145/3359619.3359746
https://www.youtube.com/watch?v=YywetLaa8rQ

GraalVM LSP

• supports multiple languages based on Truffle Framework only
  • While Scala and Java aren't built on top of the Truffle
  • (we have to develop Scala interpreter using Truffle like TruffleRuby)
• The research focuses on how can we implement (multiple languages supported) LSP server for dynamic runtime (Truffle interpreter)
  • Rich language features based on semantic information for each language won't be available.
• AFAIK, GraalVM LSP is an experimental project
  • Features such as connecting with the build system are not available.

(off topic: on while GraalVM LSP is "production-ready", I believe it's still valuable, especially for education, and we, Programming Research Group at Tokyo Institute of Technology are working on live programming environment using Graal Toward a Multi-Language and Multi-Environment Framework for Live Programming – Programming Research Group )

[spangaer]

While Scala and Java aren't built on top of the Truffle

Not sure how realistic it is, but: https://www.graalvm.org/22.0/reference-manual/java-on-truffle/

RedHat LSP is not able to compile Scala

A smooth experience 'out-of-the-box' for 'first-java-then-scala compilation', where the Java part is backed by RedHat LSP, would already be a good deal. Without the need to resort to SBT Eclipse that is.

[spangaer]

I'm a little confused.

When upgrading to the latest sbt-bloop (1.5.3), when I run sbt-eclipse's eclipse command a new eclipse prefs file is being generated, specifically one that was missing before to correctly configure the Java version in the RedHat Java lsp.

😮

Update:
Nevermind, it looks like the new RedHat LSP is the one that's doing the writing

[spangaer]

Btw, I see sbt-eclipse got a reboot
https://github.com/sbt/sbt-eclipse
So just the two (metals and redhat lsp) just playing nice side-by-side is already good.
(past years I got that to work reasonably)

[JYInMyHeart]

Kotlin also has this problem. Maybe this issue can help.kotlin-java

[mdedetrich]

Know that I am maintaining a massive mixed Java/Scala codebase (Pekko, open source fork of Akka) this is also something I am really looking forward too and currently I am "forced" to use Intellij due to non trivial Java usage in a lot of the Pekko modules.

[lego-eden]

Stumbled upon this issue now and this would be awesome.