Java 9 WARNING: Illegal reflective access by com.google.protobuf.UnsafeUtil

[eed3si9n]

steps

Run on Java 9.

problem

[info] Compiling 1 Scala source to /root/work/hello/project/target/scala-2.12/sbt-1.0/classes ...
[info] Non-compiled module 'compiler-bridge_2.12' for Scala 2.12.4. Compiling...
[info]   Compilation completed in 9.719s.
[info] Done compiling.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.protobuf.UnsafeUtil (file:/root/.sbt/boot/scala-2.12.4/org.scala-sbt/sbt/1.0.4-LOCAL1/protobuf-java-3.3.1.jar) to field java.nio.Buffer.address
WARNING: Please consider reporting this to the maintainers of com.google.protobuf.UnsafeUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

expectation

No warnings

notes

See protocolbuffers/protobuf#3781

[mkurz]

I am getting something different:

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by sbt.internal.inc.javac.DiagnosticsReporter$PositionImpl$ (file:/home/mkurz/.sbt/boot/scala-2.12.4/org.scala-sbt/sbt/1.1.2/zinc-compile-core_2.12-1.1.3.jar) to field com.sun.tools.javac.api.ClientCodeWrapper$DiagnosticSourceUnwrapper.d
WARNING: Please consider reporting this to the maintainers of sbt.internal.inc.javac.DiagnosticsReporter$PositionImpl$
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

I already use sbt 1.1.2 (released 3 hours ago)

[eed3si9n]

protocolbuffers/protobuf#4557

[eximius313]

On JDK10:

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.protobuf.UnsafeUtil (file:/C:/Users/X/.sbt/boot/scala-2.12.6/org.scala-sbt/sbt/1.1.5/protobuf-java-3.3.1.jar) to field java.nio.Buffer.address
WARNING: Please consider reporting this to the maintainers of com.google.protobuf.UnsafeUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

[ghost]

this is getting annoying. is there a way to suppress this warning? Also i get that error even with jdk8. the problem is if jdk9 or 10 its installed at all

[mengwong]

This error shows up as part of the standard Scala Hello World tutorial from https://docs.scala-lang.org/getting-started-sbt-track/getting-started-with-scala-and-sbt-on-the-command-line.html running against the latest JDK 10.

Speaking on behalf of Scala noobs everywhere – this feels alarming!

It's probably not a good idea to amend the tutorial to say "ignore this warning", because ignoring warnings is not a habit people should be getting into.

Getting to the root of the problem and fixing it is a little beyond me right now, though.

20180729-20:35:49 mengwong@venice4:~/tmp/scala% sbt new scala/hello-world.g8
[info] Set current project to scala (in build file:/Users/mengwong/tmp/scala/)
[info] Set current project to scala (in build file:/Users/mengwong/tmp/scala/)

A template to demonstrate a minimal Scala application

name [Hello World template]: Hello World

Template applied in ./hello-world

20180729-20:36:02 mengwong@venice4:~/tmp/scala% cd hello-world
20180729-20:36:08 mengwong@venice4:~/tmp/scala/hello-world% javac -version
javac 10.0.1
20180729-20:36:36 mengwong@venice4:~/tmp/scala/hello-world% (grep scalaVersion build.sbt; grep sbt.version project/build.properties) | grep -v '//'
scalaVersion := "2.12.6"
sbt.version=1.1.6
20180729-20:36:41 mengwong@venice4:~/tmp/scala/hello-world% sbt
[info] Loading project definition from /Users/mengwong/tmp/scala/hello-world/project
[info] Updating ProjectRef(uri("file:/Users/mengwong/tmp/scala/hello-world/project/"), "hello-world-build")...
[info] Done updating.
[info] Loading settings from build.sbt ...
[info] Set current project to hello-world (in build file:/Users/mengwong/tmp/scala/hello-world/)
[info] sbt server started at local:///Users/mengwong/.sbt/1.0/server/2f22de7e2f2b615217b5/sock
sbt:hello-world> run
[info] Updating ...
[info] Done updating.
[info] Compiling 1 Scala source to /Users/mengwong/tmp/scala/hello-world/target/scala-2.12/classes ...
[info] Done compiling.
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.protobuf.UnsafeUtil (file:/Users/mengwong/.sbt/boot/scala-2.12.6/org.scala-sbt/sbt/1.1.6/protobuf-java-3.3.1.jar) to field java.nio.Buffer.address
WARNING: Please consider reporting this to the maintainers of com.google.protobuf.UnsafeUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
[info] Packaging /Users/mengwong/tmp/scala/hello-world/target/scala-2.12/hello-world_2.12-1.0.jar ...
[info] Done packaging.
[info] Running Main
Hello, World!
[success] Total time: 2 s, completed Jul 29, 2018, 8:37:05 PM
sbt:hello-world>

[dashuser33]

Also getting this on JVM 11.

[mkurz]

protocolbuffers/protobuf#3781 has been fixed!

[eed3si9n]

Nice.

[matschaffer]

I'm getting the same error as #459 (comment) which seems to be coming from https://github.com/sbt/zinc/blob/develop/internal/zinc-compile-core/src/main/scala/sbt/internal/inc/javac/DiagnosticsReporter.scala#L115

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by sbt.internal.inc.javac.DiagnosticsReporter$PositionImpl$ (file:/Users/mat/.sbt/boot/scala-2.12.7/org.scala-sbt/sbt/1.2.8/zinc-compile-core_2.12-1.2.5.jar) to field com.sun.tools.javac.api.ClientCodeWrapper$DiagnosticSourceUnwrapper.d
WARNING: Please consider reporting this to the maintainers of sbt.internal.inc.javac.DiagnosticsReporter$PositionImpl$
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

Looks like the reflection there is a pretty old line 3f92b2b#diff-9b267fe75b6831da3dca25ceb91082e9R75 - before the Java 9 release date, so seems likely it's just not compatible.

[adampauls]

Looks like protocolbuffers/protobuf#5812 got fixed and 3.7.0 can be used now!

[eatkins]

This issue was not actually fixed by this change because the zinc-persist and zinc-core still have a transitive dependency on scalapb-runtime:0.6.0. The warnings are still printed with java 9 using sbt 1.3.0-M2.

[eatkins]

The head of scalapb-runtime updates the dependency to 3.7.0.1, but they haven't made a release yet. I opened up an issue: scalapb/ScalaPB#559.

[thesamet]

Hey all, ScalaPB has 0.9.0-RC1 which depends on protobuf-java 3.7.0. Note that you are using a very old version of ScalaPB and we had an artifact name change since then. Migration instructions are here: https://scalapb.github.io/migrating.html

In the past, zinc dependncy on ScalaPB caused issues to SBT users that use ScalaPB since they might end up with incompatible versions of ScalaPB or protobuf. It would be great to add some integration tests to SBT to ensure an updated zinc/sbt doesn't cause a regression to ScalaPB users.

[smarter]

This issue was not actually fixed by this change because the zinc-persist and zinc-core still have a transitive dependency on scalapb-runtime:0.6.0. The warnings are still printed with java 9 using sbt 1.3.0-M2.

The same dependencyOverrides that was added in zinc needs to be added in sbt: bc7a8f5

The head of scalapb-runtime updates the dependency to 3.7.0.1, but they haven't made a release yet. I opened up an issue: scalapb/ScalaPB#559.

Upgrading scalapb in zinc is apparently verboten due to unfortunate technical decisions: #546, though there's one proposal to get out of this mess: sbt/sbt#4183

[eatkins]

@smarter I still maintain that this is a zinc issue because we are forcing every downstream consumer of zinc to manually override the protobuf-java dependency.

[eed3si9n]

Upgrading scalapb in zinc is apparently verboten due to unfortunate technical decisions: #546, though there's one proposal to get out of this mess: sbt/sbt#4183

I am happy to explore all avenues:

1. get sbt-big to work (likely requires some code gen)
2. bump to latest ScalaPB
3. fork / shade ScalaPB under our own organization name

[eed3si9n]

I just noticed #647 that @eatkins sent. As far as I'm concerned, this is a good enough fix.

[eatkins]

Resolved by #647

[mkurz]

Using just released sbt 1.3.0-M3 there is still a problem:

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by sbt.internal.inc.javac.DiagnosticsReporter$PositionImpl$ (file:/home/mkurz/.sbt/boot/scala-2.12.8/org.scala-sbt/sbt/1.3.0-M3/zinc-compile-core_2.12-1.3.0-M4.jar) to field com.sun.tools.javac.api.ClientCodeWrapper$DiagnosticSourceUnwrapper.d
WARNING: Please consider reporting this to the maintainers of sbt.internal.inc.javac.DiagnosticsReporter$PositionImpl$
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

OpenJDK 11:

$ java -version
openjdk version "11.0.2" 2019-01-15
OpenJDK Runtime Environment 18.9 (build 11.0.2+9)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.2+9, mixed mode)

You should be able to reproduce this by using the play-java-seed template:

sbt new playframework/play-java-seed.g8

[smarter]

A separate issue should be opened for that (also someone should run sbt with --illegal-access=warn to check if there aren't more hidden problems like that ...)

[mkurz]

@smarter Opened #664