Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Java 10/11 should be supported #5723

Closed
ajrepp opened this issue Jul 31, 2018 · 48 comments
Closed

Java 10/11 should be supported #5723

ajrepp opened this issue Jul 31, 2018 · 48 comments
Assignees
Labels
P2 We'll consider working on this in future. (Assignee optional) team-Rules-Java Issues for Java rules type: feature request

Comments

@ajrepp
Copy link

ajrepp commented Jul 31, 2018

Description of the problem / feature request:

It seems like Bazel doesn't support building projects with Java 10 or Java 11. Java 10 is scheduled to be deprecated when Java 11 is released, currently estimated for September 25, 2018. In order to test and update our code base for Java 11, we need to be able to build it with Bazel.

Feature requests: what underlying problem are you trying to solve with this feature?

Bazel should support building against Java 10/11.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Follow the steps at https://github.com/ajrepp/java-upgrade-toy. The output is:

$ bazel build //:x --javabase=//:jdk10-jdk --java_toolchain=@bazel_tools//tools/jdk:toolchain_java10 --host_java_toolchain=@bazel_tools//tools/jdk:toolchain_java10 --host_javabase=//:jdk10-jdk
INFO: Build options have changed, discarding analysis cache.
INFO: Analysed target //:x (0 packages loaded).
INFO: Found 1 target...
ERROR: /Users/andrew.repp/code/java-upgrade-toy/BUILD:1:1: Building x.jar (1 source file) failed (Exit 1)
java.lang.IllegalArgumentException: release version 10 not supported
	at jdk.compiler/com.sun.tools.javac.main.Arguments.error(Arguments.java:917)
	at jdk.compiler/com.sun.tools.javac.main.Arguments.handleReleaseOptions(Arguments.java:310)
	at jdk.compiler/com.sun.tools.javac.main.Arguments.processArgs(Arguments.java:370)
	at jdk.compiler/com.sun.tools.javac.main.Arguments.init(Arguments.java:246)
	at jdk.compiler/com.sun.tools.javac.api.JavacTool.getTask(JavacTool.java:185)
	at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:97)
	at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder$2.invokeJavac(SimpleJavaLibraryBuilder.java:122)
	at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:54)
	at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:125)
	at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:133)
	at com.google.devtools.build.buildjar.BazelJavaBuilder.processRequest(BazelJavaBuilder.java:105)
	at com.google.devtools.build.buildjar.BazelJavaBuilder.runPersistentWorker(BazelJavaBuilder.java:67)
	at com.google.devtools.build.buildjar.BazelJavaBuilder.main(BazelJavaBuilder.java:45)
Target //:x failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 1.190s, Critical Path: 0.94s
INFO: 0 processes.
FAILED: Build did NOT complete successfully

What operating system are you running Bazel on?

macOS X 10.13.6

What's the output of bazel info release?

development version

If bazel info release returns "development version" or "(@non-git)", tell us how you built Bazel.

bazel build //scripts/packages

What's the output of git remote get-url origin ; git rev-parse master ; git rev-parse HEAD ?

https://github.com/bazelbuild/bazel.git
0a635c5
0a635c5

Have you found anything relevant by searching the web?

Any other information, logs, or outputs that you want to share?

We haven't been able to find a definitive answer anywhere about new Java version support or easy-to-understand documentation for how to use new versions. Any information would be appreciated!

@ajrepp
Copy link
Author

ajrepp commented Jul 31, 2018

It looks like this might be related to https://github.com/bazelbuild/bazel/blob/master/tools/jdk/default_java_toolchain.bzl#L37-L38 -- those jars might not support Java 10+?

@cushon
Copy link
Contributor

cushon commented Aug 21, 2018

Support for --release 10 is comparable to --release 9 as of 30aac1c.

@cushon cushon closed this as completed Aug 21, 2018
@ajrepp
Copy link
Author

ajrepp commented Aug 22, 2018

@cushon is Java 11 supported as well? Java 10 will no longer receive support once Java 11 is released, which is scheduled for September 25th (less than 5 weeks away!).

@cushon cushon reopened this Aug 22, 2018
@cushon
Copy link
Contributor

cushon commented Aug 22, 2018

Note that using JDK 11 as a --javabase should already work. The part that requires changes is supporting the Java 11 language level (--release 11, etc.), I will defer to @lberki for that.

@davido
Copy link
Contributor

davido commented Aug 24, 2018

Note that using JDK 11 as a --javabase should already work.

How would I set-up local jdk 11 to be used as --javabase to build gerrit with local jdk 11?

  $ echo $JAVA_HOME
  /usr/lib64/jvm/java-11

As you pointed out in this comment: [1], there are three JDKs used in Bazel, I think we want to change 2) and 3).

So I define this java_runtime rule:

java_runtime(
    name = "jdk11",
    java_home = "/usr/lib64/jvm/java-11",
    visibility = ["//visibility:public"],
)

And using it like this:

 $ bazel build --javabase //:jdk11 --host_javabase //:jdk11 java/com/google/gerrit/server:server

That works, and still raises the question how to pass post Java arguments to the tests, like we do for Java 9 and Java 10:

  $ bazel build --host_java_toolchain=@bazel_tools//tools/jdk:toolchain_java10 --java_toolchain=@bazel_tools//tools/jdk:toolchain_java10 :release

However, I would expect that the class version produced by the command above is major version 55 that corresponds to Java 11. To my surprise, the major version is still 52 (Java 8):

$ /usr/lib64/jvm/java-11/bin/javap -v -classpath bazel-bin/java/com/google/gerrit/server/libserver.jar com.google.gerrit.server.util.SocketUtil | more
Classfile jar:file:///home/davido/projects/gerrit2/bazel-bin/java/com/google/gerrit/server/libserver.jar!/com/google/gerrit/server/util/SocketUtil.class
  Last modified Jan 1, 1980; size 3601 bytes
  MD5 checksum 4379cabbb77f09e7317b9d14f4e6336b
  Compiled from "SocketUtil.java"
public final class com.google.gerrit.server.util.SocketUtil
  minor version: 0
  major version: 52

Am I missing something? Are we passing somewhere in the Bazel toolchain hard coded Java target = 8? Should I file own issue for that problem? Or is this the missing support for --release 11 in Bazel that you pointed out in last comment?

Update: Note, that when I use alternative toolchain //tools/jdk:toolchain_java10 then the major version is as expected for Java 10: 54:

$ bazel build --host_java_toolchain=@bazel_tools//tools/jdk:toolchain_java10 --java_toolchain=@bazel_tools//tools/jdk:toolchain_java10 java/com/google/gerrit/server:server
INFO: Build options have changed, discarding analysis cache.
INFO: Analysed target //java/com/google/gerrit/server:server (1 packages loaded).
INFO: Found 1 target...
INFO: From Building external/com_google_protobuf/libprotobuf_java.jar (77 source files, 1 source jar):
warning: [options] bootstrap class path not set in conjunction with -source 6
warning: [options] source value 6 is obsolete and will be removed in a future release
warning: [options] target value 1.6 is obsolete and will be removed in a future release
warning: [options] To suppress warnings about obsolete options, use -Xlint:-options.
INFO: From Building proto/libcache_proto-speed.jar (1 source jar):
warning: [options] bootstrap class path not set in conjunction with -source 7
Target //java/com/google/gerrit/server:server up-to-date:
  bazel-bin/java/com/google/gerrit/server/libserver.jar
INFO: Elapsed time: 0.749s, Critical Path: 0.24s
INFO: 55 processes: 55 remote cache hit.
INFO: Build completed successfully, 59 total actions
davido@wizball:~/projects/gerrit2 (master *%>)$ /usr/lib64/jvm/java-11/bin/javap -v -classpath bazel-bin/java/com/google/gerrit/server/libserver.jar com.google.gerrit.server.util.SocketUtil | more
Classfile jar:file:///home/davido/projects/gerrit2/bazel-bin/java/com/google/gerrit/server/libserver.jar!/com/google/gerrit/server/util/SocketUtil.class
  Last modified Jan 1, 1980; size 3601 bytes
  MD5 checksum f62f6ea242e64c848c0f6723ff47e57a
  Compiled from "SocketUtil.java"
public final class com.google.gerrit.server.util.SocketUtil
  minor version: 0
  major version: 54

@davido
Copy link
Contributor

davido commented Aug 24, 2018

If I pass, --release 11 in my previous command, that I'm getting error: release version 11 not supported. Can it be, that the JDK 11 doesn't actually used?

$ bazel build --javabase //:jdk11 --host_javabase //:jdk11 --javacopt="--release 11" //java/com/google/gerrit/server:server
INFO: Build options have changed, discarding analysis cache.
INFO: Analysed target //java/com/google/gerrit/server:server (1 packages loaded).
INFO: Found 1 target...
INFO: From Executing genrule @bazel_tools//tools/jdk:platformclasspath [for host]:
warning: [options] bootstrap class path not set in conjunction with -source 8
1 warning
ERROR: /home/davido/projects/gerrit2/java/org/eclipse/jgit/BUILD:37:1: Building java/org/eclipse/jgit/libserver.jar (2 source files) failed (Exit 1)
java.lang.IllegalArgumentException: release version 11 not supported
	at jdk.compiler/com.sun.tools.javac.main.Arguments.error(Arguments.java:897)
	at jdk.compiler/com.sun.tools.javac.main.Arguments.handleReleaseOptions(Arguments.java:311)
	at jdk.compiler/com.sun.tools.javac.main.Arguments.processArgs(Arguments.java:350)
	at jdk.compiler/com.sun.tools.javac.main.Arguments.init(Arguments.java:246)
	at jdk.compiler/com.sun.tools.javac.api.JavacTool.getTask(JavacTool.java:185)
	at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:97)
	at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder$1.invokeJavac(SimpleJavaLibraryBuilder.java:106)
	at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:54)
	at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:109)
	at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:117)
	at com.google.devtools.build.buildjar.BazelJavaBuilder.processRequest(BazelJavaBuilder.java:105)
	at com.google.devtools.build.buildjar.BazelJavaBuilder.runPersistentWorker(BazelJavaBuilder.java:67)
	at com.google.devtools.build.buildjar.BazelJavaBuilder.main(BazelJavaBuilder.java:45)
Target //java/com/google/gerrit/server:server failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 6.300s, Critical Path: 5.86s
INFO: 3 processes: 1 remote cache hit, 2 worker.
FAILED: Build did NOT complete successfully

@cushon
Copy link
Contributor

cushon commented Aug 24, 2018

I think we want to change [--host_javabase] and [--javabase].

--host_javabase=11 isn't tested or supported yet. It probably works, but the recommended configuration today is to use the default embedded JDK 10 as the --host_javabase. (That may be changing; we'll see.)

The part I expected to work was specifically using JDK 11 as a --javabase.

However, I would expect that the class version produced by the command above is major version 55. To my surprise, the major version is 52 (Java 8):

I can't reproduce. With gerrit @ 8f26af11797075aab14d9c17d1e69a30e00aa9e5:

$ bazel version
Build label: 0.17.0rc1
Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Wed Aug 22 18:43:25 2018 (1534963405)
Build timestamp: 1534963405
Build timestamp as int: 1534963405
$ bazel build --host_java_toolchain=@bazel_tools//tools/jdk:toolchain_java10 --java_toolchain=@bazel_tools//tools/jdk:toolchain_java10 //java/com/google/gerrit/server:libserver.jar
...
Target //java/com/google/gerrit/server:libserver.jar up-to-date:
  bazel-bin/java/com/google/gerrit/server/libserver.jar
$ javap -v -cp bazel-bin/java/com/google/gerrit/server/libs
erver.jar com.google.gerrit.server.util.SocketUtil | head
Classfile jar:file:///usr/local/google/home/cushon/src/gerrit/bazel-bin/java/com/google/gerrit/server/libserver.jar!/com/google/gerrit/server/util/SocketUtil.class
  Last modified Jan 1, 1980; size 3601 bytes
  MD5 checksum f62f6ea242e64c848c0f6723ff47e57a
  Compiled from "SocketUtil.java"
public final class com.google.gerrit.server.util.SocketUtil
  minor version: 0
  major version: 54

@cushon
Copy link
Contributor

cushon commented Aug 24, 2018

If I pass, --release 11 in my previous command, that I'm, getting error: release version 11 not supported. Can it be, that the JDK 11 doesn't actually used?

--release is a javac flag, javac is configured separately from the javabases as part of the java_toolchain, and the toolchain that ships with Bazel is using a JDK 10 javac. It will need to be updated to a JDK 11 javac to support --release 11. That's partly what this bug is tracking.

@davido
Copy link
Contributor

davido commented Aug 24, 2018

So yes, I can confirm that:

bazel build --host_java_toolchain=@bazel_tools//tools/jdk:toolchain_java10 --java_toolchain=@bazel_tools//tools/jdk:toolchain_java10 //java/com/google/gerrit/server:libserver.jar

works as expected and producing major version: 54.

My question was to make Java 11 work, as you pointed out, it should be supported with javabase option. How can I produce today major version: 55 with Bazel@HEAD?

This doesn't work and produces major version 52:

$ bazel build --javabase //:jdk11 --host_javabase //:jdk11 java/com/google/gerrit/server:server

@cushon
Copy link
Contributor

cushon commented Aug 24, 2018

How can I produce today major version: 55 with Bazel@HEAD?

That isn't currently supported with the default toolchain: it's is using a JDK 10 javac, which will need to be updated to a JDK 11 javac to support --release 11. Again, that is part of the work being tracked by this bug.

--javabase //:jdk11 --host_javabase //:jdk11 doesn't work and produces major version 52

That isn't supposed to produce v55 class files: the language level is configured by --java_toolchain, not --host_javabase or --javabase.

@davido
Copy link
Contributor

davido commented Aug 24, 2018

@cushon

That isn't currently supported with the default toolchain: it's is using a JDK 10 javac, which will need to be updated to a JDK 11 javac to support --release 11. Again, that is part of the work being tracked by this bug.

I see, thanks for clarifying. I see 2 fundamental problems with this approach:

  • Waiting indefinitely for Bazel to support new and shiny JDK versions
  • During upgrade to latest and greatest JDK in Bazel as default toolchain, we see breakages supporting older, but still new enough JDK versions, as discovered here. Yes, it is Java bug, but that doesn't change the fact, that there currently no way to build Gerrit with JDK9 on Bazel@HEAD, since it switch to JDK 10 as default toolchain. Strictly speaking this is a regression, as it was possible to build Gerrit with JDK9 toolchain in 0.16.0 and not any more in upcoming 0.17.0

That isn't supposed to produce v52 class files: the language level is configured by --java_toolchain, not --host_javabase or --javabase.

I guess you meant: "That isn't supposed to produce v55 class files", right?

Can't language level be configured by --host_javabase or --javabase instead of --java_toolchain? Or can --java_toolchain and --host_java_toolchain be easily used for custom JDKs (and provide copy/paste examples how to set it up and running)?

Do you have a working example how could I set-up custom toolchain with JDK11 to produce Gerrit with --release 11 (v55) with Bazel@HEAD?

@cushon
Copy link
Contributor

cushon commented Aug 24, 2018

there currently no way to build Gerrit with JDK9 on Bazel@HEAD

To confirm: do you want to build with the Java 9 language level and target JDK 9? Do you care about the --host_javabase version as long as it supports that language level?

Can't language level be configured by --host_javabase or

If you want to use whatever the latest supported language level is from the --host_javabase, you could use VanillaJavaBuilder (which should work with most host JDKs) and leave the source/target/bootclasspath unset so they default to the latest supported:

default_java_toolchain(
    name = "toolchain_vanilla",
    forcibly_disable_header_compilation = True,
    javabuilder = [":vanillajavabuilder"],
    jvm_opts = [],
)

e.g.

for v in $(seq 8 11); do
  bazel build \
    --host_javabase=:jdk${v} \
    --host_java_toolchain=@bazel_tools//tools/jdk:toolchain_vanilla \
    --java_toolchain=@bazel_tools//tools/jdk:toolchain_vanilla \
     //java/com/google/gerrit/server:libserver.jar
  javap -v -cp bazel-bin/java/com/google/gerrit/server/libserver.jar \
    com.google.gerrit.server.util.SocketUtil | grep "major version"
done
  major version: 52
  major version: 53
  major version: 54
  major version: 55

@davido
Copy link
Contributor

davido commented Aug 24, 2018

To confirm: do you want to build with the Java 9 language level and target JDK 9?

Yes.

Do you care about the --host_javabase version as long as it supports that language level?

No.

If you want to use whatever the latest supported language level is from the --host_javabase, you could use VanillaJavaBuilder [...]

In which case I wouldn't be able to use error prone and strict deps, and that's unfortunate.

cushon added a commit to cushon/bazel that referenced this issue Aug 24, 2018
Using --release to target versions that support modules and are not the
latest supported version (e.g. --release 9 on JDK 10) doesn't work:
https://bugs.openjdk.java.net/browse/JDK-8209865

Related: bazelbuild#5723
@cushon
Copy link
Contributor

cushon commented Aug 24, 2018

re: error prone and strict deps--I'm glad those features are useful, you may want to read this thread if you haven't already and share any feedback you have.

I had an idea for working around the bug with JDK 10 and --release 9: #5982

With that patch, the following works:

for v in $(seq 8 10); do
  bazel build --verbose_failures \
    --java_toolchain=@bazel_tools//tools/jdk:toolchain_java${v} \
    //java/com/google/gerrit/server:libserver.jar 
  javap -v -cp bazel-bin/java/com/google/gerrit/server/libserver.jar com.google.gerrit.server.util.SocketUtil | grep "major version"
done
  major version: 52
  major version: 53
  major version: 54

cushon added a commit to cushon/bazel that referenced this issue Aug 25, 2018
Using --release to target versions that support modules and are not the
latest supported version (e.g. --release 9 on JDK 10) doesn't work:
https://bugs.openjdk.java.net/browse/JDK-8209865

Related: bazelbuild#5723
davido added a commit to davido/bazel that referenced this issue Aug 25, 2018
To use the latest supported language level from the --host_javabase, use
VanillaJavaBuilder (which should work with most host JDKs) and leave the
source/target/bootclasspath unset so they default to the latest
supported versions.

With this in place, new java_runtime can be used, e.g.:

  java_runtime(
      name = "jdk11",
      java_home = "/usr/lib64/jvm/java-11",
      visibility = ["//visibility:public"],
  )

Now the JDK11 can be used with the language level 11:

  $ bazel build --host_javabase=:jdk11 \
    --host_java_toolchain=@bazel_tools//tools/jdk:toolchain_vanilla \
    --java_toolchain=@bazel_tools//tools/jdk:toolchain_vanilla \
    :a

Related: bazelbuild#5723
@davido
Copy link
Contributor

davido commented Aug 25, 2018

I had an idea for working around the bug with JDK 10 and --release 9: #5982

Thanks!

I tried to use vanilla java_toolchain as you suggested and it feels, like that case could be simplify and natively supported by Bazel: #5984. WDYT?

Meantime, I added vanilla java_toolchain to Gerrit build tool chain: [1] to be able to build with upcoming JDK 11. It turns out, that the build is failing in rules_closure: [2] and all tests are failing too. I'm not sure yet, why the acceptance tests are failing, all I see is this:

 $ bazel test -s --host_javabase=:jdk11 --host_java_toolchain=//:toolchain_vanilla --java_toolchain=//:toolchain_vanilla //javatests/com/google/gerrit/acceptance/ssh:ssh
[...]
  $ cat /home/davido/.cache/bazel/_bazel_davido/5c01f4f713b675540b8b424c5c647f63/execroot/gerrit/bazel-out/k8-fastbuild/testlogs/javatests/com/google/gerrit/acceptance/ssh/ssh/test.log
There were 36 failures:
1) withoutMessage(com.google.gerrit.acceptance.ssh.AbandonRestoreIT)

BazelTestRunner exiting with a return value of 1

It seems that the JUnit-Log is not complete, may be there is some Bazel/JUnit integration issues on JDK11 (and would worth its own issue)?

@cushon
Copy link
Contributor

cushon commented Aug 25, 2018

It seems that the JUnit-Log is not complete may be there is some Bazel/JUnit integration issues on JDK11 (and would worth its own issue)?

If you look at the test log (or use --test_summary=detailed) it's a guice/ASM issue:

1) Error injecting method, com.google.common.util.concurrent.UncheckedExecutionException: com.google.common.util.concurrent.UncheckedExecutionException: com.google.common.util.concurrent.UncheckedExecutionException: java.lang.UnsupportedOperationException
  at com.google.inject.assistedinject.FactoryProvider2.initialize(FactoryProvider2.java:716)
  at com.google.inject.assistedinject.FactoryModuleBuilder$1.configure(FactoryModuleBuilder.java:317) (via modules: com.google.gerrit.sshd.SshModule -> com.google.inject.assistedinject.FactoryModuleBuilder$1)
...
Caused by: java.lang.UnsupportedOperationException
        at org.objectweb.asm.ClassVisitor.visitNestHostExperimental(ClassVisitor.java:158)
        at org.objectweb.asm.ClassReader.accept(ClassReader.java:541)
        at org.objectweb.asm.ClassReader.accept(ClassReader.java:391)
        at com.google.inject.internal.util.LineNumbers.<init>(LineNumbers.java:64)

guice is using the ASM6 API level, and the nest-related attributes that were added in 11 are only supported with ASM7_EXPERIMENTAL: https://gitlab.ow2.org/asm/asm/blob/master/asm/src/main/java/org/objectweb/asm/ClassVisitor.java#L158

But in general, sure: any time you have a minimal repro that demonstrates there's a bug in Bazel, please report it.

@davido
Copy link
Contributor

davido commented Aug 26, 2018

If you look at the test log (or use --test_summary=detailed) it's a guice/ASM issue:[...]

Thanks for pointing this out. Somehow the full stack trace was reported on JDK10 for the same guice/ASM issue without providing this option: --test_summary=detailed. May be this is related to JavaBuilder vs. VanillaJavaBuilder difference?

Anyway, I fixed that issue in this PR: [1], conducted this new guice release: [2] with new artifacts and consumed them in this CL: [3]. Now gerrit can be built on JDK11.

Thanks for your help and big thank to the whole Bazel team! I think it's very impressive that such complex project as Gerrit (from build tool chain point of view) can be built on all currently released JDKs: 8,9,10 and even on the upcoming JDK11 ;-).

[1] google/guice#1203
[2] https://github.com/davido/guice/releases/tag/4.2.1
[3] https://gerrit-review.googlesource.com/c/gerrit/+/194040

@cushon
Copy link
Contributor

cushon commented Aug 26, 2018

Somehow the full stack trace was reported on JDK10 for the same guice/ASM issue without providing this option: --test_summary=detailed. May be this is related to JavaBuilder vs. VanillaJavaBuilder difference?

The choice of JavaBuilder implementation shouldn't affect the way test output is handled, and I can't observe a difference between the test output with JDK 10 and 11. Can you share the repro for that?

I think it's very impressive that such complex project as Gerrit (from build tool chain point of view) can be built on all currently released JDKs: 8,9,10 and even on the upcoming JDK11 ;-).

Thank you for testing with JDK 11 :) The feedback is helpful, and hopefully support for future JDK releases will go more smoothly as we get more practice with the new release cadence.

@davido
Copy link
Contributor

davido commented Aug 26, 2018

The choice of JavaBuilder implementation shouldn't affect the way test output is handled, and I can't observe a difference between the test output with JDK 10 and 11. Can you share the repro for that?

Yes, you are right. I cannot reproduce this any more. I tried on JDK 11 and reverted my Guice/ASM fix and now I see, in fact even without --test_summary=detailed all failure stacks for all tests are there: [1]. Something was wrong on my side.

@wcurrie
Copy link

wcurrie commented Nov 8, 2018

One (minor?) problem with the setup @uri-canva shows above. When compiling using java_hostbase pointed to jdk10 (or bazel packaged with nix to use jdk10) external libraries are missing from the .jdeps files written by StrictJavaDepsPlugin and DependencyModule. The only real problem this causes is the intellij bazel plugin fails to import external libraries (for modules that aren't in the working set). We have --strict_java_deps=OFF.

From debugging it looks like with jdk10 as java_hostbase during compilation

returns absolute Paths instead of relative ones. This breaks DependencyModule.buildDependenciesProto.

Example wcurrie/examples@13f8b7e based off java-maven in https://github.com/bazelbuild/examples. With jdk8 as java_hostbase guava is included in the .jdeps output. With jdk10 it's missing.

Maybe it's more practical to work around this issue at the intellij plugin level. We've done this internally.

With this hack of DependencyModule (wcurrie@452042c) I could get .jdeps files including external libraries and jdk 10 support.

@cushon
Copy link
Contributor

cushon commented Nov 8, 2018

absolute Paths instead of relative ones. This breaks DependencyModule.buildDependenciesProto.

The default javac has a fix for that issue:

google/error-prone-javac@9901a45

That's one of the reasons Strict Deps doesn't work with the 'vanilla' toolchain:

bazel/tools/jdk/BUILD

Lines 212 to 214 in 44cadaf

# It does not provider any of the following features:
# * Error Prone
# * Strict Java Deps

@wcurrie
Copy link

wcurrie commented Nov 15, 2018

Sorry for the newbie question but will the error prone fix (google/error-prone-javac@9901a45) eventually make its way into the javac bundled with bazel?

IIUC we could configure a toolchain pointed to a different java_compiler.jar. Eg reverting the revert 12dcd35. Might be quite the can of worms. I guess java 11 support in the error prone repo will appear once/if it becomes a priority upstream?

@cushon
Copy link
Contributor

cushon commented Nov 15, 2018

make its way into the javac bundled with bazel?

Yep. However it's blocked on #6316. The JDK 11 javac needs a JDK 11 --host_javabase, so in order to provide a JDK 11 javac without regressing support for earlier --host_javabase versions, we're planning to decouple the Java tools (including javac) from the Bazel distribution so you can select the appropriate version.

@benjaminp
Copy link
Collaborator

You can almost get Bazel itself to build wtih Java 11 using the vanilla toolchain. However, it fails because the vendored jarjar uses an old version of asm. Is there a plan update that?

@cushon
Copy link
Contributor

cushon commented Nov 16, 2018

@benjaminp Bazel at head is using ASM 7: f4a35eb

@benjaminp
Copy link
Collaborator

Right, the problem is jarjar:

$ bazel build --stamp src:bazel_nojdk --java_toolchain=@bazel_tools//tools/jdk:toolchain_vanilla --host_java_toolchain=@bazel_tools//tools/jdk:toolchain_vanilla
ERROR: src/java_tools/junitrunner/java/com/google/testing/coverage/BUILD:54:1: Executing genrule //src/java_tools/junitrunner/java/com/google/testing/coverage:Jacoco_jarjar failed (Exit 1) bash failed: error executing command /bin/bash -c ... (remaining 1 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
Exception in thread "main" java.lang.UnsupportedOperationException: This feature requires ASM7
	at org.objectweb.asm.ClassVisitor.visitNestHost(ClassVisitor.java:150)
	at org.objectweb.asm.ClassVisitor.visitNestHost(ClassVisitor.java:153)
	at org.objectweb.asm.commons.ClassRemapper.visitNestHost(ClassRemapper.java:185)
	at org.objectweb.asm.ClassReader.accept(ClassReader.java:541)
	at org.objectweb.asm.ClassReader.accept(ClassReader.java:391)
	at com.tonicsystems.jarjar.util.JarTransformer.process(JarTransformer.java:34)
	at com.tonicsystems.jarjar.util.JarProcessorChain.process(JarProcessorChain.java:36)
	at com.tonicsystems.jarjar.MainProcessor.process(MainProcessor.java:96)
	at com.tonicsystems.jarjar.util.StandaloneJarProcessor.run(StandaloneJarProcessor.java:44)
	at com.tonicsystems.jarjar.Main.process(Main.java:93)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.tonicsystems.jarjar.MainUtil.runMain(MainUtil.java:36)
	at com.tonicsystems.jarjar.Main.main(Main.java:49)
Target //src:bazel_nojdk failed to build

@cushon
Copy link
Contributor

cushon commented Nov 16, 2018

That error message is referring to the API level, not the library version: #6692

@lberki lberki added team-Rules-Java Issues for Java rules P2 We'll consider working on this in future. (Assignee optional) and removed category: rules > java labels Dec 3, 2018
@iirina iirina assigned iirina and unassigned lberki Jan 15, 2019
@uri-canva
Copy link
Contributor

Now that Oracle Java 8 is past EOL is there any update on this? I see there's quite a bit of progress on #6316, but is that the only thing that is blocking Java 11 support? Can we use Java 11 with --incompatible_use_remote_java_toolchain?

@lberki
Copy link
Contributor

lberki commented Feb 12, 2019

That's the state of the affairs, I think. The flag flip for #6316 should happen pretty soon (/cc @iirina ).

@tomikazi
Copy link

Does this mean 0.23.0? Our project community is eagerly waiting for being able to build against JDK11 using off-the-shelf Bazel.

@lberki
Copy link
Contributor

lberki commented Feb 13, 2019

0.24, unfortunately. The flag is not flipped in 0.23 by default :(

@cushon
Copy link
Contributor

cushon commented Feb 13, 2019

I think #6316 is necessary but not sufficient: it makes it possible to add support for the Java 11 language level to the toolchain without regressing JDK 8 host_javabase support (because we can provide different versions of the tools). But the work to add support for 11 hasn't happened yet, and I'm not sure it's in scope for #6316.

@softprops
Copy link

@greggdonovan
Copy link
Member

If built-in support for Java 11/12 is going to take more time, documentation on how to enable it yourself with toolchains and/or other configuration would be very helpful.

@davido
Copy link
Contributor

davido commented Apr 5, 2019

@greggdonovan It is already possible to support Java 11/12/... with recent Bazel releases with toolchain_vanilla. See my coment above: #5723 (comment)

This feature is also documented in Gerrit Code Review build toolchain documentation: [1] and there is even CI job in Bazel@CI for that.

[1] https://github.com/GerritCodeReview/gerrit/blob/master/Documentation/dev-bazel.txt#L17-L67

@cushon cushon removed their assignment May 1, 2019
@gmishkin gmishkin mentioned this issue May 22, 2019
3 tasks
@cushon cushon closed this as completed May 22, 2019
gergelyfabian added a commit to gergelyfabian/intellij that referenced this issue May 11, 2020
There is a bug in Bazel, that it does not produce proper jdeps files for
the IntelliJ plugin, thus breaking the existing API.

The only workaround seems to be to force the plugin to treat all files to
be part of the working set, and load external dependencies for them.

More info at:
bazelbuild#490
bazelbuild/bazel#5723
bazelbuild/bazel#6587
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P2 We'll consider working on this in future. (Assignee optional) team-Rules-Java Issues for Java rules type: feature request
Projects
None yet
Development

No branches or pull requests