Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] Release build of v22.04.0 FAILED on "Execution attach-javadoc failed: NullPointerException" with maven option '-P source-javadoc' #5050

Closed
NvTimLiu opened this issue Mar 25, 2022 · 3 comments
Closed

[BUG] Release build of v22.04.0 FAILED on "Execution attach-javadoc failed: NullPointerException" with maven option '-P source-javadoc' #5050

NvTimLiu opened this issue Mar 25, 2022 · 3 comments
Assignees
@NvTimLiu
Labels
bug Something isn't working build Related to CI / CD or cleanly building P0 Must have for release

Comments

@NvTimLiu
Copy link
Collaborator

Describe the bug
Release build of v22.04.0 FAILED with maven option '-P source-javadoc' as below
CLI:

mvn clean package -pl '!tools' -P source-javadoc -Dcuda.version=cuda11 -Dbuildver=311 -DskipTests=false

BUILD ERRORS:

[2022-03-25T11:08:36.021Z] [INFO] 
[2022-03-25T11:08:36.021Z] [INFO] RAPIDS Accelerator for Apache Spark Root Project ... SUCCESS [01:20 min]
[2022-03-25T11:08:36.021Z] [INFO] RAPIDS Accelerator for Apache Spark Common ......... SUCCESS [ 32.771 s]
[2022-03-25T11:08:36.021Z] [INFO] RAPIDS Accelerator for Apache Spark SQL Plugin ..... FAILURE [04:53 min]
[2022-03-25T11:08:36.021Z] [INFO] RAPIDS Accelerator for Apache Spark Shuffle Plugin . SKIPPED
[2022-03-25T11:08:36.021Z] [INFO] RAPIDS Accelerator for Apache Spark Scala UDF Plugin SKIPPED
[2022-03-25T11:08:36.021Z] [INFO] RAPIDS Accelerator for Apache Spark Aggregator ..... SKIPPED
[2022-03-25T11:08:36.021Z] [INFO] RAPIDS Accelerator for Apache Spark Distribution ... SKIPPED
[2022-03-25T11:08:36.021Z] [INFO] rapids-4-spark-integration-tests_2.12 .............. SKIPPED
[2022-03-25T11:08:36.021Z] [INFO] RAPIDS Accelerator for Apache Spark Tests .......... SKIPPED
[2022-03-25T11:08:36.021Z] [INFO] RAPIDS Accelerator for Apache Spark Tests For 3.1.X+ SKIPPED
[2022-03-25T11:08:36.021Z] [INFO] ------------------------------------------------------------------------
[2022-03-25T11:08:36.021Z] [INFO] BUILD FAILURE
[2022-03-25T11:08:36.021Z] [INFO] ------------------------------------------------------------------------
[2022-03-25T11:08:36.021Z] [INFO] Total time:  06:47 min
[2022-03-25T11:08:36.021Z] [INFO] Finished at: 2022-03-25T11:08:33Z
[2022-03-25T11:08:36.021Z] [INFO] ------------------------------------------------------------------------
[2022-03-25T11:08:36.021Z] [ERROR] Failed to execute goal org.apache.maven.plugins:maven-javadoc-plugin:3.0.0:jar (attach-javadoc) on project rapids-4-spark-sql_2.12: Execution attach-javadoc of goal org.apache.maven.plugins:maven-javadoc-plugin:3.0.0:jar failed. NullPointerException -> [Help 1]
[2022-03-25T11:08:36.021Z] [ERROR] 
[2022-03-25T11:08:36.021Z] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[2022-03-25T11:08:36.021Z] [ERROR] Re-run Maven using the -X switch to enable full debug logging.
[2022-03-25T11:08:36.021Z] [ERROR] 
[2022-03-25T11:08:36.021Z] [ERROR] For more information about the errors and possible solutions, please read the following articles:
[2022-03-25T11:08:36.021Z] [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
[2022-03-25T11:08:36.021Z] [ERROR] 
[2022-03-25T11:08:36.021Z] [ERROR] After correcting the problems, you can resume the build with the command
[2022-03-25T11:08:36.021Z] [ERROR]   mvn <goals> -rf :rapids-4-spark-sql_2.12
script returned exit code 1

Steps/Code to reproduce bug

  • Run build CLI: mvn clean package -pl '!tools' -P source-javadoc -Dcuda.version=cuda11 -Dbuildver=311 -DskipTests=false

Expected behavior
Build get PASS, and sources.jar/javadoc.jar can be generated along with spark-rapids artifacts

Environment details (please complete the following information)

  • Env that can maven build CLI

Additional context
OSS requires sources.jar/javadoc.jar along with spark-rapids artifacts for the pubic release,

so we weed run maven with option -P source-javadoc to generate sources.jar/javadoc.jar files.
@NvTimLiu NvTimLiu added bug Something isn't working ? - Needs Triage Need team to review and classify build Related to CI / CD or cleanly building labels Mar 25, 2022
@jlowe
Copy link
Contributor

jlowe commented Mar 25, 2022

Is JAVA_HOME set in the environment when this command runs? I was able to reproduce it, and adding -X to the Maven command-line reveals this additional information about the failure:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-javadoc-plugin:3.0.0:jar (attach-javadoc) on project rapids-4-spark-sql_2.12: Execution attach-javadoc of goal org.apache.maven.plugins:maven-javadoc-plugin:3.0.0:jar failed.: NullPointerException -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-javadoc-plugin:3.0.0:jar (attach-javadoc) on project rapids-4-spark-sql_2.12: Execution attach-javadoc of goal org.apache.maven.plugins:maven-javadoc-plugin:3.0.0:jar failed.
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: org.apache.maven.plugin.PluginExecutionException: Execution attach-javadoc of goal org.apache.maven.plugins:maven-javadoc-plugin:3.0.0:jar failed.
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:148)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: java.lang.NullPointerException
    at java.io.File.<init> (File.java:279)
    at org.apache.maven.plugins.javadoc.JavadocUtil.getJavaHome (JavadocUtil.java:1302)
    at org.apache.maven.plugins.javadoc.JavadocUtil.invokeMaven (JavadocUtil.java:941)
    at org.apache.maven.plugins.javadoc.AbstractJavadocMojo.getModulesLinks (AbstractJavadocMojo.java:5620)
    at org.apache.maven.plugins.javadoc.AbstractJavadocMojo.addLinkofflineArguments (AbstractJavadocMojo.java:4026)
    at org.apache.maven.plugins.javadoc.AbstractJavadocMojo.addStandardDocletOptions (AbstractJavadocMojo.java:4916)
    at org.apache.maven.plugins.javadoc.AbstractJavadocMojo.executeReport (AbstractJavadocMojo.java:2074)
    at org.apache.maven.plugins.javadoc.JavadocJar.doExecute (JavadocJar.java:190)
    at org.apache.maven.plugins.javadoc.AbstractJavadocMojo.execute (AbstractJavadocMojo.java:1909)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)

exporting JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 fixed it for me.

@sameerz sameerz added P0 Must have for release and removed ? - Needs Triage Need team to review and classify labels Mar 25, 2022
@NvTimLiu
Copy link
Collaborator Author

NvTimLiu commented Mar 28, 2022 •
edited
Loading

Thanks @jlowe exporting JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 works for me.
I'll add JAVA_HOME in the docker environment to fix the issue.

I got another failure as below after adding -P source-javadoc , debugging it.

[INFO] --- maven-javadoc-plugin:3.0.0:jar (attach-javadoc) @ rapids-4-spark-integration-tests_2.12 ---
[INFO] The goal 'org.apache.maven.plugins:maven-javadoc-plugin:3.0.0:javadoc' has not been previously called for the module:
 'com.nvidia:rapids-4-spark_2.12:jar:22.04.0'. Trying to invoke it...
[WARNING] Creating fake javadoc directory to prevent repeated invocations: /spark-rapids/dist/target/apidocs
[ERROR] Error fetching link: /spark-rapids/dist/target/apidocs/package-list. Ignored it.

.....

[2022-03-27T07:53:23.715Z] + mvn -B install:install-file -s jenkins/settings.xml -P mirror-apache-to-urm,source-javadoc -Dcuda.version=cuda11 -Dmaven.repo.local=/home/jenkins/agent/workspace/jenkins-tim-rapids-release-11/.m2 -Dfile=dist/target/rapids-4-spark_2.12-22.04.0.jar -DpomFile=dist/target/extra-resources/META-INF/maven/com.nvidia/rapids-4-spark_2.12/pom.xml -DgroupId=com.nvidia -DartifactId=rapids-4-spark_2.12 -Dversion=22.04.0 -Dpackaging=jar
[2022-03-27T07:53:25.084Z] [INFO] Scanning for projects...
[2022-03-27T07:53:25.341Z] [INFO] 
[2022-03-27T07:53:25.341Z] [INFO] ------------------< com.nvidia:rapids-4-spark-parent >------------------
[2022-03-27T07:53:25.341Z] [INFO] Building RAPIDS Accelerator for Apache Spark Root Project 22.04.0
[2022-03-27T07:53:25.341Z] [INFO] --------------------------------[ pom ]---------------------------------
[2022-03-27T07:53:25.341Z] [INFO] 
[2022-03-27T07:53:25.341Z] [INFO] --- maven-install-plugin:2.4:install-file (default-cli) @ rapids-4-spark-parent ---
[2022-03-27T07:53:25.906Z] [ERROR] The specified file '/home/jenkins/agent/workspace/jenkins-tim-rapids-release-11/dist/target/rapids-4-spark_2.12-22.04.0.jar' not exists

@NvTimLiu
Copy link
Collaborator Author

NvTimLiu commented Mar 28, 2022 •
edited
Loading

As javadoc of rapids-4-spark-integration-tests_2.12 has dependency to com.nvidia:rapids-4-spark-integration-tests_2.12 as below:

[INFO] --- maven-javadoc-plugin:3.3.0:jar (attach-javadoc) @ rapids-4-spark-integration-tests_2.12 ---

[DEBUG]   (f) reactorProjects = [MavenProject: com.nvidia:rapids-4-spark-parent:22.04.0 @ /spark-rapids/pom.xml, MavenProject: com.nvidia:rapids-4-spark_2.12:22.04.0 @ /spark-rapids/dist/pom.xml, 
MavenProject: com.nvidia:rapids-4-spark-integration-tests_2.12:22.04.0 @ /spark-rapids/integration_tests/pom.xml]

rapids-4-spark_2.12 would NOT generate javadoc.jar, because it does not have dist/src/main/java dir,

root@d03af8a1b975:/spark-rapids# ls dist/target/*.jar -l
-rw-r--r-- 1 root root     5006 Mar 28 07:28 dist/target/rapids-4-spark_2.12-22.04.0-sources.jar
-rw-r--r-- 1 root root 12627126 Mar 28 07:28 dist/target/rapids-4-spark_2.12-22.04.0.jar

so (attach-javadoc) @ rapids-4-spark-integration-tests_2.12 would FAILED as below, when it tries to deleted offline link of rapids-4-spark by default detectOfflineLinks=true

[INFO] --- maven-javadoc-plugin:3.0.0:jar (attach-javadoc) @ rapids-4-spark-integration-tests_2.12 ---
[INFO] The goal 'org.apache.maven.plugins:maven-javadoc-plugin:3.0.0:javadoc' has not been previously called for the module: 
'com.nvidia:rapids-4-spark_2.12:jar:22.04.0'. Trying to invoke it...
[WARNING] Creating fake javadoc directory to prevent repeated invocations: /spark-rapids/dist/target/apidocs
[ERROR] Error fetching link: /spark-rapids/dist/target/apidocs/package-list. Ignored it.

Invoking javadoc on rapids-4-spark_2.12 removes all its jars, but failed to generate them again.

Then mvn CLI: would failed with no jars rapids-4-spark_2.12

https://docs.github.com/github/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax)
[ERROR] The specified file '/home/jenkins/agent/workspace/jenkins-tim-rapids-release-11/dist/target/rapids-4-spark_2.12-22.04.0.jar' not exists

To fix: Add configuration <detectOfflineLinks=false> for maven-javadoc-plugin: Not detecting the links for all modules defined in the project: https://maven.apache.org/plugins/maven-javadoc-plugin/javadoc-no-fork-mojo.html#detectOfflineLinks

NvTimLiu added a commit to NvTimLiu/spark-rapids that referenced this issue Mar 28, 2022
@NvTimLiu
Update for the public release
816d4e2 
To fix issue NVIDIA#5050

1, Add JAVA_HOME for maven opt `-P source-javadoc`

2, Change to use maven opt `-P release311` instead of `-P release301`

3, Add configuration `<detectOfflineLinks>` for maven-javadoc-plugin: Not detecting the links for all modules defined in the project: https://maven.apache.org/plugins/maven-javadoc-plugin/javadoc-no-fork-mojo.html#detectOfflineLinks
    To [ERROR] `The specified file '/home/jenkins/agent/workspace/jenkins-tim-rapids-release-11/dist/target/rapids-4-spark_2.12-22.04.0.jar' not exists`

Signed-off-by: Tim Liu <timl@nvidia.com>
@NvTimLiu NvTimLiu mentioned this issue Mar 28, 2022
Merged
pxLi pushed a commit that referenced this issue Apr 1, 2022
@NvTimLiu
Update for the public release [skip ci] (#5069)
679721e 
* Update for the public release

To fix issue #5050

1, Add JAVA_HOME for maven opt `-P source-javadoc`

2, Change to use maven opt `-P release311` instead of `-P release301`

3, Add configuration `<detectOfflineLinks>` for maven-javadoc-plugin: Not detecting the links for all modules defined in the project: https://maven.apache.org/plugins/maven-javadoc-plugin/javadoc-no-fork-mojo.html#detectOfflineLinks
    To [ERROR] `The specified file '/home/jenkins/agent/workspace/jenkins-tim-rapids-release-11/dist/target/rapids-4-spark_2.12-22.04.0.jar' not exists`

Signed-off-by: Tim Liu <timl@nvidia.com>

* Update 'integration-tests/pom.xml' to keep links for all modules defined in the project

Signed-off-by: Tim Liu <timl@nvidia.com>
@NvTimLiu NvTimLiu closed this as completed Apr 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@NvTimLiu NvTimLiu

Labels
bug Something isn't working build Related to CI / CD or cleanly building P0 Must have for release
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jlowe @sameerz @NvTimLiu