Error build a project with the same module and submodule name #810

sergeygalkin · 2018-08-08T06:31:55Z

Plugin can't build project with module structure like this
basic
module/basic
module/submodule

Part of mvn compile -X -pl module/submodule jib:buildTar

> [DEBUG]     Snapshot dependencies:
> [DEBUG]         /usr/home/gals/.m2/repository/io/site/0.0.1-SNAPSHOT/basic-0.0.1-SNAPSHOT.jar
> [DEBUG]         /usr/home/gals/.m2/repository/io/site/module/basic/0.0.1-SNAPSHOT/basic-0.0.1-SNAPSHOT.jar
> [DEBUG]         /usr/home/gals/.m2/repository/io/site/module/submodule/0.0.1-SNAPSHOT/submodule-0.0.1-SNAPSHOT.jar

Trace output is

> [ERROR] Failed to execute goal com.google.cloud.tools:jib-maven-plugin:0.9.8:buildTar (default-cli) on project eth: Building image tarball failed: entry 'app/libs//basic-0.0.1-SNAPSHOT.jar' closed at '5340' before the '15583' bytes specified in the header were written -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal com.google.cloud.tools:jib-maven-plugin:0.9.8:buildTar (default-cli) on project eth: Building image tarball failed
> 	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
> 	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
> 	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
> 	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
> 	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
> 	at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
> 	at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
> 	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
> 	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
> 	at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
> 	at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
> 	at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
> 	at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:498)
> 	at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
> 	at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
> 	at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
> 	at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
> Caused by: org.apache.maven.plugin.MojoExecutionException: Building image tarball failed
> 	at com.google.cloud.tools.jib.maven.BuildTarMojo.execute(BuildTarMojo.java:148)
> 	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
> 	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
> 	... 20 more
> Caused by: java.io.IOException: entry 'app/libs//basic-0.0.1-SNAPSHOT.jar' closed at '5340' before the '15583' bytes specified in the header were written
> 	at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.closeArchiveEntry(TarArchiveOutputStream.java:419)
> 	at com.google.cloud.tools.jib.tar.TarStreamBuilder.writeEntriesAsTarArchive(TarStreamBuilder.java:52)
> 	at com.google.cloud.tools.jib.blob.WriterBlob.writeTo(WriterBlob.java:36)
> 	at com.google.cloud.tools.jib.cache.CacheWriter.writeLayer(CacheWriter.java:73)
> 	at com.google.cloud.tools.jib.builder.steps.BuildAndCacheApplicationLayerStep.call(BuildAndCacheApplicationLayerStep.java:121)
> 	at com.google.cloud.tools.jib.builder.steps.BuildAndCacheApplicationLayerStep.call(BuildAndCacheApplicationLayerStep.java:37)
> 	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:127)
> 	at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
> 	at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:80)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> 	at java.lang.Thread.run(Thread.java:748)
> 	Suppressed: java.io.IOException: This archive contains unclosed entries.
> 		at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.finish(TarArchiveOutputStream.java:289)
> 		at org.apache.commons.compress.archivers.tar.TarArchiveOutputStream.close(TarArchiveOutputStream.java:306)
> 		at com.google.cloud.tools.jib.tar.TarStreamBuilder.writeEntriesAsTarArchive(TarStreamBuilder.java:54)
> 		... 10 more

workaround is rename module/basic

The text was updated successfully, but these errors were encountered:

coollog · 2018-08-10T14:01:09Z

Hi @sergeygalkin , thanks for reporting this issue! It looks like this is a consequence of Jib placing all the dependency JARs in the /app/libs directory, so if two dependency JARs had the same name (basic-0.0.1-SNAPSHOT.jar), this failure will occur. I think the solution is to rename dependency JARs to include their group IDs as well when Jib adds the entries for the JARs.

coollog · 2018-11-20T15:00:18Z

The issue in Gradle is that the dependencies are only resolved with their file path, and not with their groupId like in Maven. Therefore, we cannot easily get this groupId in Gradle without performing logic on the file path.

We should probably keep the logic for Maven/Gradle the same in terms of generating the filename for the dependency jar in the container image. Therefore, there are a few strategies:

name the JAR with a unique ID - /app/libs/e22k1nckie.jar
name the JAR with its original name plus a unique ID - /app/libs/basic-0.0.1-SNAPSHOT-gw2nck3.jar
name only duplicate JAR names with a unique ID - second /app/libs/basic-0.0.1-SNAPSHOT.jar becomes /app/libs/e22k1nckie.jar

Thoughts? @GoogleContainerTools/java-tools

chanseokoh · 2018-11-20T16:24:14Z

The first option would be the simplest and has a merit, but I think retaining the original name outweighs the benefit of the simple implementation here for debugging purposes when something goes wrong. The option 2 seems to work, but I wonder how long the unique ID would be.

coollog · 2018-11-20T19:03:35Z

I think with option 3, we could even number further occurrences such that the second one becomes /app/libs/basic-0.0.1-SNAPSHOT-2.jar, for example.

coollog · 2018-11-20T19:04:46Z

With option 2, the unique ID can also just be a counter, so the length would just be log number of jars.

chanseokoh · 2018-11-20T19:13:37Z

I've thought about using numbers. I'd still add some special, discernible prefix, like -renamed-2, so that it doesn't look like part of the semver versioning (e.g., -2 in 0.1.0-2 might look like the actual version of the dep). I vote for option 3 with a simple scheme, but if detecting dup seems unnecessarily complex, option 2 might be better.

coollog · 2018-11-20T20:03:15Z

Yea, detecting dup shouldn't be too complex since all the dependency jars are processed in the same code loop.

loosebazooka · 2018-11-20T21:29:08Z

I dunno, I mean the final jar artifact and the maven coordinates are two different things. In the end they are filenames that would be included in a jar referencing it's classpath through metadata in the same way.

Class-Path: one.jar two.jar

Or in war as files in WEB-INF/lib.

It really seems like the user should not be reusing names. And if they do it's unexpected behavior. I'm curious what the war plugin in gradle/maven does?

chanseokoh · 2018-11-20T21:41:09Z

The problem is that, the filename of a dependency JAR is basically the artifact ID of its Maven coordinates. For example, you can have multiple dependency JAR files with the same filename. (BTW, one should not include both javax.servlet.jstl:jstl and jstl:jstl.)

.m2/repository/javax/servlet/jstl/1.2/jstl-1.2.jar
.m2/repository/jstl/jstl/1.2/jstl-1.2.jar

What we currently do is to copy those jstl-1.2.jar into the same location, /app/libs, so one file ends up overwriting the other.

This is OK in WAR, since the group ID is appended.

$ ls WEB-INF/lib/
javax.servlet-jstl-1.2.jar  jstl-jstl-1.2.jar

It really seems like the user should not be reusing names

I think it is very possible that different dependencies may have the same artifact ID, which is OK because they will have different group IDs.

loosebazooka · 2018-11-20T21:44:29Z

This is OK in WAR, since the group ID is appended.

You're right, I seemed to have missed this. I guess we should solve it. Although I wonder if resolving the jars is deterministic in the build system? Should they be sorted by size?

coollog · 2018-12-19T20:29:22Z

Hi @sergeygalkin , we've released v1.0.0-rc1 with a fix for this issue.

sergeygalkin · 2018-12-20T08:02:47Z

@coollog A lot of thanks ! great job

coollog added this to the v0.9.10 milestone Aug 10, 2018

coollog modified the milestones: v0.9.10, v0.9.11 Aug 29, 2018

coollog added the priority :p0 label Aug 31, 2018

coollog mentioned this issue Sep 7, 2018

Change LayerEntry to be more like TarArchiveEntry. #956

Closed

coollog modified the milestones: v0.9.11, v0.10.0 Sep 17, 2018

coollog modified the milestones: v0.10.0, v0.11.0 Oct 2, 2018

coollog modified the milestones: v1.1.0, v1.0.0 Nov 9, 2018

TadCordle self-assigned this Dec 11, 2018

TadCordle mentioned this issue Dec 12, 2018

Add file size to dependency filenames to avoid filename collisions #1338

Merged

TadCordle closed this as completed in #1338 Dec 14, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Error build a project with the same module and submodule name #810

Error build a project with the same module and submodule name #810

sergeygalkin commented Aug 8, 2018 •

edited by chanseokoh

Loading

coollog commented Aug 10, 2018

coollog commented Nov 20, 2018

chanseokoh commented Nov 20, 2018

coollog commented Nov 20, 2018

coollog commented Nov 20, 2018

chanseokoh commented Nov 20, 2018

coollog commented Nov 20, 2018

loosebazooka commented Nov 20, 2018 •

edited

Loading

chanseokoh commented Nov 20, 2018 •

edited

Loading

loosebazooka commented Nov 20, 2018 •

edited

Loading

coollog commented Dec 19, 2018

sergeygalkin commented Dec 20, 2018

Error build a project with the same module and submodule name #810

Error build a project with the same module and submodule name #810

Comments

sergeygalkin commented Aug 8, 2018 • edited by chanseokoh Loading

coollog commented Aug 10, 2018

coollog commented Nov 20, 2018

chanseokoh commented Nov 20, 2018

coollog commented Nov 20, 2018

coollog commented Nov 20, 2018

chanseokoh commented Nov 20, 2018

coollog commented Nov 20, 2018

loosebazooka commented Nov 20, 2018 • edited Loading

chanseokoh commented Nov 20, 2018 • edited Loading

loosebazooka commented Nov 20, 2018 • edited Loading

coollog commented Dec 19, 2018

sergeygalkin commented Dec 20, 2018

sergeygalkin commented Aug 8, 2018 •

edited by chanseokoh

Loading

loosebazooka commented Nov 20, 2018 •

edited

Loading

chanseokoh commented Nov 20, 2018 •

edited

Loading

loosebazooka commented Nov 20, 2018 •

edited

Loading