dokkaJavadocJar fires "java.lang.OutOfMemoryError: Metaspace"

[elect86]

Question

I added the following in my build.gradle.kts

val dokkaJavadocJar by tasks.register<Jar>("dokkaJavadocJar") {
    dependsOn(tasks.dokkaJavadoc)
    from(tasks.dokkaJavadoc.get().outputDirectory.get())
    archiveClassifier.set("javadoc")
}

val dokkaHtmlJar by tasks.register<Jar>("dokkaHtmlJar") {
    dependsOn(tasks.dokkaHtml)
    from(tasks.dokkaHtml.get().outputDirectory.get())
    archiveClassifier.set("html-doc")
}

val sourceJar = task("sourceJar", Jar::class) {
    dependsOn(tasks["classes"])
    archiveClassifier.set("sources")
    from(sourceSets.main.get().allSource)
}

artifacts {
    archives(dokkaJavadocJar)
    archives(dokkaHtmlJar)
    archives(sourceJar)
}

However dokkaJavadocJar produces the titled error:

java.lang.OutOfMemoryError: Metaspace

and the following

Task :compileJava FAILED

But I'm pretty sure is dokkaJavadocJar the error root because if I comment it out, it works flawless

In order to fix this, I added in gradle.properties

org.gradle.jvmargs=-XX:MaxMetaspaceSize=512m

Installation

• Operating system: macOS/Windows/Linux
• Build tool: Gradle 6.6
• Dokka version: 1.4.0-dev-62

I wonder if this is "fine" or something nasty is happening in the background I shall be aware of

Ps: the very same problem happens also in other projects

[Kordyjan]

This is really bizarre. Dokka is indeed memory-intensive in some parts, which are not yet optimized, but it should only affect heap space. I don't know what could cause overflow of metaspace. We will definitely profile that.

[Kordyjan]

Also thanks for suggesting a workaround. I'll repost it for the better visibility.

Workaround:

put org.gradle.jvmargs=-XX:MaxMetaspaceSize=512m in gradle.properties

[Kordyjan]

Could you check if this is reproducible if you run ./gradlew --stop before running any tasks or even put org.gradle.daemon=false in your gradle.properties?

[gabrielittner]

If you need a project to reproduce it happens in the integration tests of our Gradle plugin which runs dokka.

The commit vanniktech/gradle-maven-publish-plugin@3ef0227
The ci run https://travis-ci.org/github/vanniktech/gradle-maven-publish-plugin/builds/723577862 (also happens locally when running ./gradlew integrationTest)

[elect86]

Could you check if this is reproducible if you run ./gradlew --stop before running any tasks or even put org.gradle.daemon=false in your gradle.properties?

Both seem working

[elect86]

What does this have to do with the daemon, @Kordyjan?

[elect86]

Just got again java.lang.OutOfMemoryError: Metaspace with org.gradle.daemon=false

[altavir]

I get the same error for dokkaHtml for a not-so-large multi-module project. And yes, it is reproducible. Enlarging metaspace seems to be working.

[gciatto]

I'm getting the same problem here, with or without the daemon.
I'm working on a multi-platform & multi-project project.
Here's the branch I'm working on: https://gitlab.com/pika-lab/tuprolog/2p-in-kotlin/-/commits/feature/dokka-1.4

[joffrey-bion]

In my case I had to bump the metaspace even further (1024m).
I have a multi-module project with 8 modules, half of which are multiplatform.
Note that I'm running dokkaHtml, not dokkaJavadoc, but I get the same error with javadoc.

[ansman]

I believe one reason for this is parallel execution which spawns multiple processes which fight over the same resource. Running with the --no-parallel seems to help, at least for our project.

[ansman]

Although --no-parallel gets further (as in more modules get completes), eventually I do hit the same error. This could indicate that this is caused by a memory leak.

512MB is not enough for my project, I also had to bump to 1024. I wouldn't be surprised you need even more if you have more modules

[ansman]

Any updates on this? It's turning out to be a pretty massive issue because we're having trouble publishing our library that has lots of modules. Happy to spend some time debugging if someone can point me in the right direction.

[ansman]

This issue is even more problematic since the Gradle Daemon doesn't die but rather hangs so you have to manually kill it. The error in question something like this:

  Process exited with code 1
    Gradle failure report
    FAILURE: Build failed with an exception.

    * What went wrong:
    Gradle could not start your build.
    > Could not create service of type ResourceSnapshotterCacheService using GradleUserHomeServices.createResourceSnapshotterCacheService().
       > Timeout waiting to lock file hash cache (/Users/user/.gradle/caches/6.6/fileHashes). It is currently in use by another Gradle instance.
         Owner PID: 71496
         Our PID: 71896
         Owner Operation:
         Our operation:
         Lock file: /Users/user/.gradle/caches/6.6/fileHashes/fileHashes.lock

[cohenor]

This issue is even more problematic since the Gradle Daemon doesn't die but rather hangs so you have to manually kill it. The error in question something like this:

  Process exited with code 1
    Gradle failure report
    FAILURE: Build failed with an exception.

    * What went wrong:
    Gradle could not start your build.
    > Could not create service of type ResourceSnapshotterCacheService using GradleUserHomeServices.createResourceSnapshotterCacheService().
       > Timeout waiting to lock file hash cache (/Users/user/.gradle/caches/6.6/fileHashes). It is currently in use by another Gradle instance.
         Owner PID: 71496
         Our PID: 71896
         Owner Operation:
         Our operation:
         Lock file: /Users/user/.gradle/caches/6.6/fileHashes/fileHashes.lock

I'm having the exact same experience in a rather small multi-module project. Already tried to increase metaspace and to disable parallelism but it didn't provide any deterministic results.
Any updates on this? 🙏

[willbuck]

I am also experiencing the same issue in https://github.com/micronaut-projects/micronaut-kotlin/tree/issue-78

Running --no-parallel with org.gradle.daemon=false and org.gradle.jvmargs=-XX:MaxMetaspaceSize=2g does seem to solve the problem for a clean build (though I do still see these bizarre warnings, see below) but that isn't really a practical solve for our github actions to publish our documentation.

Definitely seems a memory leak is the culprit, please let us know how profiling that bit goes!

WARN: error in opening zip file: /home/will/.gradle/caches/modules-2/files-2.1/org.graalvm.nativeimage/svm-hosted-native-linux-amd64/20.2.0/55f876b0688181a47436128f0e06b9ab93e17a32/svm-hosted-native-linux-amd64-20.2.0.tar.gz
java.util.zip.ZipException: error in opening zip file

WARN: error in opening zip file: /home/will/.gradle/caches/modules-2/files-2.1/org.graalvm.truffle/truffle-nfi-native-darwin-amd64/20.2.0/d3ad1011cba788a2b36792ee2ea128b90d5a4f8e/truffle-nfi-native-darwin-amd64-20.2.0.tar.gz
java.util.zip.ZipException: error in opening zip file

[asarkar]

Frequent OOM error in a fairly small project.

> Task :okgrpc-cli:dokkaHtml
Dokka 1.4.* is an alpha project
Initializing plugins
Validity check
Creating documentation models

Exception in thread "DefaultDispatcher-worker-8" java.lang.OutOfMemoryError: Metaspace
Daemon will be stopped at the end of the build after running out of JVM memory
Exception in thread "DefaultDispatcher-worker-1" java.lang.OutOfMemoryError: Metaspace
Exception in thread "DefaultDispatcher-worker-3" java.lang.OutOfMemoryError: Metaspace

Since the usual first response from any OOS maintainer is "have you tried the latest version", I'm running v1.4.10.2.

@willbuck A Javadoc engine shouldn't need 2G RAM to generate a dozen or so HTML files (that's how many for me).

[ileasile]

Mee too, pretty small project: https://github.com/Kotlin/kotlin-jupyter/tree/notebook-api
Getting the error only on CI, locally I had org.gradle.jvmargs=-Xmx2048m, and it worked well.

[MarvinSchramm]

I have the same issue, simple multimodule object.
This prevents me from publishing the libraries with javadoc at the moment

The gradle properties helps locally, but my CI system is github actions, i dont have 2G available there

[ansman]

Today I saw some new errors related to this:

...
java.lang.OutOfMemoryError: Metaspace
WARN: Could not read file: /Users/nicklas/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib/1.4.20/9be77b243a362b745e365f286627b8724337009c/kotlin-stdlib-1.4.20.jar!/kotlin/SuspendKt.class; size in bytes: 1108; file type: CLASS
java.lang.OutOfMemoryError: Metaspace
WARN: Could not read file: /Users/nicklas/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib/1.4.20/9be77b243a362b745e365f286627b8724337009c/kotlin-stdlib-1.4.20.jar!/kotlin/ThrowsKt.class; size in bytes: 540; file type: CLASS
java.lang.OutOfMemoryError: Metaspace
WARN: Could not read file: /Users/nicklas/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib/1.4.20/9be77b243a362b745e365f286627b8724337009c/kotlin-stdlib-1.4.20.jar!/kotlin/TuplesKt.class; size in bytes: 1904; file type: CLASS
java.lang.OutOfMemoryError: Metaspace

This time it actually shows some more info on when you get OOM. It seems to be while reading the metadata/class files for the kotlin stdlib in my case.

This happened while generating gfm docs specifically.

[AlexisRiksman-TomTom]

In our project the problem disappeared after upgrading to Dokka 1.6.10.
I guess this ticket can be closed now.

[GlenKPeterson]

Just got again this using dokka 1.6.21 with Kotlin 1.7.0.

$ gradle --version

------------------------------------------------------------
Gradle 7.4.2
------------------------------------------------------------

Build time:   2022-03-31 15:25:29 UTC
Revision:     540473b8118064efcc264694cbcaa4b677f61041

Kotlin:       1.5.31
Groovy:       3.0.9
Ant:          Apache Ant(TM) version 1.10.11 compiled on July 10 2021
JVM:          11.0.15 (Private Build 11.0.15+10-Ubuntu-0ubuntu0.18.04.1)
OS:           Linux 4.15.0-184-generic amd64

$ gradle --no-daemon clean build publish publishToMavenLocal
To honour the JVM settings for this build a single-use Daemon process will be forked. See https://docs.gradle.org/7.4.2/userguide/gradle_daemon.html#sec:disabling_the_daemon.
Daemon will be stopped at the end of the build 

...

> Task :org-organicdesign:test-utils-basic:dokkaJavadoc
Initializing plugins
Dokka is performing: documentation for test-utils-basic
Validity check
Creating documentation models

WARN: Attempt to load key 'java.correct.class.type.by.place.resolve.scope' for not yet loaded registry

> Task :org-organicdesign:test-utils-basic:dokkaJavadoc


> Task :org-organicdesign:test-utils-basic:dokkaJavadoc FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':org-organicdesign:test-utils-basic:dokkaJavadoc'.
> Metaspace

If I build again without making any changes, it usually works. I have org.gradle.jvmargs="-XX:MaxMetaspaceSize=1024m" in my root project gradle.properties.kts.

[GlenKPeterson]

Just failed again, differently:

> Task :org-organicdesign:test-utils-http:dokkaJavadoc
Initializing plugins
Dokka is performing: documentation for test-utils-http
Validity check
Creating documentation models

WARN: Attempt to load key 'java.correct.class.type.by.place.resolve.scope' for not yet loaded registry
WARN: Attempt to load key 'java.correct.class.type.by.place.resolve.scope' for not yet loaded registry
Exception in thread "DefaultDispatcher-worker-2" java.lang.OutOfMemoryError: Metaspace
Exception in thread "DefaultDispatcher-worker-1" java.lang.OutOfMemoryError: Metaspace
Exception in thread "DefaultDispatcher-worker-1" kotlinx.coroutines.CoroutinesInternalError: Fatal exception in coroutines machinery for DispatchedContinuation[Dispatchers.Default, Continuation at org.jetbrains.dokka.base.translators.psi.DefaultPsiToDocumentableTranslator$DokkaPsiParser$parseClasslike$2$1$classlikes$1$invokeSuspend$$inlined$parallelMap$1$1.invokeSuspend(DefaultPsiToDocumentableTranslator.kt:19)@45ade651]. Please read KDoc to 'handleFatalException' method and report this incident to maintainers
        at kotlinx.coroutines.DispatchedTask.handleFatalException(DispatchedTask.kt:144)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:115)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:571)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:678)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:665)
Caused by: java.lang.OutOfMemoryError: Metaspace
<===========--> 92% EXECUTING [35m 18s]
> IDLE
> IDLE
> :org-organicdesign:test-utils-http:dokkaJavadoc
> IDLE
> IDLE

[IgnatBeresnev]

Hi @GlenKPeterson! Are you using samples configuration property by any chance?

tasks.withType<DokkaTask>().configureEach {
    dokkaSourceSets {
        configureEach {
            samples.from("samples/basic.kt", "samples/advanced.kt")
        }
    }
}

If not, I'm afraid that might be related to Gradle daemons not freeing memory. Try running your builds with --no-daemon and see if it helps.

Running pre-release Dokka 1.7.0 I can see that it's freeing up metaspace quite well for a project with >250 modules (running with --parallel and 2G Metaspace), and no major leaks are happening, certainly not as before:

[GlenKPeterson]

Thanks for your response. I'm always using --no-daemon at the command line. I think having IntelliJ open causes a separate daemon to run?

The Gradle Tooling API that is used by IDEs and other tools to integrate with Gradle always uses the Gradle Daemon to execute builds. If you are executing Gradle builds from within your IDE you are using the Gradle Daemon and do not need to enable it for your environment.
https://docs.gradle.org/current/userguide/gradle_daemon.html#sec:tools_and_ides

Could that be what's messing me up?

I searched my entire project for "samples" and only found an image: sampleScreenShot.png. I'm still out of metaspace on and off.

I only have one gradle.properties.kts in the master project with org.gradle.jvmargs="-XX:MaxMetaspaceSize=1024m". I assume that's the right place for it.

[IgnatBeresnev]

@GlenKPeterson is gradle.properties.kts a thing? Shouldn't it be just plain gradle.properties?

## gradle.properties
org.gradle.jvmargs=-XX:MaxMetaspaceSize=2G

If you're not using samples, you should not experience any major leaks. If it's an open source project, I can have a look, but if it's closed source, there's little I can do.

---

Try increasing metaspace and have a look at the Metaspace Graph in VisualVM. You should see the saw-like pattern just like in the screenshot above. If it only goes up and never down, try pressing Perform GC button, it should stall for a few seconds and then drop down. If it doesn't happen, there might be a leak. Generally any info that you can gather would be useful, we'd like to get this fixed if it's a leak.

If you need help or have questions, feel free to reach out to me at ignat.beresnev@jetbrains.com or by messaging me in the Kotlin Community Slack (same name there)

[GlenKPeterson]

I was just clearing large files out of my drive and found more data for you. I still get this issue from time to time. After looking at this briefly, I wonder if I'm using too many links in my KotlinDocs? Or linking to the Java or Kotlin documentation on the web too often? I have a vague recollection of using a compile parameter to specify which version of the standard library documentation to link to, but don't remember if that's for this project or not.

[aSemy]

I am working on a replacement for the Dokka Gradle Plugin, https://github.com/adamko-dev/dokkatoo, which uses the Gradle Worker API to run Dokka in an isolated process.

I had hoped that running Dokka in an isolated daemon would help, especially as Gradle should re-use the daemon, but unfortunately it hasn't helped, and anecdotally OOM errors seem to be triggered more often for even the basic examples andintegration-test projects.

Here is an example failure: https://github.com/adamko-dev/dokkatoo/actions/runs/4211535460/jobs/7310041760

Since I have set up Dokka to run in a separate process it should be possible to analyze the process independently of other work. I will see if I can set this up, but if anyone would like to try, please check out the project and run the tests, and hints or tips would be appreciated!

[IgnatBeresnev]

We've fixed all known Metaspace leaks as of today, and added some tests to detect such problems in the future. Unfortunately, Dokka still requires a considerable amount of memory to document large projects.

If you are experiencing Metaspace-related issues when running Dokka:

• Upgrade to the latest version of Dokka
• Try increasing Metaspace size (-XX:MaxMetaspaceSize=2G) as the defaults can be modest. The larger the project, the more you might need to allocate.

If you you have suspicions that Dokka is leaking memory or the Metaspace size you've set goes beyound reasonable, please create a separate issue for your case. We would appreciate having a reproducer and as much information as possible to collect.

I'm closing this general issue so that we can concentrate on specific cases.