-
Notifications
You must be signed in to change notification settings - Fork 50
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Memory leaked every time formatKotlin
task is run
#204
Comments
Hey @marshallpierce 👋 I'd shoot you're facing the kotlin gradle plugin memory leak: https://youtrack.jetbrains.com/issue/KT-46368 which will be fixed in kotlin 1.5.10 Have you tried to apply a workaround, for example Zac Sweers plugin: https://github.com/ZacSweers/kgp-150-leak-patcher? |
I see the same leak behavior with kotlin 1.4.32 selected. The same goes for the workaround plugin + 1.5.0. |
I made a teeny tiny repro project: https://github.com/marshallpierce/kotlinter-leak-repro With less code it leaks more slowly but it's still leaking (easily visible with GC.class_histogram). |
Thanks for the research and the repro repo. I have also recently noticed some jvm heap space errors from gradle in a local setup with similar versions. Have we looked to see how far back this issue goes? I've looked over the changes starting with 3.4.0 and don't see any recent changes in the plugin logic that seem impactful -- the main changes have been to dependencies: new ktlint, kotlin, gradle. Has anyone tried 3.3.0? |
At the risk of sounding naive, are we sure this isn't normal lazy garbage collection behavior on the gradle daemon? A ran a series of Are you seeing the baseline climb after GC? This is using gradle 7. I ran the same tests with kotlinter 3.3.0 and kotlin 1.4.32 and behavior seemed the same. |
Nothing naive about being thorough! :) Unfortunately, in my measurements above, I'm running a GC (the I'd try |
Ok, digging into a bigger codebase with thousands of lines of code, I do see the problem.
I can comb through the changes between the versions again. My best guess it's a side effect of upgrading ktlint 0.40.0 -> 0.41.0. Although certainly maybe we're using it wrong in some way? |
I haven't forgotten about this issue, but I haven't had the chance to delve into the guts of ktlint where the leak most likely is. hiding. I have a feeling, not scientific whatsoever, that ktlint is going to spin a new release soon upgrading to kotlin 1.5 and this leak will be fixed. I could be wrong though. |
This memory leak can happen on lintKotlin as well, most easily seen when I wonder if the root cause is the same as JLLeitschuh/ktlint-gradle#507? People have pointed to the usage of gradle workers as being the potential culprit. Perhaps this could be, but I believe workers are being used according to gradle's instructions here and there's nothing to explicitly close or return. It could be newer versions of gradle have introduced a bug with workers leaking memory, but I'm doubtful. I still suspect there's a memory leak at the ktlint level, perhaps even in the embedded kotlin compiler that we use to parse files. |
Update on this. The leak is still present in kotlinter 3.5.0 and ktlint 0.42.1. |
With 3.7.0 (ktlint 0.43.0) this issue appears to be fixed. I was able to run Closing, but please let me know if anyone sees the issue crop up again. |
It seems like each invocation of
formatKotlin
leaks tens of MiB of heap on a small codebase (2500loc). This is using Gradle 7.0 with kotlinter 3.4.4 and kotlin 1.5.0.Here's the sequence of output from the class histogram (obtained via
jcmd 15861 GC.run && jcmd 15861 GC.class_histogram | grep kotlin | head
) after running theformatKotlin
task several times, capturing the histogram after each run:Here's the same type of data, this time on a fresh daemon (
./gradlew --stop && ./gradlew classes
to create a warmed up daemon), then showing the first two invocations offormatKotlin
.I've attached a heap graph created with my heap analysis tools in case that's instructive. It's clear that there are a whole lot of PicoContainers being made, among other kotlin internal classes, so perhaps there is some reference being cached when it shouldn't be either in the plugin or in ktlint itself?
gradle daemon heap graph.svg.gz
The text was updated successfully, but these errors were encountered: