Clean build fails in compileKotlin task when parallel execution enabled #1380
Labels
Milestone
Comments
|
thanks for the repro - parallelizing the gradle plugin is something im working on currently so I'll make sure to verify it works consistently for your sample project |
|
Thanks for the reply, let me know if I can help in any way. I'm keen to be able to use gradle |
|
I think I've found the main cause of this problem, This means that if any of the verifyMigration tasks run before a compile task, then the sources generated by the generateXXXDatabaseInterface tasks are deleted and so the compile task fails. The verifyMigration task runs as part of You can see in my builds scans above, on the build that failed, This also explains the trouble with reproducing this reliably, as it depends on the task execution ordering. A reliable workaround for me is to exclude all the A quick way to demonstrate the deletion is to run:
|
lfielke
added a commit
to lfielke/sqldelight
that referenced
this issue
Jul 9, 2019
Avoids clobbering source code generated by SqlDelightTask, which will cause compilation to fail if it runs after this task.
|
Seeing similar behavior in Cash App. Here's a stack trace: |
|
ah i see, i misclosed this one probably because it has parallel in the name, but seems unrelated to parallel execution? |
|
okay im up to speed now - they use the same output folder and both of them delete things in that folder which can race and have problems. Suggested what we need to move forward there in the PR, I think there's a pretty simple line of sight to getting a fix in |
lfielke
added a commit
to lfielke/sqldelight
that referenced
this issue
Aug 1, 2019
Working towards supporting parallel task execution cashapp#1380
lfielke
added a commit
to lfielke/sqldelight
that referenced
this issue
Aug 1, 2019
So it is not shared with tasks that generate code. For supporting parallel task execution cashapp#1380
|
I think this is fixed now...there are a few moving parts and some PRs that went through to address it. If not do you mind opening a new separate issue with the remaining pieces missing? |
|
ugh still seeing it so reopening |
|
It seems that our nightly build reproduces this at 90% rate. What it does is building of The project is private (and is big), but maybe I can do something to help diagnose the issue? |
|
We have the same issue, Now we add workaround, where every SQLDelight module with dependencies on other SQLDelight modules adds So it doesn't allow generation tasks run in parallel. So far so good, we never experience this crash after this change |
|
Is this related to #1649 |
|
Tracking here #1439 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I'm using SqlDelight in an Android library module as part of an Android application.
Development builds and IDE experience is working well, however clean builds consistently fail both on my Linux workstation and a MacOS machine used as a build server.
The build will fail in either the
compileReleaseKotlinorcompileDebugKotlintasks in the gradle project that's using SqlDelight, with Unresolved reference errors for the generated classes/interfaces (see build scan below for full output).Repeating the same build command (without cleaning) will succeed.
I suspect there is a timing problem or race, because disabling Gradle Parallel execution (
--no-parallel) or reducingmax-workersto 1 makes the build reliably succeed.However this is not a great workaround, because this makes the builds a lot slower.
I cannot share our app project where the problem happens consistently. I've had trouble making a small, reliable repro project. The app that this is occurring in has multiple gradle modules and uses dagger via the
kotlin-kaptplugin, so these are included in the repro. The only way I've been able to trigger the problem in a small project though, is adding a sleep to the SqlDelight generate interface tasks. Our actual app doesn't do this.I've uploaded the small repro here:
https://github.com/lfielke/sqldelightbug
Checkout the project and run this a few times:
./gradlew clean && ./gradlew --max-workers=8 --parallel -x lint build assembleAndroidTestOn my computer this starts failing after the second run.
Build scan of failed build (command as above):
https://scans.gradle.com/s/jpbakohx64zee/activate/klvs6thkipury
Build scan of successful build (command changed --max-workers=1):
https://scans.gradle.com/s/v3ijkbfs6j7vy/activate/jaxujpzu5disq
Note that adding
--scanto enable a build scan seems to alter timings and makes it suceed more often.Also note that the first build where the jettify transforms run seems to always succeed (this is also true after changing the build scripts or environment).
Subsequent clean builds then fail most of the time for me.
Just running the second
gradlewcommand will successfully build the project after a failure for me, so there doesn't appear to be anything wrong with the source code.The repro project is using Android Gradle Plugin 3.5.0-beta04, which seems to trigger the problem more often in the repro project. The app with the problem is using AGP 3.4.1.
Removing
--parallelfrom the above command also makes it succeed for me.Note that adding or changing Gradle properties seems to cause more stuff to be reevaluated and then the first clean build will succeed. This has made it tricky to get a repro!
I don't know how to troubleshoot this any further. From what I can tell, the
compileKotlintask depends on the generate interfaces task and is being executed after they've finished in the failed build.I'm using the default kotlin compiler strategy, which uses a daemon if I understand correctly. Could the kotlin compiler daemon be mis-caching that there's no generated sources to add to the compilation?
The text was updated successfully, but these errors were encountered: