Some properties make builds non incremental when it's expected to be

[artemptushkin]

Some of the properties like gradle.build.host and all the other gradle.build properties make builds less reproducible as the latter may differ in the pipeline, especially host one.

The example scenario.

You have a set of tasks:

• compileKotlin
• generateGitProperties
• processResources
• compileTestKotlin

And you run it inside a pipeline with virtual envs as a runtime environment, in this case you might use a different host (runner) from job-to-job

Most probably you will have jobs designed in a next way:

• pipeline build job:
  • compileKotlin
    ...
  • generateGitProperties
    ...
  • processResources
• pipeline test job:
  • compileKotlin
    ...
  • generateGitProperties
    ...
  • processResources
  • compileTestKotlin <- this job depends on main sourseSet compilation, hence on generateGitProperties
  • test

Problem

Whenever the task generateGitProperties is not UP-TO-DATE it will rerun and will generate a new git.properties having a different host inside the file. This will enforce to rerun processResources and compileTestKotlin and test where two latter tasks are heavy cacheable tasks and not expected to rerun sometimes

In other words:
Some git properties make a cascade break of up-to-date logic of many tasks and don't let it use Gradle binary cache

Workaround

Two options:

1. Exclude the host (or any other property) in your root build.gradle

gradle.taskGraph.whenReady {
    if (this.allTasks.any { it.name.contains("jib") || it.name.contains("jar") }) {
        tasks.withType(com.gorylenko.GenerateGitPropertiesTask::class.java).all { enabled = true }
    } else {
        tasks.withType(com.gorylenko.GenerateGitPropertiesTask::class.java).all { enabled = false }
    }
}

In the GitLab pipeline I have the next workaround to make incremental build works

Update:

Having any property actually causes compileTestKotlin to rerun, here is the prove

gitProperties {
    customProperty("git.build.custom", Instant.now().toString())
}

Running ./gradlew test

Output:

> Task :compileTestKotlin
Build cache key for task ':compileTestKotlin' is fcf471630902ca48ab6be93e4c0b7a2f
Task ':compileTestKotlin' is not up-to-date because:
  Input property 'classpathSnapshotProperties.classpath' file /Users/artemptushkin/...build/resources/main/git.properties has changed.

Proposal

1. Short term. Define a property for gitProperties to exclude some git properties as ...keys.findAll is not a very convenient way

2. Long-term. There is a high chance that another git property will break someone and this will be not clear why. I do think it's possible to tweak this somehow else but maybe we should log something here?

[artemptushkin]

This is exactly the paragraph that explains the concept

[igormukhin]

Gradle has "normalization" for that purpose:
https://docs.gradle.org/current/userguide/build_cache_concepts.html#normalization

normalization {
    runtimeClasspath {
        ignore("build-info.properties")
    }
}