Prestart / Init task continues to render templates

[joliver]

• Nomad v1.4.3
• Ubuntu 22.04.1 LTS (Linux 5.15.0-53-generic)

When a given init/prestart task completes successfully, it appears that any template defintions for that task with Vault secrets are continually updated--even beyond the lifetime of the prestart task.

The expected behavior is that, once a task concludes, all templates associated with it are no longer updated.

Here's an example job file that demonstrates the issue:

job "my-job" {
  datacenters = ["default"]

  group "primary" {
    count = 1

    task "main" {
      driver = "exec"

      config {
        command = "sleep"
        args    = ["infinity"]
      }
    }

    task "init" {
      driver = "exec"

      lifecycle { hook = "prestart" } # init task, once it exits, the template shouldn't be re-rendered

      vault { policies = [ "vault-policy-here" ] }

      config {
        command = "echo" # arbitrary command; typically much more happening here
        args    = ["done"]
      }

      # new secret.txt file is re-rendered every few minutes even after completion of init (prestart) task
      template {
        change_mode = "noop"
        destination = "${NOMAD_SECRETS_DIR}/secret.txt"
        data        = <<-EOT
          {{- with secret "gcp/my-service-account/my-reader/token" -}}
          {{ .Data.token }}
          {{- end -}}
        EOT
      }
    }
  }
}

[lgfa29]

Thanks for the report @joliver!

#14127 refactored how tasks with lifecycle run, preventing them from fully stopping to allow restarting all tasks in the allocation. This left to some of the background work that tasks do (such as updating templates) alive unnecessarily.

I opened #15436 to address this issue.