Job run CLI exit code is zero even when deployment fails

[martinmcnulty]

Nomad version

Nomad v1.2.3 (a79efc8422082c4790046c3f5ad92c542592a54f)

Operating system and Environment details

Issue

When using nomad job run to trigger a deployment, if the deployment fails, the command still exits with a zero exit code, making it difficult to detect problems in shell scripts.

Reproduction steps

1. Pass the included job file to nomad job run
2. Check the exit code with echo $?

Expected Result

Exit code should be 1 on a deployment failure, according to the docs:

Any other errors, including deployment failures, client connection issues, or internal errors, are indicated by exit code 1.

Actual Result

Exit code is 0:

$ nomad job run fail.hcl
==> 2021-12-20T12:51:34Z: Monitoring evaluation "b9c0dff9"
    2021-12-20T12:51:34Z: Evaluation triggered by job "echo-fail"
    2021-12-20T12:51:34Z: Evaluation within deployment: "3eefc5eb"
    2021-12-20T12:51:34Z: Allocation "b38c1107" created: node "44624543", group "echo"
    2021-12-20T12:51:34Z: Evaluation status changed: "pending" -> "complete"
==> 2021-12-20T12:51:34Z: Evaluation "b9c0dff9" finished with status "complete"
==> 2021-12-20T12:51:34Z: Monitoring deployment "3eefc5eb"
  ! Deployment "3eefc5eb" failed

    2021-12-20T12:51:54Z
    ID          = 3eefc5eb
    Job ID      = echo-fail
    Job Version = 0
    Status      = failed
    Description = Failed due to progress deadline

    Deployed
    Task Group  Desired  Placed  Healthy  Unhealthy  Progress Deadline
    echo        1        1       0        1          2021-12-20T12:52:06Z
$ echo $?
0

Job file (if appropriate)

job "echo-fail" {

  type = "service"
  datacenters = ["dc1"]

  group "echo" {

    network {
      port "http" {
        # Deliberately forward to the wrong port so the health check fails
        to = 8081
      }
    }

    service {
      name = "echo"
      port = "http"
      tags = ["http"]

      check {
        type = "http"
        port = "http"
        path = "/nope"
        interval = "5s"
        timeout = "2s"
      }

    }

    update {
      healthy_deadline = "15s"
      progress_deadline = "20s"
    }

    task "echo" {
      driver = "docker"
      config {
        image = "mendhak/http-https-echo:21"
        ports = ["http"]
      }
    }
  }

}

Nomad Server logs (if appropriate)

Nomad Client logs (if appropriate)

[lgfa29]

Hi @martinmcnulty 👋

I believe this was fixed in #11550 and will ship in our next release, so I'm going to close this one for now but let us know if that fix is not what you are looking for.

[martinmcnulty]

Hi @lgfa29 :)

Thanks for that - that PR sounds like exactly what I'm looking for! Sorry, not sure how I missed the existing issues when I searched (maybe I only searched for open ones? Who knows....)

Anyway, thanks again. Look forward to the next release!

[lgfa29]

No worries. It's always hard to keep up with issues and PRs.

We appreciate you reaching out and providing such a detailed issue report ❤️

[github-actions]

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.