A job evaluation stops allocations before missing allocations are started

[clinta]

Nomad version

v0.6.0

Operating system and Environment details

Ubuntu 16.04

If you have a job with max_parallel set to 1 and a count of some higher number, say 5, then you update the job with an invalid configuration which causes the allocation to fail, the evaluation will, as expected, stop one allocation, try to start one new allocation and fail to do so. The deployment will fail and the 4 existing allocations will continue running as expected.

If you then trigger a gc via the nomad api, the failed allocation will be deleted. Then trigger a re-evaluation of the same job. The reevaluation will stop another running allocation, leaving you with only 3 running allocations and will try to start up two failed allocations.

You can keep doing this, losing one running allocation each time.

The evaluation should take into account that you only have 4 of 5 desired allocations running, and try to start the missing allocation and make sure it is healthy before shutting down any additional allocations. It should never shut down allocations if doing so would result in more than max_parallel allocations being down.

[github-actions]

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.