Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nomad status reports "placement failure" even when job is running ok #2124

Closed
drscre opened this issue Dec 16, 2016 · 5 comments · Fixed by #2155
Closed

Nomad status reports "placement failure" even when job is running ok #2124

drscre opened this issue Dec 16, 2016 · 5 comments · Fixed by #2155

Comments

@drscre
Copy link

drscre commented Dec 16, 2016

Nomad v0.5.1

First, I tried do deploy a job with wrong datacenter "nomad-dc1". Nomad gave me Placement Failure as expected.

Then I redeployed the same job, but now with correct datacenter "production".
Job was successfully launched, but nomad status still reports Placement Failure:

ID          = test-service-default
Name        = test-service
Type        = service
Priority    = 50
Datacenters = production
Status      = running
Periodic    = false

Summary
Task Group       Queued  Starting  Running  Failed  Complete  Lost
grammar-service  0       0         1        0       0         0

Evaluations
ID        Priority  Triggered By  Status    Placement Failures
55f4cfd3  50        job-register  complete  false
614ce8b7  50        job-register  blocked   N/A - In Progress
08cba1a6  50        job-register  complete  true

Placement Failure
Task Group "test-service":
  * No nodes were eligible for evaluation
  * No nodes are available in datacenter "nomad-dc1"

Allocations
ID        Eval ID   Node ID   Task Group       Desired  Status   Created At
67cea612  55f4cfd3  5d9330ea  test-service  run      running  12/16/16 20:56:13 UTC
@pgporada
Copy link
Contributor

If you rename the job and try to nomad run newname.nomad, do you still get the placement error?

@drscre
Copy link
Author

drscre commented Dec 16, 2016

No, nomad status does not report Placement Failure if I rename a job.

I guess it has something to do with this loop https://github.com/hashicorp/nomad/blob/master/command/status.go#L267
which picks up a failed placement from the past evaluations even if it is no more relevant

@drscre
Copy link
Author

drscre commented Dec 16, 2016

I added evaluations to the client log. As you can see, the most recent evaluation 55f4cfd3 was successfull and job is running ok.

@drscre drscre closed this as completed Dec 16, 2016
@drscre drscre reopened this Dec 16, 2016
@dadgar
Copy link
Contributor

dadgar commented Dec 19, 2016

Hey this is a known bug. It is only a UX issue and won't actually impact your job. Still an issue tho!

dadgar added a commit that referenced this issue Jan 4, 2017
This PR causes blocked evaluations to be cancelled if there is a
subsequent successful evaluation for the job. This fixes UX problems
showing failed placements when there are not any in reality and makes GC
possible for these jobs in certain cases.

Fixes #2124
dadgar added a commit that referenced this issue Jan 5, 2017
This PR causes blocked evaluations to be cancelled if there is a
subsequent successful evaluation for the job. This fixes UX problems
showing failed placements when there are not any in reality and makes GC
possible for these jobs in certain cases.

Fixes #2124
@github-actions
Copy link

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Dec 16, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants