Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backport of Bugfix: auto-promote canary taskgroups when mixed with non-canary taskgroups into release/1.2.x #12607

Conversation

hc-github-team-nomad-core
Copy link
Contributor

Backport

This PR is auto-generated from #11878 to be assessed for backporting due to the inclusion of the label backport/1.2.x.

The below text is copied from the body of the original PR.


When using the auto promote feature with canary deployments that have task groups without canaries, the deployment will never auto promote and hang even when the canaries are all healthy for the task groups that it has been enabled for.

This PR fixes this bug by skipping task groups that have no canaries set during the auto promote validation and adds a test to catch this case specifically. To see what occurs when this fix is not implemented (as has been observed in mainstream Nomad):

  • comment out/delete L292-L294 of nomad/deploymentwatcher/deployment_watcher.go
  • run tests for this package

Let me know if there's anything I can do to help shepard this through into a release, currently there's manual intervention in some of our deployments that this occurs in and it would be great to alleviate that.

@hc-github-team-nomad-core hc-github-team-nomad-core force-pushed the backport/fix/multi-task-group-canary-deploys/routinely-able-pangolin branch from f98b729 to 2bb3f63 Compare April 19, 2022 14:40
@hc-github-team-nomad-core hc-github-team-nomad-core merged commit bc25328 into release/1.2.x Apr 19, 2022
@hc-github-team-nomad-core hc-github-team-nomad-core deleted the backport/fix/multi-task-group-canary-deploys/routinely-able-pangolin branch April 19, 2022 14:40
@github-actions
Copy link

I'm going to lock this pull request because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 19, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants