consul/connect: fixed a bug where restarting proxy tasks failed. #16815
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jrasell
added
backport/1.3.x
jrasell
added a commit
that referenced
this pull request
Apr 6, 2023
The first start of a Consul Connect proxy sidecar triggers a run of the envoy_version hook which modifies the task config image entry. The modification takes into account a number of factors to correctly populate this. Importantly, once the hook has run, it marks itself as done so the taskrunner will not execute it again. When the client receives a non-destructive update for the allocation which the proxy sidecar is a member of, it will update and overwrite the task definition within the taskerunner. In doing so it overwrite the modification performed by the hook. If the allocation is restarted, the envoy_version hook will be skipped as it previously marked itself as done, and therefore the sidecar config image is incorrect and causes a driver error. The fix removes the hook in marking itself as done to the view of the taskrunner.
jrasell
force-pushed
the
b-gh-9887-envoy-proxy-restart-hook
branch
from
1b00845
to
11dc5e3
Compare
jrasell
added a commit
that referenced
this pull request
Apr 11, 2023
) The first start of a Consul Connect proxy sidecar triggers a run of the envoy_version hook which modifies the task config image entry. The modification takes into account a number of factors to correctly populate this. Importantly, once the hook has run, it marks itself as done so the taskrunner will not execute it again. When the client receives a non-destructive update for the allocation which the proxy sidecar is a member of, it will update and overwrite the task definition within the taskerunner. In doing so it overwrite the modification performed by the hook. If the allocation is restarted, the envoy_version hook will be skipped as it previously marked itself as done, and therefore the sidecar config image is incorrect and causes a driver error. The fix removes the hook in marking itself as done to the view of the taskrunner.
jrasell
added a commit
that referenced
this pull request
Apr 11, 2023
) The first start of a Consul Connect proxy sidecar triggers a run of the envoy_version hook which modifies the task config image entry. The modification takes into account a number of factors to correctly populate this. Importantly, once the hook has run, it marks itself as done so the taskrunner will not execute it again. When the client receives a non-destructive update for the allocation which the proxy sidecar is a member of, it will update and overwrite the task definition within the taskerunner. In doing so it overwrite the modification performed by the hook. If the allocation is restarted, the envoy_version hook will be skipped as it previously marked itself as done, and therefore the sidecar config image is incorrect and causes a driver error. The fix removes the hook in marking itself as done to the view of the taskrunner.
jrasell
added a commit
that referenced
this pull request
Apr 11, 2023
) (#16844) The first start of a Consul Connect proxy sidecar triggers a run of the envoy_version hook which modifies the task config image entry. The modification takes into account a number of factors to correctly populate this. Importantly, once the hook has run, it marks itself as done so the taskrunner will not execute it again. When the client receives a non-destructive update for the allocation which the proxy sidecar is a member of, it will update and overwrite the task definition within the taskerunner. In doing so it overwrite the modification performed by the hook. If the allocation is restarted, the envoy_version hook will be skipped as it previously marked itself as done, and therefore the sidecar config image is incorrect and causes a driver error. The fix removes the hook in marking itself as done to the view of the taskrunner.
jrasell
added a commit
that referenced
this pull request
Apr 11, 2023
) (#16843) The first start of a Consul Connect proxy sidecar triggers a run of the envoy_version hook which modifies the task config image entry. The modification takes into account a number of factors to correctly populate this. Importantly, once the hook has run, it marks itself as done so the taskrunner will not execute it again. When the client receives a non-destructive update for the allocation which the proxy sidecar is a member of, it will update and overwrite the task definition within the taskerunner. In doing so it overwrite the modification performed by the hook. If the allocation is restarted, the envoy_version hook will be skipped as it previously marked itself as done, and therefore the sidecar config image is incorrect and causes a driver error. The fix removes the hook in marking itself as done to the view of the taskrunner.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The first start of a Consul Connect proxy sidecar triggers a run of the envoy_version hook which modifies the task config image entry. The modification takes into account a number of factors to correctly populate this. Importantly, once the hook has run, it marks itself as done so the taskrunner will not execute it again.
When the client receives a non-destructive update for the allocation which the proxy sidecar is a member of, it will update and overwrite the task definition within the taskerunner. In doing so it overwrite the modification performed by the hook. If the allocation is restarted, the envoy_version hook will be skipped as it previously marked itself as done, and therefore the sidecar config image is incorrect and causes a driver error.
The fix removes the hook in marking itself as done to the view of the taskrunner, and instead tracks this internally. The hook also implements the TaskPreKillHook interface, so that it's state can be correctly modified when required by task events.
closes #9887
The linked issue has reproduction steps for testing.