Move IndexLifecycleMetadata installation to put-lifecycle-action #31346
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
There is a problematic scenario with x-pack-cluster master-nodes attempting to install custom metadata into the cluster-state and broadcasting that to non-x-pack-enabled nodes. Since those nodes are not aware of this custom metadata, their cluster-state recovery will be broken. This change ensures that newly-elected x-pack master nodes bootstrap IndexLifecycleMetadata upon the first request to leverage its features. This means that PutLifecycleAction is now responsible for installing the metadata. Since this X-Pack API can only be called once all nodes in the cluster have x-pack enabled, it is safe to assume that the cluster will appropriately handle the cluster-state recovery with the new set of index-lifecycle metadata.
talevy
added
review
:Data Management/ILM+SLM
|
Pinging @elastic/es-core-infra |
talevy
commented
Jun 14, 2018
| private LongSupplier nowSupplier; | ||
| private SchedulerEngine.Job scheduledJob; | ||
| private IndexLifecycleRunner lifecycleRunner; | ||
|
|
||
| public IndexLifecycleService(Settings settings, Client client, ClusterService clusterService, Clock clock, |
There was a problem hiding this comment.
ThreadPool is no longer used since the service does not issue a cluster-state-update to install metadata anymore
talevy
commented
Jun 14, 2018
| @@ -59,6 +59,8 @@ public PolicyStepsRegistry() { | |||
| @SuppressWarnings({ "unchecked", "rawtypes" }) | |||
| public void update(ClusterState currentState, Client client, LongSupplier nowSupplier) { | |||
| IndexLifecycleMetadata meta = currentState.metaData().custom(IndexLifecycleMetadata.TYPE); | |||
| assert meta != null : "IndexLifecycleMetadata cannot be null when updating the policy steps registry"; | |||
There was a problem hiding this comment.
I've seen this pattern of assertion elsewhere in the codebase. I am not sure how I feel about it, but I chose to try it out here to be self-documenting. Currently this method is only ever called once, and it is behind a null check
talevy
commented
Jun 14, 2018
| @@ -32,11 +32,12 @@ | |||
| import java.util.SortedMap; | |||
| import java.util.TreeMap; | |||
|
|
|||
| public class TransportDeleteLifcycleAction extends TransportMasterNodeAction<Request, Response> { | |||
| public class TransportDeleteLifecycleAction extends TransportMasterNodeAction<Request, Response> { | |||
There was a problem hiding this comment.
missed an existing typo
talevy
commented
Jun 14, 2018
| @@ -74,7 +75,8 @@ public ClusterState execute(ClusterState currentState) { | |||
| } | |||
| ClusterState.Builder newState = ClusterState.builder(currentState); | |||
| IndexLifecycleMetadata currentMetadata = currentState.metaData().custom(IndexLifecycleMetadata.TYPE); | |||
| if (currentMetadata.getPolicyMetadatas().containsKey(request.getPolicyName()) == false) { | |||
| if (currentMetadata == null | |||
There was a problem hiding this comment.
since this is an anonymous class, we do not have good unit test coverage, so I've added a yaml test to walk this branch
talevy
commented
Jun 14, 2018
| @@ -64,6 +68,9 @@ protected Response newResponse(boolean acknowledged) { | |||
| public ClusterState execute(ClusterState currentState) throws Exception { | |||
| ClusterState.Builder newState = ClusterState.builder(currentState); | |||
| IndexLifecycleMetadata currentMetadata = currentState.metaData().custom(IndexLifecycleMetadata.TYPE); | |||
| if (currentMetadata == null) { // first time using index-lifecycle feature, bootstrap metadata | |||
There was a problem hiding this comment.
existing yaml tests should walk this branch
talevy
commented
Jun 14, 2018
| @@ -6,6 +6,16 @@ setup: | |||
|
|
|||
| --- | |||
| "Test Basic Policy CRUD": | |||
| - do: | |||
| catch: missing | |||
| xpack.index_lifecycle.get_lifecycle: | |||
There was a problem hiding this comment.
added to walk the branch in TransportGetLifecycleAction that checks for null metadata
|
woohoo elasticsearch-ci went green! |
colings86
reviewed
Jun 15, 2018
|
@colings86 good catch. I've updated set-policy-action, but I do not see it for RetryAction or MoveToStepAction because they deal with the steps registry directly. the rest of the actions like explain seem to deal with settings directly, as well. |
jasontedor
pushed a commit
that referenced
this pull request
Aug 17, 2018
) There is a problematic scenario with x-pack-cluster master-nodes attempting to install custom metadata into the cluster-state and broadcasting that to non-x-pack-enabled nodes. Since those nodes are not aware of this custom metadata, their cluster-state recovery will be broken. This change ensures that newly-elected x-pack master nodes bootstrap IndexLifecycleMetadata upon the first request to leverage its features. This means that PutLifecycleAction is now responsible for installing the metadata. Since this X-Pack API can only be called once all nodes in the cluster have x-pack enabled, it is safe to assume that the cluster will appropriately handle the cluster-state recovery with the new set of index-lifecycle metadata.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There is a problematic scenario with x-pack-cluster master-nodes
attempting to install custom metadata into the cluster-state and
broadcasting that to non-x-pack-enabled nodes. Since those nodes
are not aware of this custom metadata, their cluster-state recovery
will be broken. This change ensures that newly-elected x-pack master
nodes bootstrap IndexLifecycleMetadata upon the first request to
leverage its features. This means that PutLifecycleAction is
now responsible for installing the metadata. Since this X-Pack API
can only be called once all nodes in the cluster have x-pack enabled,
it is safe to assume that the cluster will appropriately handle the
cluster-state recovery with the new set of index-lifecycle metadata.
in response to rolling upgrade test failure here: https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+index-lifecycle+feature-branch/69/