Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CCR] Auto follow pattern APIs adjustments #34518

Merged
merged 7 commits into from
Oct 23, 2018
Merged

[CCR] Auto follow pattern APIs adjustments #34518

merged 7 commits into from
Oct 23, 2018

Conversation

martijnvg
Copy link
Member

@martijnvg martijnvg commented Oct 16, 2018 •
edited
Loading

  • Changed the resource id of auto follow patterns to be a user defined name in all auto follow pattern crud APIs.
  • Which leader cluster to use is now specified in the body of an auto follow pattern.
  • Fail when an unfollowed leader index matches with two or more auto follow patterns. The unfollow leader index will then not be followed and the error can be seen in the auto follow stats API. The reason this validation is needed is, because multiple auto follow patters can be defined for a single leader cluster.

@martijnvg martijnvg added >non-issue :Distributed Indexing/CCR Issues around the Cross Cluster State Replication features labels Oct 16, 2018
@martijnvg martijnvg requested a review from jasontedor October 16, 2018 12:36
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed

@martijnvg
Copy link
Member Author

run gradle build tests

bleskes
bleskes reviewed Oct 17, 2018
View reviewed changes
x-pack/plugin/ccr/src/test/java/org/elasticsearch/xpack/ccr/action/AutoFollowIT.java Outdated
if (randomBoolean()) {
putAutoFollowPatterns("my-pattern", new String[] {"logs-*", "transactions-*"});
} else {
putAutoFollowPatterns("my-pattern1", new String[] {"logs-*"});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💯

x-pack/plugin/ccr/src/test/java/org/elasticsearch/xpack/ccr/action/AutoFollowIT.java Outdated
@@ -172,9 +180,54 @@ public void testAutoFollowParameterAreDelegated() throws Exception {
});
}

private void putAutoFollowPatterns(String... patterns) {
public void testConflictingPatterns() throws Exception {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if two patterns match an index, why is that a problem? we used to be able to do it before no?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So only one auto follow pattern would trigger the index following and when applying the second one would result in an error (same follow index already exists in follow cluster). Also which auto follow pattern gets applied first is undefined.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I looked at it more and I realized I got confused - with the new model we can still specify multiple patterns in a single auto follow config (is this a better name?). All good. Thanks for explaining.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

with the new model we can still specify multiple patterns in a single auto follow config (is this a better name?)

Yes, we can still do that.

jasontedor
jasontedor approved these changes Oct 23, 2018
View reviewed changes
Copy link
Member

@jasontedor jasontedor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@martijnvg martijnvg merged commit 36baf38 into elastic:master Oct 23, 2018
martijnvg added a commit that referenced this pull request Oct 23, 2018
@martijnvg
[CCR] Auto follow pattern APIs adjustments (#34518)
cdf3a7c 
* Changed the resource id of auto follow patterns to be a user defined name
instead of being the leader cluster alias name.
* Fail when an unfollowed leader index matches with two or more auto follow patterns.
jasontedor added a commit to jasontedor/elasticsearch that referenced this pull request Oct 23, 2018
@jasontedor
Merge branch 'master' into ccr-docs
f7bbdd1 
* master: (24 commits)
  ingest: better support for conditionals with simulate?verbose (elastic#34155)
  [Rollup] Job deletion should be invoked on the allocated task (elastic#34574)
  [DOCS] .Security index is never auto created (elastic#34589)
  CCR: Requires soft-deletes on the follower (elastic#34725)
  re-enable bwc tests (elastic#34743)
  Empty GetAliases authorization fix (elastic#34444)
  INGEST: Document Processor Conditional (elastic#33388)
  [CCR] Add total fetch time leader stat (elastic#34577)
  SQL: Support pattern against compatible indices (elastic#34718)
  [CCR] Auto follow pattern APIs adjustments (elastic#34518)
  [Test] Remove dead code from ExceptionSerializationTests (elastic#34713)
  A small typo in migration-assistance doc (elastic#34704)
  ingest: processor stats (elastic#34724)
  SQL: Implement IN(value1, value2, ...) expression. (elastic#34581)
  Tests: Add checks to GeoDistanceQueryBuilderTests (elastic#34273)
  INGEST: Rename Pipeline Processor Param. (elastic#34733)
  Core: Move IndexNameExpressionResolver to java time (elastic#34507)
  [DOCS] Force Merge: clarify execution and storage requirements (elastic#33882)
  TESTING.asciidoc fix examples using forbidden annotation (elastic#34515)
  SQL: Implement `CONVERT`, an alternative to `CAST` (elastic#34660)
  ...
kcm pushed a commit that referenced this pull request Oct 30, 2018
@martijnvg @kcm
[CCR] Auto follow pattern APIs adjustments (#34518)
6c65667 
* Changed the resource id of auto follow patterns to be a user defined name
instead of being the leader cluster alias name.
* Fail when an unfollowed leader index matches with two or more auto follow patterns.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bleskes bleskes bleskes left review comments

@jasontedor jasontedor jasontedor approved these changes

Assignees
No one assigned
Labels
:Distributed Indexing/CCR Issues around the Cross Cluster State Replication features >non-issue
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@martijnvg @elasticmachine @bleskes @jasontedor