Remove _primary and _replica shard preferences #26791

dnhatn · 2017-09-26T14:19:36Z

The shard preference _primary, _replica and its variants were useful
for the asynchronous replication. However, with the current impl, they
are no longer useful and should be removed.

Closes #26335

The shard preference `_primary`, `_replica` and its variants were useful for the asynchronous replication. However, with the current impl, they are no longer useful and should be removed. Closes elastic#26335

jasontedor

I left some comments. We will also separately need to deprecate these preferences in the 6.x branch and this PR needs a breaking changes note in the migration docs for 7.0.0.

jasontedor · 2017-10-04T11:16:22Z

core/src/test/java/org/elasticsearch/cluster/structure/RoutingIteratorTests.java

+        String[] removedPreferences = {"_primary", "_primary_first", "_replica", "_replica_first"};
+        for (String pref : removedPreferences) {
+            try{
+                operationRouting.searchShards(clusterState, new String[]{"test"}, null, pref);


This can be replaced with expectThrows.

jasontedor · 2017-10-04T11:17:07Z

core/src/test/java/org/elasticsearch/index/translog/TruncateTranslogIT.java

@@ -210,7 +211,10 @@ public void testCorruptTranslogTruncation() throws Exception {
        logger.info("--> starting the replica node to test recovery");
        internalCluster().startNode();
        ensureGreen("test");
-        assertHitCount(client().prepareSearch("test").setPreference("_replica").setQuery(matchAllQuery()).get(), numDocsToKeep);
+        for(String node : internalCluster().nodesInclude("test")){


Can you add extra spacing like:

for( -> for ( )){ -> )) {

Your IDE can help with this.

jasontedor · 2017-10-04T11:17:51Z

core/src/test/java/org/elasticsearch/index/translog/TruncateTranslogIT.java

@@ -308,7 +312,9 @@ public void testCorruptTranslogTruncationOfReplica() throws Exception {
        logger.info("--> starting the replica node to test recovery");
        internalCluster().startNode();
        ensureGreen("test");
-        assertHitCount(client().prepareSearch("test").setPreference("_replica").setQuery(matchAllQuery()).get(), totalDocs);
+        for(String node : internalCluster().nodesInclude("test")){


Extra spacing here:

for( -> for ( )){ -> )) {

jasontedor · 2017-10-04T11:18:18Z

core/src/test/java/org/elasticsearch/index/translog/TruncateTranslogIT.java

@@ -308,7 +312,9 @@ public void testCorruptTranslogTruncationOfReplica() throws Exception {
        logger.info("--> starting the replica node to test recovery");
        internalCluster().startNode();
        ensureGreen("test");
-        assertHitCount(client().prepareSearch("test").setPreference("_replica").setQuery(matchAllQuery()).get(), totalDocs);
+        for(String node : internalCluster().nodesInclude("test")){
+            assertHitCount(client().prepareSearch("test").setPreference("_only_nodes:"+node).setQuery(matchAllQuery()).get(), totalDocs);


Spacing here:

:"+node -> :" + node

jasontedor · 2017-10-04T11:19:07Z

core/src/test/java/org/elasticsearch/search/basic/SearchWhileCreatingIndexIT.java

-        SearchResponse searchResponse = client().prepareSearch("test").setQuery(QueryBuilders.termQuery("field", "test")).execute().actionGet();
-        assertHitCount(searchResponse, 1);
+
+        for(String node : internalCluster().nodesInclude("test")){


Spacing here:

for( -> for ( )){ -> )) {

jasontedor · 2017-10-04T11:19:49Z

qa/full-cluster-restart/src/test/java/org/elasticsearch/upgrades/FullClusterRestartIT.java

-            assertEquals(foundHits1, foundHits2);
-            // TODO: do something more with the replicas! index?
+            Set<Integer> counts = new HashSet<>();
+            for(String node : dataNodes(index, client())){


Spacing here:

for( -> for ( )){ -> )) {

jasontedor · 2017-10-04T11:20:05Z

qa/full-cluster-restart/src/test/java/org/elasticsearch/upgrades/FullClusterRestartIT.java

+            Set<Integer> counts = new HashSet<>();
+            for(String node : dataNodes(index, client())){
+                Map<String, Object> responseBody = toMap(client().performRequest("GET", "/" + index + "/_search",
+                    Collections.singletonMap("preference", "_only_nodes:"+node)));


Spacing here:

:"+node -> :" + node

jasontedor · 2017-10-04T11:20:53Z

qa/mixed-cluster/src/test/java/org/elasticsearch/backwards/IndexingIT.java

-            assertCount(index, "_primary", numDocs);
-            assertCount(index, "_replica", numDocs);
+
+            for(Shard shard : buildShards(index, nodes, newNodeClient)){


Spacing:

for( -> for ( )){ -> )) {

dnhatn · 2017-10-04T20:01:47Z

@jasontedor, I have addressed your comments. Please have a look. Thank you.

jasontedor

Two more nits and it looks good.

jasontedor · 2017-10-04T21:42:43Z

docs/reference/migration/migrate_7_0/cluster.asciidoc

@@ -11,3 +11,6 @@ cluster names may no longer contain `:`.
 The default value for the `wait_for_active_shards` parameter of the open index API
 is changed from 0 to 1, which means that the command will now by default wait for all
 primary shards of the opened index to be allocated.
+
+==== shard preferences `_primary`, `_primary_first`, `_replica`, and `_replica_first` are removed


Nit: capitalize shard here as it's a section title.

jasontedor · 2017-10-04T21:43:28Z

core/src/test/java/org/elasticsearch/search/preference/SearchPreferenceIT.java

@@ -67,7 +66,7 @@ public void testStopOneNodePreferenceWithRedState() throws InterruptedException,
        refresh();
        internalCluster().stopRandomDataNode();
        client().admin().cluster().prepareHealth().setWaitForStatus(ClusterHealthStatus.RED).execute().actionGet();
-        String[] preferences = new String[] {"_primary", "_local", "_primary_first", "_prefer_nodes:somenode", "_prefer_nodes:server2", "_prefer_nodes:somenode,server2"};
+        String[] preferences = new String[] {"_local", "_prefer_nodes:somenode", "_prefer_nodes:server2", "_prefer_nodes:somenode,server2"};


Nit: [] { -> []{.

jasontedor · 2017-10-04T22:42:59Z

@dnhatn Please add an area label, a type label, and version labels to this PR.

jasontedor · 2017-10-07T17:30:28Z

@dnhatn I don’t think this should have the deprecation label.

jasontedor

LGTM.

dnhatn · 2017-10-08T15:03:51Z

Thanks @jasontedor.

* master: update Lucene version for 6.0-RC2 version Calculate and cache result when advanceExact is called (elastic#26920) Test query builder bwc against previous supported versions instead of just the current version. Set minimum_master_nodes on rolling-upgrade test (elastic#26911) Return List instead of an array from settings (elastic#26903) remove _primary and _replica shard preferences (elastic#26791) fixing typo in datehistogram-aggregation.asciidoc (elastic#26924) [API] Added the `terminate_after` parameter to the REST spec for "Count" API Setup debug logging for qa.full-cluster-restart Enable BWC testing against other remotes

the only nodes preference was used as a replacement of `_primary` which was removed. Sadly, it's not the same as we also check that it makes sense - i.e., that the given node has a shard copy. Since the test uses indices with >1 shards, the primaries may be spread to multiple nodes. Using one (like it currently does) will fail for some primaries. Using all will probably end up hitting all nodes. This commit removed the `_only_nodes` usage in favor a simple search Relates to #26791

* master: (22 commits) Allow only a fixed-size receive predictor (elastic#26165) Add Homebrew instructions to getting started ingest: Fix bug that prevent date_index_name processor from accepting timestamps specified as a json number Scripting: Fix expressions to temporarily support filter scripts (elastic#26824) Docs: Add note to contributing docs warning against tool based refactoring (elastic#26936) Fix thread context handling of headers overriding (elastic#26068) SearchWhileCreatingIndexIT: remove usage of _only_nodes update Lucene version for 6.0-RC2 version Calculate and cache result when advanceExact is called (elastic#26920) Test query builder bwc against previous supported versions instead of just the current version. Set minimum_master_nodes on rolling-upgrade test (elastic#26911) Return List instead of an array from settings (elastic#26903) remove _primary and _replica shard preferences (elastic#26791) fixing typo in datehistogram-aggregation.asciidoc (elastic#26924) [API] Added the `terminate_after` parameter to the REST spec for "Count" API Setup debug logging for qa.full-cluster-restart Enable BWC testing against other remotes Use LF line endings in Painless generated files (elastic#26822) [DOCS] Added info about snapshotting your data before an upgrade. Add documentation about disabling `_field_names`. (elastic#26813) ...

yznima · 2018-06-18T23:42:13Z

@dnhatn

The shard preference _primary, _replica and its variants were used for the asynchronous replication.

Based on the PR message, I assume that the replication has been asynchronous at some previous version. Is there a specific version or PR that the replication model has been changed in?

Today it is unclear what guarantees are offered by the search preference feature, and we claim a guarantee that is stronger than what we really offer: > A custom value will be used to guarantee that the same shards will be used > for the same custom value. This commit clarifies this documentation and explains more clearly why `_primary`, `_replica`, etc. are deprecated in `6.x` and removed in `master`. Relates elastic#31929 elastic#26335 elastic#26791.

Today it is unclear what guarantees are offered by the search preference feature, and we claim a guarantee that is stronger than what we really offer: > A custom value will be used to guarantee that the same shards will be used > for the same custom value. This commit clarifies this documentation and explains more clearly why `_primary`, `_replica`, etc. are deprecated in `6.x` and removed in `master`. Relates #31929 #26335 #26791.

Today the `?preference=custom_string_value` search preference will only change its choice of a shard copy if something changes the `IndexShardRoutingTable` for that specific shard. Users can use this behaviour to route searches to a consistent set of shard copies, which means they can reliably hit copies with hot caches, and use the other copies only for redundancy in case of failure. However we do not assert this property anywhere, so we might break it in future. This commit adds a test that shows that searches are routed consistently even if other indices are created/rebalanced/deleted. Relates https://discuss.elastic.co/t/176598, elastic#41115, elastic#26791

Today the `?preference=custom_string_value` search preference will only change its choice of a shard copy if something changes the `IndexShardRoutingTable` for that specific shard. Users can use this behaviour to route searches to a consistent set of shard copies, which means they can reliably hit copies with hot caches, and use the other copies only for redundancy in case of failure. However we do not assert this property anywhere, so we might break it in future. This commit adds a test that shows that searches are routed consistently even if other indices are created/rebalanced/deleted. Relates https://discuss.elastic.co/t/176598, #41115, #26791

Today the `?preference=custom_string_value` search preference will only change its choice of a shard copy if something changes the `IndexShardRoutingTable` for that specific shard. Users can use this behaviour to route searches to a consistent set of shard copies, which means they can reliably hit copies with hot caches, and use the other copies only for redundancy in case of failure. However we do not assert this property anywhere, so we might break it in future. This commit adds a test that shows that searches are routed consistently even if other indices are created/rebalanced/deleted. Relates https://discuss.elastic.co/t/176598, elastic#41115, elastic#26791

Relates: elastic/elasticsearch#26791

remove _primary and _replica shard preferences

3edc889

The shard preference `_primary`, `_replica` and its variants were useful for the asynchronous replication. However, with the current impl, they are no longer useful and should be removed. Closes elastic#26335

dnhatn requested a review from jasontedor September 26, 2017 14:19

truncate a long line

adc7aa0

jasontedor reviewed Oct 4, 2017

View reviewed changes

dnhatn added 3 commits October 4, 2017 09:46

use expectThrows

c442730

adjust code format

e5b2b24

add migration note

8866933

dnhatn added the >breaking label Oct 4, 2017

dnhatn added 2 commits October 4, 2017 12:47

loosen the preferences with _prefer_nodes

0460a0f

Merge branch 'master' into remove_shard_preferences

5f2cd43

jasontedor reviewed Oct 4, 2017

View reviewed changes

address comments

9c022a6

dnhatn added v7.0.0 :Search/Search Search-related issues that do not fall into other categories >deprecation labels Oct 4, 2017

jasontedor approved these changes Oct 7, 2017

View reviewed changes

dnhatn removed the >deprecation label Oct 8, 2017

dnhatn merged commit bf4c364 into elastic:master Oct 8, 2017

dnhatn deleted the remove_shard_preferences branch October 9, 2017 00:25

ywelsch mentioned this pull request Oct 9, 2017

[CI] SearchWhileCreatingIndexIT#testNoReplicas fails assertion, hits count is 0 but 1 was expected #25053

Closed

clintongormley changed the title ~~remove _primary and _replica shard preferences (#26335)~~ Remove _primary and _replica shard preferences Oct 9, 2017

DaveCTurner mentioned this pull request Jul 16, 2018

Improve docs for search preferences #32098

Merged

colings86 added v7.0.0-beta1 and removed v7.0.0 labels Feb 7, 2019

DaveCTurner mentioned this pull request Apr 12, 2019

Assert the stability of custom search preferences #41150

Merged

russcam added a commit to elastic/elasticsearch-net that referenced this pull request Jun 27, 2019

Remove _primary and _replica execution preferences

26626b6

Relates: elastic/elasticsearch#26791

russcam mentioned this pull request Jun 27, 2019

Remove _primary and _replica execution preferences elastic/elasticsearch-net#3884

Merged

Mpdreamz pushed a commit to elastic/elasticsearch-net that referenced this pull request Jun 27, 2019

Remove _primary and _replica execution preferences (#3884)

a79ac59

Relates: elastic/elasticsearch#26791

mattweber mentioned this pull request Jan 27, 2023

[DISCUSS] Add back preference for searching _primaries or _replicas opensearch-project/OpenSearch#6046

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove _primary and _replica shard preferences #26791

Remove _primary and _replica shard preferences #26791

dnhatn commented Sep 26, 2017

jasontedor left a comment

jasontedor Oct 4, 2017

jasontedor Oct 4, 2017

jasontedor Oct 4, 2017

jasontedor Oct 4, 2017

jasontedor Oct 4, 2017

jasontedor Oct 4, 2017

jasontedor Oct 4, 2017

jasontedor Oct 4, 2017

dnhatn commented Oct 4, 2017

jasontedor left a comment

jasontedor Oct 4, 2017

jasontedor Oct 4, 2017

jasontedor commented Oct 4, 2017

jasontedor commented Oct 7, 2017

jasontedor left a comment

dnhatn commented Oct 8, 2017

yznima commented Jun 18, 2018

Remove _primary and _replica shard preferences #26791

Remove _primary and _replica shard preferences #26791

Conversation

dnhatn commented Sep 26, 2017

jasontedor left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dnhatn commented Oct 4, 2017

jasontedor left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jasontedor commented Oct 4, 2017

jasontedor commented Oct 7, 2017

jasontedor left a comment

Choose a reason for hiding this comment

dnhatn commented Oct 8, 2017

yznima commented Jun 18, 2018