Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Only check inactive replicas in cancelling existing recoveries. #60564

Merged
merged 3 commits into from
Sep 3, 2021

Conversation

howardhuanghua
Copy link
Contributor

@howardhuanghua howardhuanghua commented Aug 1, 2020

Currently we check and process existing recoveries once we have INITIALIZING shards. ReplicaShardAllocator.processExistingRecoveries would be expensive as it will for-loop all the shards overall the nodes. Actually we only need to do this checking if we do have replicas in INITIALIZING state. If we only have primaries in this state, we could skip. For example:

  1. Creating a index, allocate primaries first, some of the primaries got started first would trigger reroute, now would still have some INITIALIZING primaries, we don't need to process existing recoveries, as no replicas would be allocated at the begining.
  2. Creating index that only have primries no replica, we also don't need to process existing recoveies in the whole allocation process.

@howardhuanghua howardhuanghua changed the title Only check replicas in cancelling existing recoveries. Only check inactive replicas in cancelling existing recoveries. Aug 1, 2020
@dliappis dliappis added the :Distributed Coordination/Allocation All issues relating to the decision making around placing a shard (both master logic & on the nodes) label Aug 3, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed (:Distributed/Allocation)

@elasticmachine elasticmachine added the Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. label Aug 3, 2020
@dliappis
Copy link
Contributor

dliappis commented Aug 3, 2020

Related to #46520

@henningandersen
Copy link
Contributor

@elasticmachine update branch

@elasticmachine
Copy link
Collaborator

user doesn't have permission to update head repository

@henningandersen
Copy link
Contributor

@elasticmachine test this please

@henningandersen henningandersen self-requested a review September 1, 2021 12:41
@howardhuanghua
Copy link
Contributor Author

Branch has been updated. Thank you @henningandersen .

@howardhuanghua
Copy link
Contributor Author

Hi @henningandersen would you please help to check this PR again? Thank you.

@henningandersen
Copy link
Contributor

@elasticmachine test this please

@howardhuanghua
Copy link
Contributor Author

Merged latest master.

@henningandersen
Copy link
Contributor

@elasticmachine test this please

Copy link
Contributor

@henningandersen henningandersen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for your contribution.

@henningandersen henningandersen merged commit 962e050 into elastic:master Sep 3, 2021
henningandersen pushed a commit that referenced this pull request Sep 3, 2021
Minor performance improvement when only primaries are initializing.
wjp719 added a commit to wjp719/elasticsearch that referenced this pull request Sep 4, 2021
* master: (128 commits)
  Mute DieWithDignityIT (elastic#77283)
  Fix randomization in MlNodeShutdownIT (elastic#77281)
  Add target_node_name for REPLACE shutdown type (elastic#77151)
  [DOCS] Adds information about version compatibility headers (elastic#77096)
  Fix template equals when mappings are wrapped (elastic#77008)
  Fix TextFieldMapper Retaining a Reference to its Builder (elastic#77251)
  Move die with dignity to be a test module (elastic#77136)
  Update task names for rest compatiblity (elastic#75267)
  [ML] adjusting bwc serialization for elastic#77256 (elastic#77257)
  Move `index.hidden` from Static to Dynamic settings (elastic#77218)
  Handle cgroups v2 in `OsProbe` (elastic#77128)
  Choose postings format from FieldMapper instead of MappedFieldType (elastic#77234)
  Add segment sorter for data streams (elastic#75195)
  Update skip after backport (elastic#77212)
  [ML] adding new defer_definition_decompression parameter to put trained model API (elastic#77189)
  [ML] Fix bug in inference stats persister for when feature reset is called
  Only check replicas in cancelling existing recoveries. (elastic#60564)
  Format `AbstractFilteringTestCase` (elastic#77217)
  [DOCS] Fixes line breaks. (elastic#77248)
  Convert 'routing' values in REST API tests to strings
  ...

# Conflicts:
#	server/src/main/java/org/elasticsearch/cluster/metadata/DataStream.java
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Distributed Coordination/Allocation All issues relating to the decision making around placing a shard (both master logic & on the nodes) Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants