Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow one search shard to fall back to undesired allocation #113847

Merged

Conversation

pxsalehi
Copy link
Member

@pxsalehi pxsalehi commented Oct 1, 2024

We allow a primary shard to be assigned to a node even if it is not its desired location, but not for replicas. With this change we allow at most one search shard to also fall back to allocation to an undesired nodes if necessary. This ensures that at least one search replica will be available even if the desired node cannot allocate it.

Relates ES-9617

@pxsalehi pxsalehi added >non-issue :Distributed Coordination/Allocation All issues relating to the decision making around placing a shard (both master logic & on the nodes) labels Oct 1, 2024
@elasticsearchmachine elasticsearchmachine added serverless-linked Added by automation, don't add manually v9.0.0 labels Oct 1, 2024
@pxsalehi pxsalehi marked this pull request as ready for review October 1, 2024 11:03
@elasticsearchmachine elasticsearchmachine added the Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. label Oct 1, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-distributed (Team:Distributed)

@idegtiarenko
Copy link
Contributor

I think the approach is correct 👍
Do you mind adding a unit test? I believe DesiredBalanceReconcilerTests#testFallbackAllocation is doing something similar and also allows to specify the shard role (when adding unassigned shards manually specifying the role opposed to addAsNew). Please let me know if you need any help with it.

@pxsalehi
Copy link
Member Author

pxsalehi commented Oct 1, 2024

Do you mind adding a unit test?

It's in the linked test PR. To me it seemed we'd need StatelessAllocationDecider.

@pxsalehi pxsalehi requested a review from idegtiarenko October 1, 2024 12:00
@pxsalehi pxsalehi merged commit 60e8fb8 into elastic:main Oct 1, 2024
16 checks passed
matthewabbott pushed a commit to matthewabbott/elasticsearch that referenced this pull request Oct 4, 2024
…113847)

We allow a primary shard to be assigned to a node even if it is not its desired location, but not for replicas. With this change we allow at most one search shard to also fall back to allocation to an undesired nodes if necessary. This ensures that at least one search replica will be available even if the desired node cannot allocate it.

Relates ES-9617
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Distributed Coordination/Allocation All issues relating to the decision making around placing a shard (both master logic & on the nodes) >non-issue serverless-linked Added by automation, don't add manually Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. v9.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants