-
Notifications
You must be signed in to change notification settings - Fork 25k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Allocation Index Priority #11787
Labels
:Distributed Indexing/Recovery
Anything around constructing a new shard, either from a local or a remote source.
>feature
release highlight
v1.7.0
v2.0.0-beta1
Comments
pickypg
added
discuss
:Core/Infra/Settings
Settings infrastructure and APIs
:Allocation
:Distributed Indexing/Recovery
Anything around constructing a new shard, either from a local or a remote source.
:Data Management/Indices APIs
APIs to create and manage indices and templates
labels
Jun 20, 2015
This is a duplicate of #10069, but I had been thinking about it a bit over the course of the past few weeks and I wanted to write it down. We can close this one in favor of that one (or vice versa), but I wanted to get it written down. |
clintongormley
added
>feature
and removed
:Allocation
:Data Management/Indices APIs
APIs to create and manage indices and templates
:Core/Infra/Settings
Settings infrastructure and APIs
labels
Jun 23, 2015
I like the defaults proposed here :) |
s1monw
added a commit
to s1monw/elasticsearch
that referenced
this issue
Jul 6, 2015
This commit adds logic to prefer shards with higher priority or from newer indicse to be allocated first if they are unallocated post API. This commit allows users to set `index.priority` to a non-negative integer to prioritize index recovery for certain indices. This setting is dynamically updateable and defaults to `0`. If two indices have the same priority this change takes the creation date into account to prioritize shards from newer indices which is important in the time-based indices usecase. Closes elastic#11787
s1monw
added a commit
that referenced
this issue
Jul 7, 2015
This commit adds logic to prefer shards with higher priority or from newer indicse to be allocated first if they are unallocated post API. This commit allows users to set `index.priority` to a non-negative integer to prioritize index recovery for certain indices. This setting is dynamically updateable and defaults to `0`. If two indices have the same priority this change takes the creation date into account to prioritize shards from newer indices which is important in the time-based indices usecase. Closes #11787
FYI pushed the #11975 to 1.x and master |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
:Distributed Indexing/Recovery
Anything around constructing a new shard, either from a local or a remote source.
>feature
release highlight
v1.7.0
v2.0.0-beta1
The idea is allow a simple, dynamically modifiable priority for indices. Something akin to:
By default, all priorities would default with
index.priority : 0
, thus being of equal priority unless modified. From there, the comparison could go to index creation date/time as the next priority level, then finally by name for the final order.This allows pretty much every use case to specify recovery order for best-case recovery order for their use case. Time-based solutions would pretty much work out of the box because the creation date would prioritize recovery of today's index first, thus giving the best chance to return to fully replicated indexing as soon as possible. Similarly, falling back to the reverse sorted index name gives arbitrary--yet predictable--sorting for simultaneously-created indices (however unlikely) that have no time basis (
index-z
,index-y
, ...), but YMD-formatted date/time strings will properly sort (logstash-2015.05.03
will always come beforelogstash-2015.05.02
or earlier dates). Finally, the arbitrary priority allows users to take control of the process for cases where there is no time basis, such as when a "main" index is created, but then minor indices are created from time-to-time. Even better, this would be as simple as setting it once with a high value, then not setting or selectively setting the minor indices.The text was updated successfully, but these errors were encountered: