Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve docs for disk watermarks #30249

Merged
merged 2 commits into from
Apr 30, 2018
Merged

Improve docs for disk watermarks #30249

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
d7197e9
Improve docs for disk watermarks
DaveCTurner Apr 30, 2018
83e62fa
Reword exclusion for new shards
DaveCTurner Apr 30, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 28 additions & 25 deletions docs/reference/modules/cluster/disk_allocator.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
[[disk-allocator]]
=== Disk-based Shard Allocation

Elasticsearch factors in the available disk space on a node before deciding
whether to allocate new shards to that node or to actively relocate shards
away from that node.
Elasticsearch considers the available disk space on a node before deciding
whether to allocate new shards to that node or to actively relocate shards away
from that node.

Below are the settings that can be configured in the `elasticsearch.yml` config
file or updated dynamically on a live cluster with the
Expand All @@ -15,29 +15,32 @@ file or updated dynamically on a live cluster with the

`cluster.routing.allocation.disk.watermark.low`::

Controls the low watermark for disk usage. It defaults to 85%, meaning ES will
not allocate new shards to nodes once they have more than 85% disk used. It
can also be set to an absolute byte value (like 500mb) to prevent ES from
allocating shards if less than the configured amount of space is available.
Controls the low watermark for disk usage. It defaults to `85%`, meaning
that Elasticsearch will not allocate shards to nodes that have more than
85% disk used. It can also be set to an absolute byte value (like `500mb`)
to prevent Elasticsearch from allocating shards if less than the specified
amount of space is available. This setting has no effect on brand-new
shards that have never been allocated before.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To make this a little bit more explicit, I would add something along the lines of ", for example when a new index is created".

Copy link
Contributor Author

@DaveCTurner DaveCTurner Apr 30, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed it to

This setting has no effect on the primary shards of newly-created indices or, specifically, any shards that have never previously been allocated.

WDYT?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

++


`cluster.routing.allocation.disk.watermark.high`::

Controls the high watermark. It defaults to 90%, meaning ES will attempt to
relocate shards to another node if the node disk usage rises above 90%. It can
also be set to an absolute byte value (similar to the low watermark) to
relocate shards once less than the configured amount of space is available on
the node.
Controls the high watermark. It defaults to `90%`, meaning that
Elasticsearch will attempt to relocate shards away from a node whose disk
usage is above 90%. It can also be set to an absolute byte value (similarly
to the low watermark) to relocate shards away from a node if it has less
than the specified amount of free space. This setting affects the
allocation of all shards, whether previously allocated or not.

`cluster.routing.allocation.disk.watermark.flood_stage`::
+
--
Controls the flood stage watermark. It defaults to 95%, meaning ES enforces
a read-only index block (`index.blocks.read_only_allow_delete`) on every
index that has one or more shards allocated on the node that has at least
one disk exceeding the flood stage. This is a last resort to prevent nodes
from running out of disk space. The index block must be released manually
once there is enough disk space available to allow indexing operations to
continue.
Controls the flood stage watermark. It defaults to 95%, meaning that
Elasticsearch enforces a read-only index block
(`index.blocks.read_only_allow_delete`) on every index that has one or more
shards allocated on the node that has at least one disk exceeding the flood
stage. This is a last resort to prevent nodes from running out of disk space.
The index block must be released manually once there is enough disk space
available to allow indexing operations to continue.

NOTE: You can not mix the usage of percentage values and byte values within
these settings. Either all are set to percentage values, or all are set to byte
Expand Down Expand Up @@ -67,12 +70,12 @@ PUT /twitter/_settings
`cluster.routing.allocation.disk.include_relocations`::

Defaults to +true+, which means that Elasticsearch will take into account
shards that are currently being relocated to the target node when computing a
node's disk usage. Taking relocating shards' sizes into account may, however,
mean that the disk usage for a node is incorrectly estimated on the high side,
since the relocation could be 90% complete and a recently retrieved disk usage
would include the total size of the relocating shard as well as the space
already used by the running relocation.
shards that are currently being relocated to the target node when computing
a node's disk usage. Taking relocating shards' sizes into account may,
however, mean that the disk usage for a node is incorrectly estimated on
the high side, since the relocation could be 90% complete and a recently
retrieved disk usage would include the total size of the relocating shard
as well as the space already used by the running relocation.


NOTE: Percentage values refer to used disk space, while byte values refer to
Expand Down