[elasticsearch] Why is default number of primary shards 1?

[naseemkullah]

Curious since according to google searches it is 5 by default but seems to be 1 in the helm chart?

Also, I ask because we have noticed a significant increase in performance correlated with the increase of shard count from 1 to 8 (which in our case makes our shards ~50GB each).

[nvtkaszpir]

https://www.elastic.co/guide/en/elasticsearch/reference/current/breaking-changes-7.0.html#_index_creation_no_longer_defaults_to_five_shards

please read detailed description here elastic/elasticsearch#30539

[naseemkullah]

Thank you!

[nvtkaszpir]

one more thing, more human-readable explanation is at https://www.elastic.co/blog/elasticsearch-7-0-0-released - at section:

Default to one shard

One of the biggest sources of troubles we've seen over the years from
our users has been oversharding --- and defaults play a big role in that.
In Elasticsearch 6.x and prior, we defaulted to 5 shards by default per index.
If you had one daily index for 10 different applications and each had the
default of 5 shards, you were creating 50 shards per day and it wasn't long
before you had thousands of shards even if you were only indexing a few
gigabytes of data per day. Index lifecycle management (ILM) was a first
step to help with this: providing native rollover functions to create
indexes by size instead of (just) by day and built-in shrink functionality
to shrink the number of shards per index. Defaulting indices to 1 shard
is the next step in helping to reduce oversharding. Of course, if you have
another preferred primary shard count, you can set it via the index settings.

[naseemkullah]

Ah thanks for focusing in on that! Love a good tl;dr