Skip to content

Commit

Permalink
(Doc+) Error "number of documents in the index can't exceed" (elastic…
Browse files Browse the repository at this point in the history
…#110449)

* (Doc+) Error "number of documents in the index can't exceed"

👋 howdy, team! 

This adds resolution outline for error ... which induces ongoing, lowkey support
```
Number of documents in the index can't exceed [2147483519]
```

* feedback

* feedback

Co-authored-by: David Turner <david.turner@elastic.co>

* feedback

Co-authored-by: David Turner <david.turner@elastic.co>
Co-authored-by: Liam Thompson <32779855+leemthompo@users.noreply.github.com>

* feedback

* feedback

* Test change to address docs check failure

* Revert test change

* Test docs check

---------

Co-authored-by: David Turner <david.turner@elastic.co>
Co-authored-by: Liam Thompson <32779855+leemthompo@users.noreply.github.com>
  • Loading branch information
3 people committed Jul 16, 2024
1 parent e946fa8 commit 3ce35e3
Showing 1 changed file with 33 additions and 0 deletions.
33 changes: 33 additions & 0 deletions docs/reference/how-to/size-your-shards.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -544,3 +544,36 @@ PUT _cluster/settings
}
}
----

[discrete]
==== Number of documents in the shard cannot exceed [2147483519]


Elasticsearch shards reflect Lucene's underlying https://github.com/apache/lucene/issues/5176[index
`MAX_DOC` hard limit] of 2,147,483,519 (`(2^31)-129`) docs. This figure is
the sum of `docs.count` plus `docs.deleted` as reported by the <<indices-stats,Index stats API>>
per shard. Exceeding this limit will result in errors like the following:

[source,txt]
----
Elasticsearch exception [type=illegal_argument_exception, reason=Number of documents in the shard cannot exceed [2147483519]]
----

TIP: This calculation may differ from the <<search-count,Count API's>> calculation, because the Count API does not include nested documents.


Try using the <<indices-forcemerge,Force Merge API>> to clear deleted docs. For example:

[source,console]
----
POST my-index-000001/_forcemerge?only_expunge_deletes=true
----
// TEST[setup:my_index]

This will launch an asynchronous task which can be monitored via the <<tasks,Task Management API>>.

For a long-term solution try:

* <<docs-delete-by-query,deleting unneeded documents>>
* aligning the index to recommendations on this page by either
<<indices-split-index,Splitting>> or <<docs-reindex,Reindexing>> the index

0 comments on commit 3ce35e3

Please sign in to comment.