Skip to content

Commit

Permalink
Add limits for ngram and shingle settings (elastic#27211)
Browse files Browse the repository at this point in the history
Relates to elastic#25887
  • Loading branch information
mayya-sharipova committed Nov 8, 2017
1 parent df5c8bb commit 9a4c34a
Showing 1 changed file with 18 additions and 0 deletions.
18 changes: 18 additions & 0 deletions docs/reference/migration/migrate_7_0/indices.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,21 @@ had the undocumented side-effect of refreshing the index which made new document
visible to searches and non-realtime GET operations. From now on these operations
don't have this side-effect anymore. To make documents visible an explicit `_refresh`
call is needed unless the index is refreshed by the internal scheduler.


==== Limit to the difference between max_size and min_size in NGramTokenFilter and NGramTokenizer

To safeguard against creating too many index terms, the difference between `max_ngram` and
`min_ngram` in `NGramTokenFilter` and `NGramTokenizer` has been limited to 1. This default
limit can be changed with the index setting `index.max_ngram_diff`. Note that if the limit is
exceeded a error is thrown only for new indices. For existing pre-7.0 indices, a deprecation
warning is logged.


==== Limit to the difference between max_size and min_size in ShingleTokenFilter

To safeguard against creating too many tokens, the difference between `max_shingle_size` and
`min_shingle_size` in `ShingleTokenFilter` has been limited to 3. This default
limit can be changed with the index setting `index.max_shingle_diff`. Note that if the limit is
exceeded a error is thrown only for new indices. For existing pre-7.0 indices, a deprecation
warning is logged.

0 comments on commit 9a4c34a

Please sign in to comment.