Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Forbid negative query scores #33309

Closed
5 tasks done
jimczi opened this issue Aug 31, 2018 · 1 comment
Closed
5 tasks done

Forbid negative query scores #33309

jimczi opened this issue Aug 31, 2018 · 1 comment
Labels
blocker >breaking :Search Relevance/Ranking Scoring, rescoring, rank evaluation. Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch v7.0.0-beta1

Comments

@jimczi
Copy link
Contributor

jimczi commented Aug 31, 2018 •
edited
Loading

Negative scores are not allowed in Lucene 8 so we should make sure that we don't produce any deliberately. The following is an (hopefully) exhaustive list of places where negative values should be forbidden in es 7 and deprecated in 6x:

Query and field boosts are trivial, negative values can be detected during the parsing of the query. For similarities and scripts we cannot infer easily if the values can be negative or not so we'll have to check scores during query execution and fail the request if a negative score is produced.
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search-aggs

@jimczi jimczi mentioned this issue Aug 31, 2018
Merged
jimczi added a commit that referenced this issue Sep 6, 2018
@jimczi
Upgrade to a Lucene 8 snapshot (#33310)
7ad71f9 
The main benefit of the upgrade for users is the search optimization for top scored documents when the total hit count is not needed. However this optimization is not activated in this change, there is another issue opened to discuss how it should be integrated smoothly.
Some comments about the change:
* Tests that can produce negative scores have been adapted but we need to forbid them completely: #33309

Closes #32899
jpountz added a commit to jpountz/elasticsearch that referenced this issue Sep 10, 2018
@jpountz
Add minimal sanity checks to custom/scripted similarities.
d161e9a 
Lucene 8 introduced more constraints on similarities, in particular:
 - scores must not be negative,
 - scores must not decrease when term freq increases,
 - scores must not increase when norm (interpreted as an unsigned long)
   increases.

We can't check every single case, but could at least run some sanity checks.

Relates elastic#33309
@jpountz jpountz mentioned this issue Sep 10, 2018
Merged
jpountz added a commit that referenced this issue Sep 19, 2018
@jpountz
Add minimal sanity checks to custom/scripted similarities. (#33564)
c4261ba 
Add minimal sanity checks to custom/scripted similarities.

Lucene 8 introduced more constraints on similarities, in particular:
 - scores must not be negative,
 - scores must not decrease when term freq increases,
 - scores must not increase when norm (interpreted as an unsigned long)
   increases.

We can't check every single case, but could at least run some sanity checks.

Relates #33309
jimczi added a commit to jimczi/elasticsearch that referenced this issue Oct 15, 2018
@jimczi
Disallow negative query boost
b5553d1 
This change disallows negative query boosts. Negative scores are not allowed in Lucene 8 so
it is easier to just disallow negative boosts entirely. We should also deprecate negative boosts
in 6x in order to ensure that users are aware when they'll upgrade to ES 7.

Relates elastic#33309
@jimczi jimczi mentioned this issue Oct 15, 2018
Merged
jimczi added a commit that referenced this issue Oct 16, 2018
@jimczi
Disallow negative query boost (#34486)
544de13 
This change disallows negative query boosts. Negative scores are not allowed in Lucene 8 so
it is easier to just disallow negative boosts entirely. We should also deprecate negative boosts
in 6x in order to ensure that users are aware when they'll upgrade to ES 7.

Relates #33309
jimczi added a commit to jimczi/elasticsearch that referenced this issue Oct 16, 2018
@jimczi
Deprecate negative query boost (elastic#34486)
45b9bc3 
This change deprecates negative query boosts. Negative scores are not allowed in Lucene 8 so
it is easier to just disallow negative boosts entirely.

Relates elastic#33309
@jimczi jimczi mentioned this issue Oct 16, 2018
Merged
jimczi added a commit that referenced this issue Oct 16, 2018
@jimczi
Deprecate negative query boost (#34486) (#34512)
679bc36 
This change deprecates negative query boosts. Negative scores are not allowed in Lucene 8 so
it is easier to just disallow negative boosts entirely.

Relates #33309
@colings86 colings86 added v6.6.0 and removed v6.5.0 labels Oct 25, 2018
kcm pushed a commit that referenced this issue Oct 30, 2018
@jimczi @kcm
Disallow negative query boost (#34486)
305d209 
This change disallows negative query boosts. Negative scores are not allowed in Lucene 8 so
it is easier to just disallow negative boosts entirely. We should also deprecate negative boosts
in 6x in order to ensure that users are aware when they'll upgrade to ES 7.

Relates #33309
mayya-sharipova added a commit to mayya-sharipova/elasticsearch that referenced this issue Nov 19, 2018
@mayya-sharipova
Forbid negative scores in functon_score query
a396fb3 
- Throw an exception when scores are negative in field_value_factor
function
- Throw an exception when scores are negative in script_score
function

Relates to elastic#33309
@mayya-sharipova mayya-sharipova mentioned this issue Nov 19, 2018
Merged
mayya-sharipova added a commit that referenced this issue Nov 22, 2018
@mayya-sharipova
Forbid negative scores in functon_score query (#35709)
b6014d9 
* Forbid negative scores in functon_score query

- Throw an exception when scores are negative in field_value_factor
function
- Throw an exception when scores are negative in script_score
function

Relates to #33309
original-brownbear pushed a commit that referenced this issue Nov 23, 2018
@mayya-sharipova @original-brownbear
Forbid negative scores in functon_score query (#35709)
3f79476 
* Forbid negative scores in functon_score query

- Throw an exception when scores are negative in field_value_factor
function
- Throw an exception when scores are negative in script_score
function

Relates to #33309
@mayya-sharipova mayya-sharipova mentioned this issue Nov 23, 2018
Merged
mayya-sharipova added a commit that referenced this issue Nov 23, 2018
@mayya-sharipova
Deprecate negative scores in functon_score query (#35865)
3132c29 
Relates to #33309, backport for #35709
@jimczi jimczi removed the v6.6.0 label Dec 27, 2018
@codebrain codebrain mentioned this issue Jan 25, 2019
Closed
48 tasks
jimczi added a commit to jimczi/elasticsearch that referenced this issue Jan 28, 2019
@jimczi
Forbid negative field boosts in analyzed queries
89e57f5 
This change forbids negative field boost in the `query_string`, `simple_query_string`
and `multi_match` queries.
Negative boosts are not allowed in Lucene 8 (scores must be positive).
The backport of this change to 6x will turn the error into a deprecation warning
in order to raise the awareness of this breaking change in 7.0.

Closes elastic#33309
@jimczi jimczi mentioned this issue Jan 28, 2019
Merged
jimczi added a commit that referenced this issue Feb 1, 2019
@jimczi
Forbid negative field boosts in analyzed queries (#37930)
6fa93ca 
This change forbids negative field boost in the `query_string`, `simple_query_string`
and `multi_match` queries.
Negative boosts are not allowed in Lucene 8 (scores must be positive).
The backport of this change to 6x will turn the error into a deprecation warning
in order to raise the awareness of this breaking change in 7.0.

Closes #33309
jimczi added a commit to jimczi/elasticsearch that referenced this issue Feb 1, 2019
@jimczi
Forbid negative field boosts in analyzed queries
bf9bd0d 
This change forbids negative field boost in the `query_string`, `simple_query_string`
and `multi_match` queries.
Negative boosts are not allowed in Lucene 8 (scores must be positive).
The backport of this change to 6x will turn the error into a deprecation warning
in order to raise the awareness of this breaking change in 7.0.

Closes elastic#33309
jimczi added a commit that referenced this issue Feb 1, 2019
@jimczi
Deprecate negative field boosts in analyzed queries (#38176)
51791f9 
This change forbids negative field boost in the `query_string`, `simple_query_string`
and `multi_match` queries.
Negative boosts are not allowed in Lucene 8 (scores must be positive).
The backport of this change to 6x will turn the error into a deprecation warning
in order to raise the awareness of this breaking change in 7.0.

Closes #33309
@dschneiter dschneiter mentioned this issue Apr 24, 2019
Closed
@javanna javanna added the Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch label Jul 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
blocker >breaking :Search Relevance/Ranking Scoring, rescoring, rank evaluation. Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch v7.0.0-beta1
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@colings86 @javanna @elasticmachine @jimczi