Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid negative scores with cross_fields type #89016

Merged
merged 6 commits into from
Sep 6, 2022

Commits on Aug 4, 2022

  1. Avoid negative scores with cross_fields type

    The cross_fields scoring type can produce negative scores when some documents
    are missing fields. When blending term document frequencies, we take the maximum
    document frequency across all fields. If one field appears in fewer documents
    than another, this means that its IDF can become negative. This is because IDF
    is calculated as `Math.log(1 + (docCount - docFreq + 0.5) / (docFreq + 0.5))`
    
    This change adjusts the docFreq for each field to `Math.min(docCount, docFreq)`
    so that the IDF can never become negative. It makes sense that the term document
    frequency should never exceed the number of documents containing the field.
    jtibshirani committed Aug 4, 2022
    Configuration menu
    Copy the full SHA
    78d81e7 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    18d6573 View commit details
    Browse the repository at this point in the history
  3. Fix docs typo

    jtibshirani committed Aug 4, 2022
    Configuration menu
    Copy the full SHA
    50a2790 View commit details
    Browse the repository at this point in the history

Commits on Aug 30, 2022

  1. Configuration menu
    Copy the full SHA
    04b1d77 View commit details
    Browse the repository at this point in the history

Commits on Sep 6, 2022

  1. Configuration menu
    Copy the full SHA
    4669432 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    6853a91 View commit details
    Browse the repository at this point in the history