Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add calculate-recall parameter to vector search and skip calculating recall if number clients > cpu cores #626

Merged

Conversation

finnroblin
Copy link
Contributor

@finnroblin finnroblin commented Aug 26, 2024

Description

When the number of search clients is greater than the number of cpu cores recall drops unexpectedly. @VijayanB is investigating the cause, but this PR throws a warning if num_clients > the cpu cores available and skips calculating the recall. It also adds a parameter calculate-recall to specify whether or not to calculate recall (either calculate-recall: true or calculate-recall: false).

Testing

  • New functionality includes testing

Unit tests.
Also tested on mac with 12 cores. Skips recall calculation if num_clients > 12, calculates recall otherwise. calculate-recall parameter works as expected. Will add unit tests in a second PR by EOD.


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Finn Roblin <finnrobl@amazon.com>
Signed-off-by: Finn Roblin <finnrobl@amazon.com>
@finnroblin finnroblin marked this pull request as ready for review August 27, 2024 22:54
Signed-off-by: Finn Roblin <finnrobl@amazon.com>
@IanHoang
Copy link
Collaborator

Had a sync with @VijayanB and @gkamat offline. We'll work with @VijayanB to better understand the issue and root cause it. Closing this as of no activity. Feel free to reopen if needed.

@IanHoang IanHoang closed this Oct 28, 2024
@IanHoang IanHoang reopened this Dec 4, 2024
@IanHoang IanHoang requested a review from OVI3D0 as a code owner December 4, 2024 18:14
@IanHoang IanHoang merged commit dcc4bf6 into opensearch-project:main Dec 5, 2024
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants