Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Enhancement]: Speed up Search Iterator and Cleanup SDKs #37548

Open
1 task done
PwzXxm opened this issue Nov 8, 2024 · 2 comments
Open
1 task done

[Enhancement]: Speed up Search Iterator and Cleanup SDKs #37548

PwzXxm opened this issue Nov 8, 2024 · 2 comments
Assignees
Labels
kind/enhancement Issues or changes related to enhancement

Comments

@PwzXxm
Copy link
Contributor

PwzXxm commented Nov 8, 2024

Is there an existing issue for this?

  • I have searched the existing issues

What would you like to be added?

Speed up search iterators via using iterator directly rather than calling range search and maintain radius and range_filter on the SDK, which adds complexity to SDK implementations.

Why is this needed?

E2E Search Iterator becomes fundamental to workflows like client-side post filtering and others.

Anything else?

No response

@PwzXxm PwzXxm added the kind/enhancement Issues or changes related to enhancement label Nov 8, 2024
@PwzXxm
Copy link
Contributor Author

PwzXxm commented Nov 8, 2024

Stage 1 (planning to ship before 2.5.1)

  1. Replace the underlying Range Search Calls with Knowhere iterator Next() calls. For each SDK Next() call, construct Knowhere iterators at segment level and calling Next() on them. Do NOT hold iterators at this stage.

Stage 2

  1. Introduce SearchIteratorManager to hold iterators on each QN.
  2. Limit the max number of iterator it can hold to limit the overall memory usage, modified LRU eviction.
  3. Introduce iterator timeout settings and functionality.

Stage 3

  1. Manually close search iterator, new gRPC call.
  2. Calls sticky to delegators in multi-shard.

@PwzXxm
Copy link
Contributor Author

PwzXxm commented Nov 8, 2024

/assign

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/enhancement Issues or changes related to enhancement
Projects
None yet
Development

No branches or pull requests

1 participant