Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refactor vectors package #826

Closed
davidmezzetti opened this issue Dec 1, 2024 · 0 comments
Closed

Refactor vectors package #826

davidmezzetti opened this issue Dec 1, 2024 · 0 comments
Assignees
@davidmezzetti
Milestone
v8.1.0

Comments

@davidmezzetti
Copy link
Member

Make the following improvements to the vectors package.

  • Move tokenization logic in HFVectors.prepare to base vectors class into a method named tokens
  • Override WordVectors.tokens to skip tokenization
  • Move sentence-transformers functionality into it's own class
@davidmezzetti davidmezzetti added this to the v8.1.0 milestone Dec 1, 2024
@davidmezzetti davidmezzetti self-assigned this Dec 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@davidmezzetti davidmezzetti

Labels
None yet
Projects
None yet
Milestone
v8.1.0
Development

No branches or pull requests
1 participant
@davidmezzetti