-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This PR adds preliminary support for running XTR models in LintDB. - Tokenizer runs sentencepiece and adds an eos token for XTR. - Inverted Lists change to index codes per token for XTR. Therefore, XTR makes one less database call to score. - ProductEncoder handles quantization, with the help of new inverted list scanners and distance tables. This touches a lot of code to help find better abstractions.
- Loading branch information
There are no files selected for viewing
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.