-
-
Notifications
You must be signed in to change notification settings - Fork 145
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add pgvector extension #472
Conversation
3e61c07
to
0a3e05d
Compare
Thanks for the PR!
I see in the cube docs
so we could potentially bump that cap over the standard embedding sizes to something like 4096 the other features I see in If there are other differences that are significant for common uses cases could you please add a few notes wrt what they are? |
+1 for pgvector. cube has not hada major update since 2006, if we patch it then we own it. pgvector is targeted directly at very common vector embedding use cases and is actively maintained. |
@supabase/backend please +1 and merge when you get a sec |
bump |
@olirice @Lakshmipathi Thanks for taking the time to review & thanks for the approvals. There were a few new merge conflicts as of yesterday, but I have resolved these now. |
thanks @gregnr, looks great! |
cheers |
What kind of change does this PR introduce?
Feature
What is the current behavior?
Popular ML extension
pgvector
is unavailable in Supabase. The closest related extension today iscube
, but its dimension limit (100) makes it impractical for modern vector math tasks.What is the new behavior?
Adds
pgvector
extension, one of the highest voted extensions on the Vote for Postgres extensions discussion.Additional context
As the language model ecosystem is increasing in popularity (eg. GPT-3), there is becoming a need for efficient vector operations (like vector similarity search) on large vector-based datasets (eg. GPT-3 Embeddings). The ability to use Postgres for this type of work vs. outsourcing to another system would make Supabase a compelling service offering for the AI/ML community.
This PR mirrors best practices and conventions used by other extensions to install
pgvector
through Ansible.Please let me know if there is any additional work I can do to help bring this extension to Supabase.