Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use-case: Add adaptive retrieval use case #61

Merged
merged 2 commits into from
Mar 7, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 25 additions & 23 deletions .vitepress/config.mts
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ export default defineConfig({
nav: [
{ text: 'Home', link: '/' },
{ text: 'Docs', link: '/getting-started/overview' },
{text: 'Tutorial', link: '/tutorial/'},
{text: 'Reference', link: '/reference/'},
{ text: 'Blog', link: 'https://blog.pgvecto.rs' },
],

Expand All @@ -51,6 +51,17 @@ export default defineConfig({
},

sidebar: {
'/reference/': [
{
text: 'Reference',
collapsed: false,
items: [
{ text: 'Schema', link: '/reference/schema' },
{ text: 'Indexing Options', link: '/reference/indexing_options' },
{ text: 'Search Options', link: '/reference/search_options' },
],
},
],
'/': [
{
text: 'Getting Started',
Expand Down Expand Up @@ -92,26 +103,27 @@ export default defineConfig({
],
},
{
text: 'FAQs',
text: 'Use Cases',
collapsed: false,
items: [
{ text: 'pgvecto.rs vs. pgvector', link: '/faqs/comparison-pgvector' },
{ text: 'pgvecto.rs vs. specialized vectordb', link: '/faqs/comparison-with-specialized-vectordb' },
{ text: `Benchmarks`, link: '/faqs/benchmark' },
]
{ text: 'Hybrid search', link: '/use-case/hybrid-search' },
{ text: 'Shortening embedding', link: '/use-case/adaptive-retrieval' },
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to modify the title?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I cannot figure out a good title, I think there are few people know adaptive retrieval. Should we use Matryoshka embedding?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed to adaptive retrieval

{ text: 'Image search', link: '/use-case/image-search' },
{ text: 'Multi tenancy', link: '/use-case/multi-tenancy' },
],
},
{
text: 'Reference',
text: 'FAQs',
collapsed: false,
items: [
{ text: 'Schema', link: '/reference/schema' },
{ text: 'Indexing Options', link: '/reference/indexing_options' },
{ text: 'Search Options', link: '/reference/search_options' },
{ text: 'pgvecto.rs vs. pgvector', link: '/faqs/comparison-pgvector' },
{ text: 'pgvecto.rs vs. specialized vectordb', link: '/faqs/comparison-with-specialized-vectordb' },
{ text: `Benchmarks`, link: '/faqs/benchmark' },
]
},
{
text: 'Community',
collapsed: false,
collapsed: true,
items: [
{ text: 'Contributing to pgvecto.rs', link: '/community/contributing' },
{ text: 'Roadmap', link: '/community/roadmap' },
Expand All @@ -120,23 +132,13 @@ export default defineConfig({
},
{
text: 'Developers',
collapsed: false,
collapsed: true,
items: [
{ text: 'Development tutorial', link: '/developers/development' },
],
},
],
'/tutorial/': [
{
text: 'Use Cases',
collapsed: false,
items: [
{ text: 'Hybrid Search', link: '/tutorial/hybrid-search' },
{ text: 'Image Search', link: '/tutorial/image-search' },
{ text: 'Multi Tenancy', link: '/tutorial/multi-tenancy' },
],
},
]

},

socialLinks: [
Expand Down
8 changes: 4 additions & 4 deletions src/faqs/benchmark.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ When the `vbase` mode is enabled, `pgvecto.rs` can achieve over **2x** more resp

## Shortening embeddings (a.k.a [Matryoshka Embedding](https://aniketrege.github.io/blog/2024/mrl/))

OpenAI [latest embedding models](https://openai.com/blog/new-embedding-models-and-api-updates) supports shortening embeddings, where developers could simply “remove some numbers from the end of a sequence” and still maintain a valid representation for text.
OpenAI [latest embedding models](https://openai.com/blog/new-embedding-models-and-api-updates) support shortening embeddings, where developers could simply “remove some numbers from the end of a sequence” and still maintain a valid representation for text.

> This enables very flexible usage. For example, when using a vector data store that only supports embeddings up to 1024 dimensions long, developers can now still use our best embedding model text-embedding-3-large and specify a value of 1024 for the dimensions API parameter, which will shorten the embedding down from 3072 dimensions, trading off some accuracy in exchange for the smaller vector size.

Expand All @@ -34,16 +34,16 @@ These are the performance benchmarks for the shortened embeddings. The [dbpedia-

**First Pass** in the benchmark shows that the accuracy of the shortened embeddings is not as good as the full embeddings. 256D HNSW index only achieves ~65% accuracy, while 1024D HNSW index achieves ~85% accuracy.

However, the accuracy of the shortened embeddings surpasses 95% when adaptive retrieval is employed. Additionally, adopting this approach leads to significant improvements in terms of RPS (Requests Per Second) and latency.
However, the accuracy of the shortened embeddings surpasses 95% when [adaptive retrieval](/use-case/adaptive-retrieval) is employed. Additionally, adopting this approach leads to significant improvements in terms of RPS (Requests Per Second) and latency.

## Binary vectors

`pgvecto.rs` supports [binary vectors](/usage/vector-types.html#bvector-binary-vector). The `bvector` type is optimized for storage and performance. It uses a bit-packed representation to store the binary vector. The distance calculation is also optimized for binary vectors.

Here are some performance benchmarks for the `bvector` type. We use the [dbpedia-entities-openai3-text-embedding-3-large-3072-1M](https://huggingface.co/datasets/Qdrant/dbpedia-entities-openai3-text-embedding-3-large-3072-1M) dataset for the benchmark. The VM is n2-standard-8 (8 vCPUs, 32 GB memory) on Google Cloud.

We upsert 1M binary vectors into the table and then run a KNN query for each embedding. It only takes about 600MB memory to index 1M binary vectors.
We upsert 1M binary vectors into the table and then run a KNN query for each embedding. It only takes about 600MB memory to index 1M binary vectors, while the `vector` type takes about 18GB memory to index the same number of vectors.

![bvector](./images/bvector.png)

We can see that the `bvector`'s accuracy is not as good as the `vector` type, but it exceeds 95% if we adopt adaptive retrieval.
We can see that the `bvector`'s accuracy is not as good as the `vector` type, but it exceeds 95% if we adopt [adaptive retrieval](/use-case/adaptive-retrieval).
7 changes: 7 additions & 0 deletions src/reference/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Reference

The page shows the reference.

- [Indexing options](/reference/indexing_options)
- [Search options](/reference/search_options)
- [Schema](/reference/schema)
Binary file modified src/usage/images/bvector.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion src/usage/indexing.md
Original file line number Diff line number Diff line change
Expand Up @@ -126,4 +126,4 @@ Smaller [`m`](/reference/indexing_options.html#options-for-table-hnsw)) values a

The [`ef_construction`](/reference/indexing_options.html#options-for-table-hnsw) parameter determines the dynamic candidate list size when adding new nodes; increasing this value may enhance recall but could extend index construction time.

The HNSW index is resource-intensive, requiring additional RAM and an adjustment to the `maintenance_work_mem` setting for larger datasets. If you're seeking performance that's markedly faster than IVF, with a high recall rate and scalability that matches dataset size, HNSW is an excellent option.
The HNSW index is resource-intensive, requiring additional RAM and an adjustment to the `maintenance_work_mem` setting for larger datasets. If you're seeking performance that's markedly faster than IVF, with a high recall rate and scalability that matches dataset size, HNSW is an excellent option.
4 changes: 2 additions & 2 deletions src/usage/vector-types.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,8 @@ The `bvector` type is optimized for storage and performance. It uses a bit-packe

Here are some performance benchmarks for the `bvector` type. We use the [dbpedia-entities-openai3-text-embedding-3-large-3072-1M](https://huggingface.co/datasets/Qdrant/dbpedia-entities-openai3-text-embedding-3-large-3072-1M) dataset for the benchmark. The VM is n2-standard-8 (8 vCPUs, 32 GB memory) on Google Cloud.

We upsert 1M binary vectors into the table and then run a KNN query for each embedding. It only takes about 600MB memory to index 1M binary vectors.
We upsert 1M binary vectors into the table and then run a KNN query for each embedding. It only takes about 600MB memory to index 1M binary vectors, while the `vector` type takes about 18GB memory to index the same number of vectors.

![bvector](./images/bvector.png)

We can see that the `bvector`'s accuracy is not as good as the `vector` type, but it exceeds 95% if we adopt adaptive retrieval.
We can see that the `bvector`'s accuracy is not as good as the `vector` type, but it exceeds 95% if we adopt [adaptive retrieval](/use-case/adaptive-retrieval).
Loading