Customizing Embeddings for Chromadb Vector Database

[anthonyikeda]

I started with this video on getting started with ChromaDB https://www.youtube.com/watch?v=u_N1t0CBuqA and decided, with the output database, to see how I can start querying the data using Spring AI Vector Stores.

In my Jupyter notebook, I'm configuring the ChromDB Client with:

import chromadb

from chromadb.utils.embedding_functions import OpenCLIPEmbeddingFunction
from chromadb.utils.data_loaders import ImageLoader

chroma_client = chromadb.HttpClient(host='localhost', port=8000)

image_loader = ImageLoader()

multimodal_ef = OpenCLIPEmbeddingFunction()

multimodal_db = chroma_client.get_or_create_collection(name="multimodal_db", embedding_function=multimodal_ef, data_loader=image_loader)

My data in the collection is:

multimodal_db.add(
    ids=[
        'E01',
        'E02',
        'E03',
        'E04',
        'E05',
        'E06',
    ],
    uris=[
        'images/bacon_chicken_waffle.jpeg',
        'images/chicken_blt.jpeg',
        'images/crepe.jpeg',
        'images/french_toast.jpeg',
        'images/omelette.jpeg',
        'images/salad.jpeg'
    ],
    metadatas=[
        {'item_id': 'E01', 'img_category' : 'food', 'item_name': 'Bacon, Chicken & Waffle', 'price' : '$20.00'},
        {'item_id': 'E02', 'img_category' : 'food', 'item_name': 'Chicken BLT', 'price' : '$18.00'},
        {'item_id': 'E03', 'img_category' : 'food', 'item_name': 'Banane, Fraises et Chocolat Crepe', 'price' : '$15.00'},
        {'item_id': 'E04', 'img_category' : 'food', 'item_name': 'A la Banane Brioche French Toast', 'price': '$18.00'},
        {'item_id': 'E05', 'img_category' : 'food', 'item_name': 'Andouille Omelette', 'price': '$20.00'},
        {'item_id': 'E06', 'img_category' : 'food', 'item_name': 'Farm Chicken Salad', 'price': '$20.00'},
    ]
)

 Using the following query:

        FilterExpressionBuilder b = new FilterExpressionBuilder();

        List<Document> documentList = this.vectorStore.similaritySearch(
                SearchRequest.defaults()
                        .withQuery("avocado")
                        .withTopK(5)
                        .withSimilarityThreshold(0.5)
                        .withFilterExpression(b.eq("img_category", "food").build())
        );

I've hit wall where I get the error message:

Embedding dimension 1536 does not match collection dimensionality 512

Apparently it's because the embedding function using in the Spring Application does not align with the one used in the Python code.

In the original video I'm using the OpenCLIPEmbeddingFunction in ChromaDB and I'm not sure how to reconfigure this for the Java code.

Has anyone faced this?

Should I just change my chromadb embedding function in the Python code to something that is supported in Spring AI (https://docs.spring.io/spring-ai/reference/api/embeddings.html) or can I actually create a new EmbeddingClient for Chroma DB specifically?

[yahyaghani]

did you find a resolution to this ? i am also hitting a wall trying to have a custom schema for my clip embeddings and some metadata using chromadb .

[anthonyikeda]

I gave up.