Way to write out embeddings to a file?

[edeandrea]

I'm using the easy-rag extension which loads a pdf file at startup (in dev mode) to an embedding model which is stored in redis.

Is there a way that I could write out the embeddings to a file rather than re-computing the embeddings? I'm using ollama with mistral, so my dev mode startup is about 10-15 minutes, and then 10-15 mins each time I make a change while in dev mode.

If it were to write out the embeddings to a file the first time, then each subsequent time it would only need to load the embeddings into the store, that would be much faster, especially since (in my case) the file isn't changing.

[jmartisk]

It sounds to me like the easiest solution for you would be to use a manually managed Redis (instead of Dev Services), so that you can have it populated just once, and then you would use quarkus.langchain4j.easy-rag.ingestion-strategy=OFF to avoid re-ingesting at startup every time.

Other than that, some kind of FileEmbeddingStore for dumping an embedding store into an XML or whatever format might be useful for some use cases, but I think we don't have that now

[edeandrea]

Yeah it didn't look like we did have it. I was just looking at some of the Spring AI examples, specifically https://github.com/habuma/spring-ai-rag-example/blob/main/src/main/java/com/example/springairag/SpringAIRAGExample.java

It looks like they have some kind of abstraction for a vector store. After getting the Documents and have loaded the store, they save the vectors out to a file. Then the next time the app starts, if it finds the file, it just loads it into the vector store without having to re-compute the vectors.

It would save a ton of time for my use case. Dev mode is taking about 10 minutes to start, and then each time I make a change it takes another 10 minutes to reload because it is recomputing the vectors from a pdf document I have.

[jmartisk]

I'd suggest moving this over to the upstream https://github.com/langchain4j/langchain4j/ project as this is probably something that would be (mostly) handled there, not by Quarkus. For the time being, using a manually managed database and quarkus.langchain4j.easy-rag.ingestion-strategy=OFF would be my recommended workaround

[edeandrea]

It actually already is there in upstream, at least when using the in memory embedding store. See https://github.com/langchain4j/langchain4j/blob/eba2b1cd500f7b8874d4da5e406f09935febb9b3/langchain4j/src/main/java/dev/langchain4j/store/embedding/inmemory/InMemoryEmbeddingStore.java#L132-L164

I think all we'd need here in the short term would be some stuff in the easy rag extension that did something with these methods.

[edeandrea]

I've moved this conversation over to #663