What new features would you like to see in Vectra 2?

[Stevenic]

I'm starting to think about Vectra 2 so thought I'd start a discussion collecting ideas for features that should go into the next version of Vectra. I'll seed the discussion with a few things I'm thinking about.

New File Format
I'd like to create a new more compact file format for indexes then the current index.json format. I have this file format pretty much designed in my head and can share details in a separate discussion thread. One of my primary goals is to make building larger document indexes faster and more efficient. I'm probably going to move away from having folder based document indexes and instead use a single <index name>.vectra file to store everything the Vectra CLI currently stores in folder.

One key capability of this new file format will be the ability to concatenate individual .vectra files together to create new indexes. That will let you create per/document indexes that you can roll up to create sets of documents you can query over. So if you index a README.md file, the output will be a README.md.vectra file. You can then take a folder of these individual .vectra files, concatenate them, and then search over the entire folder.

[Stevenic]

Move Crawler to a Separate Package
In Vectra 1 the CLI based crawler was a bit of an after thought. The entire LocalDocumentIndex was an after thought. Vectra 2 will make document indexing a first class concept and I'd like to break the crawler out as a separate tool that can not only be used for crawling documents into a Vectra index, but potentially other indexes (like Pinecone) via a plugin model.

[Stevenic]

So I'm planning to start working on Vectra 2 in the next week or so and I'm thinking that I'm going to break the modules apart a bit...

• vectra-core will contain all common classes and interfaces and be runtime agnostic.
• vectra-client will be a node.js specific client implementation. Depends on vectra-core.
• vectra-browser-client will be a browser specific client implementation. Depends on vectra-core.
• vectra will be the CLI tool used to build indexes from the command line. Depends on vectra-client.

[ReneR97]

Hey,
is Vectra 2 still in the works?
Just interested in the progress

[Stevenic]

Yes and no... I've started a new company called Awarity.ai and it's taking up a lot of my focus. We're using Vectra internally and I'm pushing improvements to Vectra fairly regularly but a full re-write is slightly delayed. Instead what you'll end up seeing is a series of incremental improvements that slowly drive towards an official v1 release. Those improvements will have a lot of the ideas for v2.