Add model2vec to config.json

[davidmezzetti]

Hello.

I'm planning to add a change to txtai to autodetect model2vec models. The best idea I have right now is the read the config.json file and see if it has the keys apply_pca and apply_zipf.

While I believe this will be pretty unique, have you guys considered adding something to the config.json file to signal it's a model2vec file?

[Pringled]

Hey @davidmezzetti,

That's a great suggestion! We will add the following to our config.json files:

"model_type": "model2vec",
"architectures": [
    "StaticModel"
  ],

And then you can use the model_type key to check for model2vec models. I'll ping you once we've made that change.

[davidmezzetti]

Sounds great! This will make it easier in my case as txtai is working with multiple vectorization libraries. Once this change is in, txtai will be able to automatically infer the vectorization method for model2vec models.

import txtai
embeddings = txtai.Embeddings(path="minishlab/potion-base-8M")

[Pringled]

Hey @davidmezzetti, I just updated all our configs to include the changes. Let me know if everything works as intended!

[davidmezzetti]

Excellent. Looks like this should work. I'll integrate this into txtai and report back. Thank you!

[davidmezzetti]

This change has been made, thanks again!

Just as an FYI, the potion-science models don't appear to have the updated config. But the other models are all working as expected.

[Pringled]

Hey @davidmezzetti, great, no problem! I just updated the science models as well, thanks for pointing that out!