matryoshka_dim=512 is not in a valid range. Select between 1 and 128.

[antonmark]

System Info

curl -X 'POST' 'https://localhost:8080/embeddings' -H 'accept: application/json' -H 'Content-Type: application/json' -d '{
"model": "mixedbread-ai/mxbai-embed-large-v1",
"encoding_format": "float",
"dimensions": 512,
"input": [
"test 123"
],
"modality": "text"
}'
{
"error": {
"message": "<class 'infinity_emb.primitives.MatryoshkaDimError'> -> matryoshka_dim=512 is not in a valid range. Select between 1 and 128.",
"type": null,
"param": null,
"code": 400
}
}

Information

• Docker + cli
• pip + cli
• pip + usage of Python interface

Tasks

• An officially supported CLI command
• My own modifications

Reproduction

1. Start michaelf34/infinity:latest using: v2 --model-id mixedbread-ai/mxbai-embed-large-v1 --port 8080 --embedding-dtype ubinary
2. Query embeddings API endpoint.

[michaelfeil]

Binary quantization (packed) and matryoshka does not pair well. In binary, 1 output will be packed into ~8/16 floats.

[antonmark]

Interesting. I had picked the option based on this Mixedbread blog post:

https://www.mixedbread.ai/blog/binary-mrl#using-it-in-action

I might be missing some nuance here.