1.21.8

1.21.8 (2024-12-06)

Fix

• fix: Add Model2vec (#1546)

• Added Model2Vec wrapper

• Added Model2vec models

• Added model2vec models to registry

• Added model2vec as a dependency

• Ran linting

• Update mteb/models/model2vec_models.py

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsen@gmail.com>

• Update mteb/models/model2vec_models.py

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsen@gmail.com>

• Added adapted_from and superseeded_by to model2vec models.

• Added missing import

• Moved pyproject.toml to optional dependencies

• Fixed typos

• Added import error and changed model to model_name

• Added Numpy to frameworks

• Added Numpy to frameworks

• Corrected false info on model2vec models

• Replaced np.inf with maxint

• Update mteb/models/model2vec_models.py

Co-authored-by: Isaac Chung <chungisaac1217@gmail.com>

• Added option to have infinite max tokens, added it to Model2vec

---

Co-authored-by: Kenneth Enevoldsen <kennethcenevoldsen@gmail.com>
Co-authored-by: Isaac Chung <chungisaac1217@gmail.com> (2ee8d44)

Unknown

• Made result loading more permissive, changed eval splits for HotPotQA and DBPedia (#1554)

• Removed train and dev from eval splits on HotpotQA

• Removed dev from eval splits on DBPedia

• Made task_results validation more permissive

• Readded exception in get_score

• Ran linting (2905813)

1.21.7

1.21.7 (2024-12-05)

Fix

• fix: remove curev1 from multlingual (#1552)

Seems like it was added here:
1cc6c9e (279a4ee)

1.21.6

1.21.6 (2024-12-04)

Fix

• fix: Fixed metadata errors (#1547) (a44a46c)

1.21.5

1.21.5 (2024-12-04)

Fix

• fix: add sentence trimming to OpenAIWrapper (#1526)

• fix: add sentence trimming to OpenAIWrapper

• fix: import tiktoken library inside encode function

• fix: check tokenizer library installed and update ModelMeta to pass tokenizer_name

• fix: pass tokenizer_name, max_tokens to loader

• fix: make tokenizer_name None for default

• fix: delete changes for ModelMeta

• fix: fix revision to 2 for OpenAI models

• fix: add docstring for OpenAIWrapper

• fix: lint

• feat: add openai optional dependency set

• fix: add sleep for too many requests

• fix: add lint

• fix: delete evaluate file (37fdfa1)

• fix: Added arctic models (#1541)

#1515 (df11c38)

• fix: Added all-minilm-l12-v2 (#1542)

#1515 (97ab272)

• fix: add nomic models (#1543)

#1515 (5013df8)

Unknown

• Add cohere models (#1538)

• fix: bug cohere names

• format (c2f4c26)

1.21.4

1.21.4 (2024-12-04)

Documentation

• docs: Add Model Meta parameters and metadata (#1536)

• add multi_qa_MiniLM_L6_cos_v1 model meta

• add all_mpnet_base_v2

• add parameters to model meta

• make lint

• add extra params to meta (5fa7b7b)

Fix

• fix: add more model meta (jina, e5) (#1537)

• add e5 model meta

• address review comments (36bab4d)

1.21.3

1.21.3 (2024-12-02)

Fix

• fix: Proprietary models now get correctly shown in leaderboard (#1530)

• Fixed showing proprietary models in leaderboard

• Added links to all OpenAI models

• Fixed table formatting issues

• Bumped Gradio version (39349ff)

1.21.2

1.21.2 (2024-12-01)

Fix

• fix: Task load data error for SICK-BR-STS and XStance (#1534)

• fix task load data for two tasks

• correct dataset keys (5b6f20f)

1.21.1

1.21.1 (2024-11-30)

Fix

• fix: Correct typos superseeded -> superseded (#1532)

fix typo -> superseded (343b6e0)

1.21.0

1.21.0 (2024-11-29)

Feature

• feat: Evaluate missing splits (#1525)

• fix: evaluate missing splits (#1268)

• implement partial evaluation for missing splits

• lint

• requested changes done from scratch

• test for missing split evaluation added

• uncomment test

• lint

• avoid circular import

• use TaskResult

• skip tests for now

---

Co-authored-by: Isaac Chung <chungisaac1217@gmail.com>

• got test_all_splits_evaluated passing

• tests passing

• address review comments

• make lint

• handle None cases for kg_co2_emissions

• use new results info

---

Co-authored-by: Thivyanth <thivyanth2004@gmail.com> (8e12250)

1.20.6

1.20.6 (2024-11-29)

Fix

• fix: Adding missing metadata on models and mathcing names up with the results repo (#1528)

• Added Voyage 3 models

• Added correct metadata to Cohere models and matched names with the results repo (b02ae82)