Awesome Multi Vector Representations

A list of multi-vector resources to understand the use and tradeoffs of using multi vector representations in retrieval.

Table of Contents

• Papers
• Software
• Tutorials

Papers

1. ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT Omar Khattab, & Matei Zaharia. (2020). ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT.

2. ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction Keshav Santhanam, Omar Khattab, Jon Saad-Falcon, Christopher Potts, & Matei Zaharia. (2022). ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction.

3. PLAID: An Efficient Engine for Late Interaction Retrieval Keshav Santhanam, Omar Khattab, Christopher Potts, & Matei Zaharia. (2022). PLAID: An Efficient Engine for Late Interaction Retrieval.

4. Efficient Multi-Vector Dense Retrieval with Bit Vectors Franco Maria Nardini, Cosimo Rulli, & Rossano Venturini. (2024). Efficient Multi-Vector Dense Retrieval Using Bit Vectors.

5. A Reproducibility Study of PLAID

6. PLAID SHIRTTT for Large-Scale Streaming Dense Retrieval

7. Rethinking the Role of Token Retrieval in Multi-Vector Retrieval Jinhyuk Lee, Zhuyun Dai, Sai Meher Karthik Duddu, Tao Lei, Iftekhar Naim, Ming-Wei Chang, & Vincent Y. Zhao. (2024). Rethinking the Role of Token Retrieval in Multi-Vector Retrieval.

8. CITADEL: Conditional Token Interaction via Dynamic Lexical Routing for Efficient and Effective Multi-Vector Retrieval Minghan Li, Sheng-Chieh Lin, Barlas Oguz, Asish Ghoshal, Jimmy Lin, Yashar Mehdad, Wen-tau Yih, & Xilun Chen. (2022). CITADEL: Conditional Token Interaction via Dynamic Lexical Routing for Efficient and Effective Multi-Vector Retrieval.

9. SPLATE: Sparse Late Interaction Retrieval Thibault Formal, Stéphane Clinchant, Hervé Déjean, & Carlos Lassance. (2024). SPLATE: Sparse Late Interaction Retrieval.

10. Fine-grained Late-interaction Multi-modal Retrieval for Retrieval Augmented Visual Question Answering Weizhe Lin, Jinghong Chen, Jingbiao Mei, Alexandru Coca, & Bill Byrne. (2023). Fine-grained Late-interaction Multi-modal Retrieval for Retrieval Augmented Visual Question Answering.

11. LI-RAGE: Late Interaction Retrieval Augmented Generation with Explicit Signals for Open-Domain Table Question Answering Lin, W., Blloshmi, R., Byrne, B., de Gispert, A., & Iglesias, G. (2023). LI-RAGE: Late Interaction Retrieval Augmented Generation with Explicit Signals for Open-Domain Table Question Answering. Annual Meeting of the Association for Computational Linguistics.

12. KNOWHALU: HALLUCINATION DETECTION VIA MULTI-FORM KNOWLEDGE BASED FACTUAL CHECKING Jiawei Zhang, Chejian Xu, Yu Gai, Freddy Lecue, Dawn Song, & Bo Li. (2024). KnowHalu: Hallucination Detection via Multi-Form Knowledge Based Factual Checking.

Software

1. ColBERT

2. RAGatouille

3. LintDB

4. Vespa

5. fastRAG

6. WikiChat

Tutorials

• ColBERT intro notebook
• RAGatouille intro example
• PyVespa tutorial