Skip to content

Lite weight wrapper for the independent implementation of SPLADE++ models for search & retrieval pipelines. Models and Library created by Prithivi Da, For PRs and Collaboration checkout the readme.

Notifications You must be signed in to change notification settings

PrithivirajDamodaran/SPLADERunner

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

45 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

SPLADERunner

1. What is it?

Title is dedicated to the Original Blade Runners - Harrison Ford and the Author Philip K. Dick of "Do Androids Dream of Electric Sheep?"

A Ultra-lite & Super-fast Python wrapper for the independent implementation of SPLADE++ models for your search & retrieval pipelines. Based on the papers Naver's From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effective and Google's SparseEmbed

  • Lite weight:
    • No Torch or Transformers needed.
    • Runs on CPU for query or passage expansion.
    • FLOPS & Retrieval Efficient: Refer model card for details.

🚀 Installation:

pip install spladerunner

Usage:

# One-time only init
from spladerunner import Expander
expander = Expander('Splade_PP_en_v1', 128) #pass model, max_seq_len

# Sample passage expansion
sparse_rep = expander.expand("The Manhattan Project and its atomic bomb helped bring an end to World War II. Its legacy of peaceful uses of atomic energy continues to have an impact on history and science.")


# For solr or elastic or vanilla lucene stores.
sparse_rep = expander.expand("The Manhattan Project and its atomic bomb helped bring an end to World War II. Its legacy of peaceful uses of atomic energy continues to have an impact on history and science.", outformat="lucene")

print(sparse_rep)

(Feel free to skip to 3 If you are expert in sparse and dense representations)

2. Why Sparse Representations?

  • Lexical search with BOW based sparse vectors are strong baselines, but they famously suffer from vocabulary mismatch problem, as they can only do exact term matching.

Pros

✅ Efficient and Cheap.
✅ No need to fine-tune models.
✅️ Interpretable.
✅️ Exact Term Matches.

Cons

❌ Vocabulary mismatch (Need to remember exact terms)
  • Semantic Search Learned Neural / Dense retrievers with approximate nearest neighbors search has shown impressive results but they can

Pros

✅ Search how humans innately think.
✅ When finetuned beats sparse by long way.
✅ Easily works with Multiple modals.

Cons

❌ Suffers token amnesia (misses term matching), 
❌ Resource intensive (both index & retreival), 
❌ Famously hard to interpret.
❌ Needs fine-tuning for OOD data.
  • Getting pros of both searches made sense and that gave rise to interest in learning sparse representations for queries and documents with some interpretability. The sparse representations also double as implicit or explicit (latent, contextualized) expansion mechanisms for both query and documents. If you are new to query expansion learn more here from Daniel Tunkelang.

2a. What the Models learn?

  • The model learns to project it's learned dense representations over a MLM head to give a vocabulary distribution.

3. 💸 Why SPLADERunner?:

  • $ Concious: Serverless deployments like Lambda are charged by memory & time per invocation
  • Smaller package size = shorter cold start times, quicker re-deployments for Serverless.

4. 🎯 Models:

4a. 💸 Where and How can you use?

  • [TBD]

4b. How (and what) to contribute?

  • [TBD]

5. Criticisms and Competitions to SPLADE and Learned Sparse representations:

About

Lite weight wrapper for the independent implementation of SPLADE++ models for search & retrieval pipelines. Models and Library created by Prithivi Da, For PRs and Collaboration checkout the readme.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages