-
Notifications
You must be signed in to change notification settings - Fork 182
Examples and tutorials
Nandan Thakur edited this page Jun 29, 2022
·
1 revision
To easily understand and get your hands dirty with BEIR, we invite you to try our tutorials out π π
Name | Link |
---|---|
How to evaluate pre-trained models on BEIR datasets |
Name | Link |
---|---|
BM25 Retrieval with Elasticsearch | evaluate_bm25.py |
Anserini-BM25 (Pyserini) Retrieval with Docker | evaluate_anserini_bm25.py |
Multilingual BM25 Retrieval with Elasticsearch π | evaluate_multilingual_bm25.py |
Name | Link |
---|---|
Exact-search retrieval using (dense) Sentence-BERT | evaluate_sbert.py |
Exact-search retrieval using (dense) ANCE | evaluate_ance.py |
Exact-search retrieval using (dense) DPR | evaluate_dpr.py |
Exact-search retrieval using (dense) USE-QA | evaluate_useqa.py |
ANN and Exact-search using Faiss π | evaluate_faiss_dense.py |
Retrieval using Binary Passage Retriver (BPR) π | evaluate_bpr.py |
Dimension Reduction using PCA π | evaluate_dim_reduction.py |
Name | Link |
---|---|
Hybrid sparse retrieval using SPARTA | evaluate_sparta.py |
Sparse retrieval using docT5query and Pyserini | evaluate_anserini_docT5query.py |
Sparse retrieval using docT5query (MultiGPU) and Pyserini π | evaluate_anserini_docT5query_parallel.py |
Sparse retrieval using DeepCT and Pyserini π | evaluate_deepct.py |
Name | Link |
---|---|
Reranking top-100 BM25 results with SBERT CE | evaluate_bm25_ce_reranking.py |
Reranking top-100 BM25 results with Dense Retriever | evaluate_bm25_sbert_reranking.py |
Name | Link |
---|---|
Train SBERT with Inbatch negatives | train_sbert.py |
Train SBERT with BM25 hard negatives | train_sbert_BM25_hardnegs.py |
Train MSMARCO SBERT with BM25 Negatives | train_msmarco_v2.py |
Train (SOTA) MSMARCO SBERT with Mined Hard Negatives π | train_msmarco_v3.py |
Train (SOTA) MSMARCO BPR with Mined Hard Negatives π | train_msmarco_v3_bpr.py |
Train (SOTA) MSMARCO SBERT with Mined Hard Negatives (Margin-MSE) π | train_msmarco_v3_margin_MSE.py |
Name | Link |
---|---|
Synthetic Query Generation using T5-model | query_gen.py |
(GenQ) Synthetic QG using T5-model + fine-tuning SBERT | query_gen_and_train.py |
Synthetic Query Generation using Multiple GPU and T5 π | query_gen_multi_gpu.py |
Name | Link |
---|---|
Benchmark BM25 (Inference speed) | benchmark_bm25.py |
Benchmark Cross-Encoder Reranking (Inference speed) | benchmark_bm25_ce_reranking.py |
Benchmark Dense Retriever (Inference speed) | benchmark_sbert.py |