SymbolicAI Embeddings

This project is based on the Sentence Transformers repo and offers all models for the SymbolicAI project.

Sentence Transformers: Multilingual Sentence, Paragraph, and Image Embeddings using BERT & Co.

This framework provides an easy method to compute dense vector representations for sentences, paragraphs, and images. The models are based on transformer networks like BERT / RoBERTa / XLM-RoBERTa etc. and achieve state-of-the-art performance in various tasks. Text is embedded in vector space such that similar text are closer and can efficiently be found using cosine similarity.

The repo provides an increasing number of state-of-the-art pretrained models for more than 100 languages, fine-tuned for various use-cases.

Further, this framework allows an easy fine-tuning of custom embeddings models, to achieve maximal performance on your specific task.

For the full documentation, see www.SBERT.net.

The following publications are integrated in this framework:

• Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks (EMNLP 2019)
• Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation (EMNLP 2020)
• Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring Tasks (NAACL 2021)
• The Curse of Dense Low-Dimensional Information Retrieval for Large Index Sizes (arXiv 2020)
• TSDAE: Using Transformer-based Sequential Denoising Auto-Encoder for Unsupervised Sentence Embedding Learning (arXiv 2021)
• BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models (arXiv 2021)