Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch
-
Updated
Oct 25, 2024 - Python
#
efficient-attention
Here are 10 public repositories matching this topic...
Implementation of the conditionally routed attention in the CoLT5 architecture, in Pytorch
-
Updated
Sep 6, 2024 - Python
Efficient Infinite Context Transformers with Infini-attention Pytorch Implementation + QwenMoE Implementation + Training Script + 1M context keypass retrieval
-
Updated
May 9, 2024 - Python
Unofficial PyTorch implementation of the paper "cosFormer: Rethinking Softmax In Attention".
deep-learning neural-network pytorch artificial-intelligence transformer attention-mechanism iclr efficient-attention iclr2022
-
Updated
Oct 29, 2021 - Jupyter Notebook
The official PyTorch implementation for CascadedGaze: Efficiency in Global Context Extraction for Image Restoration, TMLR'24.
-
Updated
Oct 8, 2024 - Python
Pytorch implementation of "Compact Global Descriptor for Neural Networks" (CGD).
efficient pytorch convolutional-neural-networks attention-mechanism attention-model efficient-attention
-
Updated
Nov 24, 2019 - Python
Implementation of: Hydra Attention: Efficient Attention with Many Heads (https://arxiv.org/abs/2209.07484)
-
Updated
Jan 8, 2023 - Python
Nonparametric Modern Hopfield Models
efficient-transformers efficient-attention modern-hopfield-networks modern-hopfield-model efficient-hopfield-models efficient-hopfield-networks
-
Updated
Jan 8, 2024 - Jupyter Notebook
Official Implementation of SEA: Sparse Linear Attention with Estimated Attention Mask (ICLR 2024)
-
Updated
Oct 24, 2024 - Python
Minimal implementation of Samba by Microsoft in PyTorch
language-model ssm pytorch-implementation efficient-attention llm long-context-modeling mamba-state-space-models
-
Updated
Nov 24, 2024 - Python
Improve this page
Add a description, image, and links to the efficient-attention topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the efficient-attention topic, visit your repo's landing page and select "manage topics."