flash-attention

Here are 4 public repositories matching this topic...

Performance of the C++ interface of flash attention and flash attention v2 in large language model (LLM) inference scenarios.

gpu cuda inference nvidia cutlass mha multi-head-attention llm tensor-core large-language-model flash-attention flash-attention-2

Decoding Attention is specially optimized for multi head attention (MHA) using CUDA core for the decoding stage of LLM inference.

gpu cuda inference nvidia mha multi-head-attention llm large-language-model flash-attention cuda-core decoding-attention flashinfer

Poplar implementation of FlashAttention for IPU

deep-learning transformers pytorch ipu graphcore poplar flash-attention flash-attention-2

Vulkan & GLSL implementation of FlashAttention-2

Add a description, image, and links to the flash-attention topic page so that developers can more easily learn about it.

To associate your repository with the flash-attention topic, visit your repo's landing page and select "manage topics."