Fast inference engine for Transformer models

📚150+ Tensor/CUDA Cores Kernels, ⚡️flash-attn-mma, ⚡️hgemm with WMMA, MMA and CuTe (98%~100% TFLOPS of cuBLAS/FA2 🎉🎉).

Tuned OpenCL BLAS

BLISlab: A Sandbox for Optimizing GEMM

Several optimization methods of half-precision general matrix multiplication (HGEMM) using tensor core with WMMA API and MMA PTX instruction.

High-Performance FP32 Matrix Multiplication on CPU

Optimizing SGEMM kernel functions on NVIDIA GPUs to a close-to-cuBLAS performance.

The HPC toolbox: fused matrix multiplication, convolution, data-parallel strided tensor primitives, OpenMP facilities, SIMD, JIT Assembler, CPU detection, state-of-the-art vectorized BLAS for floats and integers

Stretching GPU performance for GEMMs and tensor contractions.

🔥🔥🔥 A collection of some awesome public CUDA, cuBLAS, cuDNN, CUTLASS, TensorRT, TensorRT-LLM, Triton, TVM, MLIR and High Performance Computing (HPC) projects.

DBCSR: Distributed Block Compressed Sparse Row matrix library

Stepwise optimizations of DGEMM on CPU, reaching performance faster than Intel MKL eventually, even under multithreading.

Single file libraries for C/C++

hipBLASLt is a library that provides general matrix-matrix operations with a flexible API and extends functionalities beyond a traditional BLAS library

Several optimization methods of half-precision general matrix vector multiplication (HGEMV) using CUDA core.

FP64 equivalent GEMM via Int8 Tensor Cores using the Ozaki scheme

PyTorch half precision gemm lib w/ fused optional bias + optional relu/gelu

Serial and parallel implementations of matrix multiplication

A Flexible and Energy Efficient Accelerator For Sparse Convolution Neural Network