🔥 Large Language Models(LLM) have taken the NLP community AI community the Whole World by storm.
The LLM space is complicated! This repo provides a curated list to help you navigate; it includes a collection of Large Language Model frameworks and tutorials, covering model training, serving, fine-tuning, and building LLM applications.
- Applications
- Pretraining
- Fine-tuning
- Serving
- Datasets
- Benchmarks
- Learn LLM Applications
- Understand LLM
- Social Accounts
Build & Auto-optimize
-
AdalFlow - The library to build & auto-optimize LLM applications, from Chatbot, RAG, to Agent. It is AI-first with PyTorch-like design patterns.
-
dspy - DSPy: The framework for programming—not prompting—foundation models.
Build
- LlamaIndex — A Python library for augmenting LLM apps with data.
- LangChain — A popular Python/JavaScript library for chaining sequences of language model prompts.
Prompt Optimization
- AutoPrompt - A framework for prompt tuning using Intent-based Prompt Calibration
- PromptFify - A library for prompt engineering that simplifies NLP tasks (e.g., NER, classification) using LLMs like GPT.
Others
- LiteLLM - Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format.
- PyTorch - PyTorch is an open source machine learning library based on the Torch library, used for applications such as computer vision and natural language processing.
- TensorFlow - TensorFlow is an open source machine learning library developed by Google.
- JAX - Google’s library for high-performance computing and automatic differentiation.
- tinygrad - A minimalistic deep learning library with a focus on simplicity and educational use, created by George Hotz.
- micrograd - A simple, lightweight autograd engine for educational purposes, created by Andrej Karpathy.
- Transformers - Hugging Face Transformers is a popular library for Natural Language Processing (NLP) tasks, including fine-tuning large language models.
- Unsloth - Finetune Llama 3.2, Mistral, Phi-3.5 & Gemma 2-5x faster with 80% less memory!
- LitGPT - 20+ high-performance LLMs with recipes to pretrain, finetune, and deploy at scale.
-
TorchServe - An open-source model serving library developed by AWS and Facebook specifically for PyTorch models, enabling scalable deployment, model versioning, and A/B testing.
-
TensorFlow Serving - A flexible, high-performance serving system for machine learning models, designed for production environments, and optimized for TensorFlow models but also supports other formats.
-
Ray Serve - Part of the Ray ecosystem, Ray Serve is a scalable model-serving library that supports deployment of machine learning models across multiple frameworks, with built-in support for Python-based APIs and model pipelines.
-
NVIDIA Triton Inference Server - A high-performance inference server supporting multiple ML/DL frameworks (TensorFlow, PyTorch, ONNX, etc.), optimized for GPU deployments, and ideal for both cloud and on-premises serving.
-
vllm - An optimized, high-throughput serving engine for large language models, designed to efficiently handle massive-scale inference with reduced latency.
-
sglang - SGLang is a fast serving framework for large language models and vision language models.
-
LitServe - LitServe is a lightning-fast serving engine for any AI model of any size. Flexible. Easy. Enterprise-scale.
Use Cases
- Datasets - A vast collection of ready-to-use datasets for machine learning tasks, including NLP, computer vision, and audio, with tools for easy access, filtering, and preprocessing.
- Argilla - A UI tool for curating and reviewing datasets for LLM evaluation or training.
- distilabel - A library for generating synthetic datasets with LLM APIs or models.
Fine-tuning
- LLMDataHub - A quick guide (especially) for trending instruction finetuning datasets
- LLM Datasets - High-quality datasets, tools, and concepts for LLM fine-tuning.
Pretraining
- IBM LLMs Granite 3.0 - Full list of datasets used to train IBM LLMs Granite 3.0
- lighteval - A library for evaluating local LLMs on major benchmarks and custom tasks.
Agent
- TravelPlanner - paper A Benchmark for Real-World Planning with Language Agents
Prompt Engineering
Reasoning & Planning
- Chip's Blog - Chip Huyen's blog on training LLMs, including the latest research, tutorials, and best practices.
- Lil'Log - Lilian Weng(OpenAI)'s blog on machine learning, deep learning, and AI, with a focus on LLMs and NLP.
-
AdalFlow documentation - Includes tutorials from building RAG, Agent, to LLM evaluation and fine-tuning.
-
CS224N - Stanford course covering NLP fundamentals, LLMs, and PyTorch-based model building, led by Chris Manning and Shikhar Murty.
-
LLM-driven Data Engineering - A playlist of 6 lectures by Zach Wilson on how LLMs will impact data pipeline development
-
LLM Course by Maxime Labonne - An end-to-end course for AI and ML engineers on open source LLMs.
- Lectures
- LLM Agents MOOC - A playlist of 11 lectures by the Berkeley RDI Center on Decentralization & AI, featuring guest speakers like Yuandong Tian, Graham Neubig, Omar Khattab, and others, covering core topics on Large Language Model agents.
- Projects
- Chip Huyen
- Damien Benveniste, PhD
- Jim Fan
- Li Yin
- Paul Iusztin LLMOps
- Armand Ruiz AI product at IBM
- Pascal Biese Daily LLM papers
- Maxime Labonne LLM fine-tuning
This is an active repository and your contributions are always welcome!
I will keep some pull requests open if I'm not sure if they are not an instant fit for this repo, you could vote for them by adding 👍 to them.
If you have any question about this opinionated list, do not hesitate to contact Li Yin