From 911dee5008f81513ef8bdf7b0e8828b9acc9c728 Mon Sep 17 00:00:00 2001 From: Sherlock113 Date: Tue, 12 Nov 2024 10:31:24 +0800 Subject: [PATCH 1/2] Update examples index Signed-off-by: Sherlock113 --- docs/source/use-cases/index.rst | 24 +++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-) diff --git a/docs/source/use-cases/index.rst b/docs/source/use-cases/index.rst index 00118e1496b..8fa855edbc5 100644 --- a/docs/source/use-cases/index.rst +++ b/docs/source/use-cases/index.rst @@ -41,23 +41,29 @@ See the following lists for a complete collection of BentoML example projects. B LLMs ---- -Serve LLMs with BentoML and vLLM: +Deploy an OpenAI-compatible LLM API service: - `Llama 3.2 90B `_ - `Llama 3.1 70B `_ - `Mistral 7B `_ - `Pixtral 12B `_ - `Phi 3 mini `_ -- Check out the `BentoVLLM project `_ to see more examples -Build compound AI systems on top of LLMs: +Choose your LLM Inference runtime: -- `Build an AI agent with function calling using BentoML `_ -- `Build a LangGraph agent application with BentoML `_ -- `Build a multi-agent application with CrewAI and BentoML `_ -- `Build an AI assistant to ensure LLM safety with BentoML and ShieldGemma `_ -- `Build a RAG system for document ingestion and search with LlamaIndex and BentoML `_ -- `Build a voice agent with open-source models using Twilio, Pipecat and BentoML `_ +- `vLLM `_ (or `OpenLLM `_) +- `TensorRT-LLM `_ +- `LMDeploy `_ +- `MLC-LLM `_ + +Advanced examples: + +- `Agent: Function calling `_ +- `Agent: LangGraph `_ +- `Multi-agent: CrewAI `_ +- `LLM safety: ShieldGemma `_ +- `RAG: LlamaIndex `_ +- `Phone call agent `_ Image and video --------------- From 62acddffb71832a7c99614d821b5bc1ffd7994dc Mon Sep 17 00:00:00 2001 From: Sherlock Xu <65327072+Sherlock113@users.noreply.github.com> Date: Tue, 12 Nov 2024 11:48:25 +0800 Subject: [PATCH 2/2] Update index.rst --- docs/source/use-cases/index.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/source/use-cases/index.rst b/docs/source/use-cases/index.rst index 8fa855edbc5..0ca6ea6aa63 100644 --- a/docs/source/use-cases/index.rst +++ b/docs/source/use-cases/index.rst @@ -55,6 +55,8 @@ Choose your LLM Inference runtime: - `TensorRT-LLM `_ - `LMDeploy `_ - `MLC-LLM `_ +- `SGLang `_ +- `Hugging Face TGI `_ Advanced examples: