From daeb3146bca68c32279190b390f153e3fb9e8a88 Mon Sep 17 00:00:00 2001 From: sachanub <110572198+sachanub@users.noreply.github.com> Date: Mon, 11 Sep 2023 15:16:32 -0700 Subject: [PATCH] Update deepspeed example (#2587) * Update dependencies in DeepSpeed example * Update dependencies in DeepSpeed example README.md --------- Co-authored-by: Ubuntu --- examples/large_models/deepspeed/Readme.md | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/examples/large_models/deepspeed/Readme.md b/examples/large_models/deepspeed/Readme.md index 4347e2758d..045e924a14 100644 --- a/examples/large_models/deepspeed/Readme.md +++ b/examples/large_models/deepspeed/Readme.md @@ -2,8 +2,25 @@ This document briefs on serving large HuggingFace (HF) models on multiple GPUs using deepspeed. We are using facebook/opt-30b in this example -To run this example we need to have deepspeed installed. This has been added to the requirement.txt which can be bundled during model packaging. +### Pre-requisites + +- Install CUDA. Verified to be working with CUDA 11.7. +- Verified to be working with: + +```bash +torch 2.0.1+cu117 +torch-model-archiver 0.8.2 +torch-workflow-archiver 0.2.10 +torchaudio 2.0.2+cu117 +torchdata 0.6.1 +torchserve 0.8.2 +torchtext 0.15.2+cpu +torchvision 0.15.2+cu117 +transformers 4.33.1 +deepspeed 0.10.2 +``` +To run this example we need to have deepspeed installed. This has been added to the requirement.txt which can be bundled during model packaging. ```bash pip install deepspeed