Skip to content

Commit

Permalink
Update deepspeed example (#2587)
Browse files Browse the repository at this point in the history
* Update dependencies in DeepSpeed example

* Update dependencies in DeepSpeed example README.md

---------

Co-authored-by: Ubuntu <ubuntu@ip-172-31-3-108.us-west-2.compute.internal>
  • Loading branch information
sachanub and Ubuntu authored Sep 11, 2023
1 parent 821f9bc commit daeb314
Showing 1 changed file with 18 additions and 1 deletion.
19 changes: 18 additions & 1 deletion examples/large_models/deepspeed/Readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,25 @@

This document briefs on serving large HuggingFace (HF) models on multiple GPUs using deepspeed. We are using facebook/opt-30b in this example

To run this example we need to have deepspeed installed. This has been added to the requirement.txt which can be bundled during model packaging.
### Pre-requisites

- Install CUDA. Verified to be working with CUDA 11.7.
- Verified to be working with:

```bash
torch 2.0.1+cu117
torch-model-archiver 0.8.2
torch-workflow-archiver 0.2.10
torchaudio 2.0.2+cu117
torchdata 0.6.1
torchserve 0.8.2
torchtext 0.15.2+cpu
torchvision 0.15.2+cu117
transformers 4.33.1
deepspeed 0.10.2
```

To run this example we need to have deepspeed installed. This has been added to the requirement.txt which can be bundled during model packaging.

```bash
pip install deepspeed
Expand Down

0 comments on commit daeb314

Please sign in to comment.