Skip to content

Commit

Permalink
Clarify GPU chat inference instructions
Browse files Browse the repository at this point in the history
  • Loading branch information
parinitarahi authored Feb 14, 2025
1 parent 818fc64 commit 80334f5
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion docs/genai/tutorials/deepseek-python.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,6 @@ python model-chat.py -m deepseek-r1-distill-qwen-1.5B/model -e cpu --chat_templa
```

```bash
# GPU Chat inference. If you pulled the model from huggingface, adjust the model directory (-m) accordingly
# On-Device GPU Chat inference. Works on devices with Nvidia GPUs. If you pulled the model from huggingface, adjust the model directory (-m) accordingly
curl -o https://raw.githubusercontent.com/microsoft/onnxruntime-genai/refs/heads/main/examples/python/model-chat.py
python model-chat.py -m deepseek-r1-distill-qwen-1.5B/model -e cuda --chat_template "<|begin▁of▁sentence|><|User|>{input}<|Assistant|>"

0 comments on commit 80334f5

Please sign in to comment.