From 5fea8c9c756d09680f354361c011215d61b8ce0f Mon Sep 17 00:00:00 2001 From: Tom Stesco Date: Tue, 14 Jan 2025 20:24:41 -0500 Subject: [PATCH 1/2] update documentation links and instructions for setup --- vllm-tt-metal-llama3/README.md | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/vllm-tt-metal-llama3/README.md b/vllm-tt-metal-llama3/README.md index 525281f..3f7a057 100644 --- a/vllm-tt-metal-llama3/README.md +++ b/vllm-tt-metal-llama3/README.md @@ -117,9 +117,9 @@ For instructions on building the Docker imagem locally see: [vllm-tt-metal-llama The script `setup.sh` automates: -1. interactively creating the .env file, -2. downloading the Llama model weights, -3. repacking the weights as required for tt-metal implementation, +1. interactively creating the model specific .env file, +2. downloading the model weights, +3. (if required) repacking the weights for tt-metal implementation, 4. creating the default persistent storage directory structure and permissions. ```bash @@ -128,11 +128,8 @@ chmod +x setup.sh ./setup.sh llama-3.1-70b-instruct ``` -NOTE: for instruct fine-tuned models, you must first input `llama-3.1-70b-instruct`, then when running `download.sh`, input `meta-llama-3.1-70b`, and finally input `meta-llama-3.1-70b-instruct`. - -If you need to modify the setup or otherwise need to manually do it please see the [Manual Setup Guide](docs/manual_setup_guide.md). - # Additional Documentation -- [FAQ](docs/faq.md) - [Development](docs/development.md) +- [Benchmarking](../benchmarking/README.md) +- [Evals](../evals/README.md) \ No newline at end of file From 8908e98ad01204c390eed2187ac150f9e18ba352 Mon Sep 17 00:00:00 2001 From: Tom Stesco Date: Tue, 14 Jan 2025 20:28:18 -0500 Subject: [PATCH 2/2] adding Locust load testing and tests docus to additional documentation list --- vllm-tt-metal-llama3/README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/vllm-tt-metal-llama3/README.md b/vllm-tt-metal-llama3/README.md index 3f7a057..b2b47a8 100644 --- a/vllm-tt-metal-llama3/README.md +++ b/vllm-tt-metal-llama3/README.md @@ -132,4 +132,6 @@ chmod +x setup.sh - [Development](docs/development.md) - [Benchmarking](../benchmarking/README.md) -- [Evals](../evals/README.md) \ No newline at end of file +- [Evals](../evals/README.md) +- [Locust load testsing](../locust/README.md) +- [tests](../tests/README.md)