-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
update eval scripts #49
Conversation
…, default to GPQA run with 1 concurrent user
@@ -8,15 +8,6 @@ Source code: | |||
## Step 1: Pull Docker image | |||
|
|||
Docker images are published to: https://ghcr.io/tenstorrent/tt-inference-server/tt-metal-llama3-70b-src-base-vllm | |||
```bash | |||
# commit SHAs | |||
export TT_METAL_COMMIT_SHA_OR_TAG=685ef1303b5abdfda63183fdd4fd6ed51b496833 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
now that the tt-metal and vllm commit sha are removed here, do we need to specify where someone can find what commit sha to use (ie. point back to vllm-tt-metal-llama3-70b/src or the vllm fork) for step 2 (the docker run)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's in the main readme for vllm-tt-metal-llama3-70b: https://github.com/tenstorrent/tt-inference-server/tree/main/vllm-tt-metal-llama3-70b#option-b-build-docker-image
This was just duplicating it and became outdated.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I improved the docker build documentation 👍
…, default to GPQA run with 1 concurrent user
8ccf400
to
e7ddc84
Compare
…rence-server into tstesco/update-evals
…tt-metal-llama3-70b/docs/development.md#step-1-build-docker-image
change log