Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update eval scripts #49

Merged
merged 8 commits into from
Dec 2, 2024
Merged

update eval scripts #49

merged 8 commits into from
Dec 2, 2024

Conversation

tstescoTT
Copy link
Contributor

change log

  • update eval scripts and documentation
  • set default port to 7000 for consistency
  • default to GPQA run with 1 concurrent user

@@ -8,15 +8,6 @@ Source code:
## Step 1: Pull Docker image

Docker images are published to: https://ghcr.io/tenstorrent/tt-inference-server/tt-metal-llama3-70b-src-base-vllm
```bash
# commit SHAs
export TT_METAL_COMMIT_SHA_OR_TAG=685ef1303b5abdfda63183fdd4fd6ed51b496833
Copy link
Contributor

@mvanniasingheTT mvanniasingheTT Nov 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

now that the tt-metal and vllm commit sha are removed here, do we need to specify where someone can find what commit sha to use (ie. point back to vllm-tt-metal-llama3-70b/src or the vllm fork) for step 2 (the docker run)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's in the main readme for vllm-tt-metal-llama3-70b: https://github.com/tenstorrent/tt-inference-server/tree/main/vllm-tt-metal-llama3-70b#option-b-build-docker-image

This was just duplicating it and became outdated.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I improved the docker build documentation 👍

@tstescoTT tstescoTT force-pushed the tstesco/update-evals branch from 8ccf400 to e7ddc84 Compare November 29, 2024 20:04
@tstescoTT tstescoTT merged commit 7712e0e into main Dec 2, 2024
1 check passed
@tstescoTT tstescoTT deleted the tstesco/update-evals branch December 3, 2024 21:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants