Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

moved llama benchmark, sglang benchmark, sglang integration, and sdxl to ossci cluster #971

Merged
merged 21 commits into from
Feb 18, 2025
Merged

moved llama benchmark, sglang benchmark, sglang integration, and sdxl to ossci cluster #971

Show file tree
Hide file tree
Changes from 19 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
03be92c
moved 3 workflows over to ossci cluster
Feb 10, 2025
7a9a34a
added back large llama tests
Feb 12, 2025
1e8633c
added print debugging
Feb 13, 2025
24e0e53
added rocmsmi test
Feb 14, 2025
fa43d6a
fixed syntax error
Feb 14, 2025
39e40c2
shortened tests for faster iterations
Feb 14, 2025
acdc3c8
attempted fix
Feb 14, 2025
4798f32
attempted fix
Feb 14, 2025
69da7c9
fixed device issue
Feb 14, 2025
06175a7
fixed device issue
Feb 15, 2025
244f062
removed docker cleanup step
Feb 15, 2025
2e3c4b3
moved big test back to old runner
Feb 15, 2025
834ce29
added ci-sharktank
Feb 15, 2025
f72b58b
add hf token
saienduri Feb 15, 2025
6c1e296
removed sharktank workflow because I dont have a HF token
Feb 17, 2025
8d8f077
added back large test
Feb 17, 2025
d13b006
reverted llama bench for merge
Feb 17, 2025
d7ebf9d
updated hf token
Feb 17, 2025
4b624fb
reverted shark-tank
Feb 18, 2025
36df85c
addressed comments
Feb 18, 2025
b495dc3
Merge branch 'main' into eliasj42/migrate-mi300-runners
saienduri Feb 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci-sdxl.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ env:
jobs:
install-and-test:
name: Install and test
runs-on: mi300x-3
runs-on: linux-mi300-1gpu-ossci

steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
Expand Down
14 changes: 6 additions & 8 deletions .github/workflows/ci-sglang-benchmark.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ jobs:
matrix:
version: [3.11]
fail-fast: false
runs-on: mi300x-3
runs-on: linux-mi300-1gpu-ossci
defaults:
run:
shell: bash
Expand Down Expand Up @@ -82,7 +82,9 @@ jobs:

- name: Login to huggingface
continue-on-error: true
run: huggingface-cli login --token ${{ secrets.HF_TOKEN }}
run: |
pip install -U "huggingface_hub[cli]"
huggingface-cli login --token ${{ secrets.HF_TOKEN }}

- name: Run Shortfin Benchmark Tests
run: |
Expand All @@ -101,7 +103,7 @@ jobs:
matrix:
version: [3.11]
fail-fast: false
runs-on: mi300x-3
runs-on: linux-mi300-1gpu-ossci
defaults:
run:
shell: bash
Expand Down Expand Up @@ -187,15 +189,11 @@ jobs:
needs: benchmark_sglang
name: "Docker Cleanup"
if: always()
runs-on: mi300x-3
runs-on: linux-mi300-1gpu-ossci
steps:
- name: Stop sglang-server
run: docker stop sglang-server || true # Stop container if it's running

# Deleting image after run due to large disk space requirement (83 GB)
- name: Cleanup SGLang Image
run: docker image rm lmsysorg/sglang:v0.3.5.post1-rocm620

merge_and_upload_reports:
name: "Merge and upload benchmark reports"
needs: [benchmark_shortfin, benchmark_sglang]
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/ci-sglang-integration-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
matrix:
version: [3.11]
fail-fast: false
runs-on: mi300x-3
runs-on: linux-mi300-1gpu-ossci
defaults:
run:
shell: bash
Expand Down Expand Up @@ -69,7 +69,6 @@ jobs:
pip install sentence_transformers

pip freeze

- name: Run Integration Tests
run: |
source ${VENV_DIR}/bin/activate
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/ci-sharktank.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,7 @@ jobs:
env:
VENV_DIR: ${{ github.workspace }}/.venv
HF_HOME: "/data/huggingface"
HF_TOKEN: ${{ secrets.HF_FLUX_TOKEN }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can remove this for now too

steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2

Expand Down
2 changes: 1 addition & 1 deletion app_tests/benchmark_tests/llm/sglang_benchmarks/shortfin_benchmark_test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ def test_shortfin_benchmark(
request,
):
# TODO: Remove when multi-device is fixed
os.environ["ROCR_VISIBLE_DEVICES"] = "1"
os.environ["ROCR_VISIBLE_DEVICES"] = "0"

process, port = server

Expand Down
2 changes: 1 addition & 1 deletion app_tests/integration_tests/llm/sglang/conftest.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ def model_artifacts(request, tmp_path_factory):

@pytest.fixture(scope="module")
def start_server(request, model_artifacts):
os.environ["ROCR_VISIBLE_DEVICES"] = "1"
os.environ["ROCR_VISIBLE_DEVICES"] = "0"
device_settings = request.param["device_settings"]

server_config = ServerConfig(
Expand Down
Loading