Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

minor: support flashinfer nightly #2295

Merged
merged 2 commits into from
Dec 1, 2024
Merged

minor: support flashinfer nightly #2295

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
db93a31
upd
zhyncs Dec 1, 2024
291bd54
fix
zhyncs Dec 1, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions .github/workflows/pr-test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,15 @@ on:
- "python/sglang/**"
- "test/**"
workflow_dispatch:
inputs:
version:
description: "FlashInfer version"
required: true
type: choice
default: 'release'
options:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use the choice method

- 'release'
- 'nightly'

concurrency:
group: pr-test-${{ github.ref }}
Expand All @@ -26,6 +35,8 @@ jobs:
uses: actions/checkout@v3

- name: Install dependencies
env:
FLASHINFER_REPO: ${{ inputs.version == 'nightly' && 'https://flashinfer.ai/whl/nightly/cu121/torch2.4/' || 'https://flashinfer.ai/whl/cu121/torch2.4/' }}
run: |
bash scripts/ci_install_dependency.sh

Expand All @@ -46,6 +57,8 @@ jobs:
uses: actions/checkout@v3

- name: Install dependencies
env:
FLASHINFER_REPO: ${{ inputs.version == 'nightly' && 'https://flashinfer.ai/whl/nightly/cu121/torch2.4/' || 'https://flashinfer.ai/whl/cu121/torch2.4/' }}
run: |
bash scripts/ci_install_dependency.sh

Expand All @@ -66,6 +79,8 @@ jobs:
uses: actions/checkout@v3

- name: Install dependencies
env:
FLASHINFER_REPO: ${{ inputs.version == 'nightly' && 'https://flashinfer.ai/whl/nightly/cu121/torch2.4/' || 'https://flashinfer.ai/whl/cu121/torch2.4/' }}
run: |
bash scripts/ci_install_dependency.sh

Expand All @@ -92,6 +107,8 @@ jobs:
uses: actions/checkout@v3

- name: Install dependencies
env:
FLASHINFER_REPO: ${{ inputs.version == 'nightly' && 'https://flashinfer.ai/whl/nightly/cu121/torch2.4/' || 'https://flashinfer.ai/whl/cu121/torch2.4/' }}
run: |
bash scripts/ci_install_dependency.sh

Expand Down Expand Up @@ -127,6 +144,8 @@ jobs:
uses: actions/checkout@v3

- name: Install dependencies
env:
FLASHINFER_REPO: ${{ inputs.version == 'nightly' && 'https://flashinfer.ai/whl/nightly/cu121/torch2.4/' || 'https://flashinfer.ai/whl/cu121/torch2.4/' }}
run: |
bash scripts/ci_install_dependency.sh

Expand Down Expand Up @@ -156,6 +175,8 @@ jobs:
uses: actions/checkout@v3

- name: Install dependencies
env:
FLASHINFER_REPO: ${{ inputs.version == 'nightly' && 'https://flashinfer.ai/whl/nightly/cu121/torch2.4/' || 'https://flashinfer.ai/whl/cu121/torch2.4/' }}
run: |
bash scripts/ci_install_dependency.sh

Expand Down Expand Up @@ -185,6 +206,8 @@ jobs:
uses: actions/checkout@v3

- name: Install dependencies
env:
FLASHINFER_REPO: ${{ inputs.version == 'nightly' && 'https://flashinfer.ai/whl/nightly/cu121/torch2.4/' || 'https://flashinfer.ai/whl/cu121/torch2.4/' }}
run: |
bash scripts/ci_install_dependency.sh

Expand All @@ -207,6 +230,8 @@ jobs:
uses: actions/checkout@v3

- name: Install dependencies
env:
FLASHINFER_REPO: ${{ inputs.version == 'nightly' && 'https://flashinfer.ai/whl/nightly/cu121/torch2.4/' || 'https://flashinfer.ai/whl/cu121/torch2.4/' }}
run: |
bash scripts/ci_install_dependency.sh

Expand Down
2 changes: 1 addition & 1 deletion python/sglang/check_env.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@

import torch

# List of packages to check versions for
# List of packages to check versions
PACKAGE_LIST = [
"sglang",
"flashinfer",
Expand Down
12 changes: 7 additions & 5 deletions scripts/ci_install_dependency.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,16 @@
"""
Install the dependency in CI.
"""
# Install the dependency in CI.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comments in bash use #

./killall_sglang.sh
# Use repo from environment variable, passed from GitHub Actions
FLASHINFER_REPO="${FLASHINFER_REPO:-https://flashinfer.ai/whl/cu121/torch2.4}"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Index URL does not need to include the flashinfer directory

SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
zhyncs marked this conversation as resolved.
Show resolved Hide resolved
bash "${SCRIPT_DIR}/killall_sglang.sh"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolve the issue of not finding the execution script


pip install --upgrade pip
pip install -e "python[all]" --find-links https://flashinfer.ai/whl/cu121/torch2.4/flashinfer/

# Force reinstall flashinfer
pip install flashinfer -i https://flashinfer.ai/whl/cu121/torch2.4/ --force-reinstall
pip install flashinfer -i ${FLASHINFER_REPO} --force-reinstall

pip install transformers==4.45.2 sentence_transformers accelerate peft

Expand Down
Loading