Skip to content

Commit

Permalink
[AMD] [CI] Clean up amd-ci (#436)
Browse files Browse the repository at this point in the history
## Summary
<!--- This is a required section; please describe the main purpose of
this proposed code change. --->
This is to clean up the `amd-ci.yml` setup steps.

<!---
## Details
This is an optional section; is there anything specific that reviewers
should be aware of?
--->

## Testing Done
<!--- This is a required section; please describe how this change was
tested. --->

<!-- 
Replace BLANK with your device type. For example, A100-80G-PCIe

Complete the following tasks before sending your PR, and replace `[ ]`
with
`[x]` to indicate you have done them. 
-->

- Hardware Type: <BLANK>
- [x] run `make test` to ensure correctness
- [x] run `make checkstyle` to ensure code style
- [x] run `make test-convergence` to ensure convergence

---------

Co-authored-by: tjtanaa <tunjian.tan@embeddedllm.com>
  • Loading branch information
tjtanaa and tjtanaa authored Dec 9, 2024
1 parent fcba35a commit 515b491
Show file tree
Hide file tree
Showing 3 changed files with 15 additions and 27 deletions.
28 changes: 1 addition & 27 deletions .github/workflows/amd-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -56,35 +56,9 @@ jobs:
with:
python-version: '3.10'

- name: Check Docker Version
run: docker version

- name: Check Ubuntu version
run: lsb_release -a

- name: Check Hardware Specs
run: lscpu

- name: ROCM-SMI Output
run: |
rocm-smi
rocm-smi --showproductname
- name: Setup Dependencies
run: |
cp -r /opt/rocm/share/amd_smi ./
cd amd_smi
python -m pip install -e .
cd ..
python -m pip install pytest pytest-xdist pytest-rerunfailures pytest-flakefinder pytest-cpp
python -m pip uninstall -y torch torchvision
python -m pip install --pre \
torch==2.6.0.dev20241113+rocm6.2 \
'setuptools-scm>=8' \
torchvision==0.20.0.dev20241113+rocm6.2 \
--extra-index-url https://download.pytorch.org/whl/nightly/rocm6.2
python -m pip install triton==3.1.0 transformers==4.46.3
python -m pip install -e .[dev]
python -m pip install -e .[dev,amd] --extra-index-url https://download.pytorch.org/whl/nightly/rocm6.2
- name: List Python Environments
run: python -m pip list
Expand Down
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,8 @@ To install from source:
git clone https://github.com/linkedin/Liger-Kernel.git
cd Liger-Kernel
pip install -e .
# or if installing on amd platform
pip install -e .[amd] --extra-index-url https://download.pytorch.org/whl/nightly/rocm6.2 # rocm6.2
# or if using transformers
pip install -e .[transformers]
```
Expand Down
12 changes: 12 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ dependencies = [
"triton>=2.3.1",
]


[project.optional-dependencies]
transformers = [
"transformers~=4.0"
Expand All @@ -27,11 +28,22 @@ dev = [
"black>=24.4.2",
"isort>=5.13.2",
"pytest>=7.1.2",
"pytest-xdist",
"pytest-rerunfailures",
"datasets>=2.19.2",
"torchvision>=0.16.2",
"seaborn",
]

amd = [
"torch>=2.6.0.dev",
"setuptools-scm>=8",
"torchvision>=0.20.0.dev",
"triton>=3.0.0",
]



[tool.setuptools.packages.find]
where = ["src"]
include = ["liger_kernel", "liger_kernel.*"]
Expand Down

0 comments on commit 515b491

Please sign in to comment.