Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
sasha0552 authored Sep 3, 2024
1 parent f43deff commit dac1839
Showing 1 changed file with 49 additions and 23 deletions.
72 changes: 49 additions & 23 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,41 @@ The main repository for building Pascal-compatible versions of ML applications a

## Installation

I recommend installing [transient-package](https://pypi.org/project/transient-package) before proceeding. It simplifies the installation of `triton`.

You can install it globally with `pipx`:

```sh
pipx install transient-package
```

> [!IMPORTANT]
> <details>
> <summary>If you don't want to install transient-package</summary>
>
> If you don't want to install `transient-package`, you'll need to replace
>
> ```sh
> transient-package install \
> --interpreter venv/bin/python \
> --source triton \
> --target triton-pascal
> ```
>
> with
>
> ```sh
> # Remove triton
> pip uninstall triton
>
> # Install patched triton
> pip install triton-pascal
> ```
>
> Note that `transient-package` does more than just `pip uninstall triton` and `pip install triton-pascal`.
> In particular, it tries to install the correct version of `triton`, and creates a bogus `triton` package in case the application checks for the presence of `triton`.
> </details>
### [vllm](https://github.com/vllm-project/vllm)
*Note: this repository holds "nightly" builds of vLLM.*
Expand All @@ -17,27 +52,24 @@ The main repository for building Pascal-compatible versions of ML applications a
export PIP_EXTRA_INDEX_URL="https://sasha0552.github.io/pascal-pkgs-ci/"
# Create virtual environment
python3 -m venv venv
python -m venv venv
# Activate virtual environment
source venv/bin/activate
# Install vLLM
pip3 install vllm-pascal
# Remove triton
pip3 uninstall triton

# Install patched triton
pip3 install triton-pascal
transient-package install \
--interpreter venv/bin/python \
--source triton \
--target triton-pascal
# Launch vLLM
vllm serve --help
```
> [!NOTE]
Installation will be simplified in the future.

#### To update a patched vLLM between same vLLM release versions (e.g. `0.5.0` (commit `000000`) -> `0.5.0` (commit `ffffff`)):
```sh
Expand All @@ -52,7 +84,7 @@ pip3 install --force-reinstall --no-cache-dir --no-deps --upgrade vllm-pascal
```
> [!WARNING]
In rare cases, this may cause dependency errors; in that case, just reinstall vLLM.
> In rare cases, this may cause dependency errors; in that case, just reinstall vLLM.
### [aphrodite-engine](https://github.com/PygmalionAI/aphrodite-engine)
Expand All @@ -70,19 +102,16 @@ source venv/bin/activate
# Install aphrodite-engine
pip3 install --extra-index-url https://downloads.pygmalion.chat/whl aphrodite-engine
# Remove triton
pip3 uninstall triton

# Install patched triton
pip3 install triton-pascal
transient-package install \
--interpreter venv/bin/python \
--source triton \
--target triton-pascal
# Launch aphrodite-engine
aphrodite --help
```
> [!NOTE]
Installation will be simplified in the future.

### [triton](https://github.com/triton-lang/triton) (for other applications)
#### To install patched Triton:
Expand All @@ -91,16 +120,13 @@ Installation will be simplified in the future.
# Use this repository
export PIP_EXTRA_INDEX_URL="https://sasha0552.github.io/pascal-pkgs-ci/"
# Remove triton
pip3 uninstall triton

# Install patched triton
pip3 install triton-pascal
transient-package install \
--interpreter venv/bin/python \
--source triton \
--target triton-pascal
```
> [!NOTE]
Installation will be simplified in the future.

---
<details>
Expand Down

0 comments on commit dac1839

Please sign in to comment.