Skip to content

Commit

Permalink
Update README.md (#36)
Browse files Browse the repository at this point in the history
  • Loading branch information
Qubitium authored Jun 20, 2024
1 parent 3e0bc4a commit 6bf62cf
Showing 1 changed file with 6 additions and 7 deletions.
13 changes: 6 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,13 +76,6 @@ GPTQModel is currently Linux only and requires Torch/Cuda capable GPU from NVIDI

## Install

GPTQModel is available for Linux only. You can install the latest stable release of AutoGPTQ from pip with pre-built wheels:

| CUDA version | Installation | Built against PyTorch |
|---------------|------------------------------------------------|-----------------------|
| CUDA 12.1 | `pip install gptq-model --no-build-isolation` | 2.3.1+cu121 |


### Install from source

```bash
Expand All @@ -93,6 +86,12 @@ git clone https://github.com/ModelCloud/GPTQModel.git && cd GPTQModel
pip install -vvv --no-build-isolation .
```

### PIP (PENDING RELEASE)

```bash
pip install gptq-model --no-build-isolation
```

### Quantization and Inference

> warning: this is just a showcase of the usage of basic apis in GPTQModel, which uses only one sample to quantize a much small model, quality of quantized model using such little samples may not good.
Expand Down

0 comments on commit 6bf62cf

Please sign in to comment.