Skip to content

Commit

Permalink
Update README and version following 0.5.0 release (AutoGPTQ#397)
Browse files Browse the repository at this point in the history
  • Loading branch information
fxmarty authored Nov 2, 2023
1 parent 8b6429a commit 99b277a
Show file tree
Hide file tree
Showing 4 changed files with 10 additions and 9 deletions.
11 changes: 5 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,13 +62,12 @@ For perplexity comparison, you can turn to [here](https://github.com/qwopqwop200
## Installation

### Quick Installation
You can install the latest stable release of AutoGPTQ from pip with pre-built wheels compatible with PyTorch 2.0.1:
You can install the latest stable release of AutoGPTQ from pip with pre-built wheels compatible with PyTorch 2.1 and PyTorch nightly:

* For CUDA 11.7: `pip install auto-gptq`
* For CUDA 12.1: `pip install auto-gptq`
* For CUDA 11.8: `pip install auto-gptq --extra-index-url https://huggingface.github.io/autogptq-index/whl/cu118/`
* For RoCm 5.4.2: `pip install auto-gptq --extra-index-url https://huggingface.github.io/autogptq-index/whl/rocm542/`

**Warning:** These wheels are not expected to work on PyTorch nightly. Please install AutoGPTQ from source when using PyTorch nightly.
* For RoCm 5.6.1: `pip install auto-gptq --extra-index-url https://huggingface.github.io/autogptq-index/whl/rocm561/`
* For RoCm 5.7.1: `pip install auto-gptq --extra-index-url https://huggingface.github.io/autogptq-index/whl/rocm571/`

AutoGPTQ can be installed with the Triton dependency with `pip install auto-gptq[triton]` in order to be able to use the Triton backend (currently only supports linux, no 3-bits quantization).

Expand All @@ -92,7 +91,7 @@ You can set `BUILD_CUDA_EXT=0` to disable pytorch extension building, but this i
To install from source for AMD GPUs supporting RoCm, please specify the `ROCM_VERSION` environment variable. Example:

```
ROCM_VERSION=5.6 pip install -v .
ROCM_VERSION=5.6 pip install -v -e .
```

The compilation can be speeded up by specifying the `PYTORCH_ROCM_ARCH` variable ([reference](https://github.com/pytorch/pytorch/blob/7b73b1e8a73a1777ebe8d2cd4487eb13da55b3ba/setup.py#L132)) in order to build for a single target device, for example `gfx90a` for MI200 series devices.
Expand Down
4 changes: 3 additions & 1 deletion README_zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@
</p>
</h4>

Note: The English README is likely to be more up to date.

## 通向 v1.0.0 之路

嗨,社区的伙伴们,好久不见!很抱歉这段时间由于个人原因,我没能以较高的频率来更新这个项目。过去几周对我的职业生涯规划而言意义重大。在不久前,我正式告别了毕业后便加入两年之久的创业团队,非常感谢团队的领导和同事们给予我的信任与指导,让我能够在两年时间里飞速地成长;同时也十分感激团队允许我自 AutoGPTQ 项目创立以来一直无偿使用内部的 A100 GPU 服务器集群以完成各项实验与性能测评。(当然今后是无法继续使用了,因此**若有新的硬件赞助我将感激不尽**!)过去的两年里,我在这个团队中担任算法工程师的角色,负责基于大语言模型的对话系统架构设计与开发,我们曾成功推出一款名为 gemsouls 的产品,但不幸的是它已经停止运营。而现在,这个团队即将推出一款名为 [modelize](https://www.beta.modelize.ai/) 的新产品,**这是一个大模型原生的 AI 智能体平台,用户可以使用多个 AI 智能体搭建一个高度自动化的团队,让它们在工作流中相互合作,高效完成复杂的项目。**
Expand Down Expand Up @@ -331,4 +333,4 @@ print(
- 特别感谢 **Elias Frantar****Saleh Ashkboos****Torsten Hoefler****Dan Alistarh** 提出 **GPTQ** 算法并开源[代码](https://github.com/IST-DASLab/gptq)
- 特别感谢 **qwopqwop200**, 本项目中涉及到模型量化的代码主要参考自 [GPTQ-for-LLaMa](https://github.com/qwopqwop200/GPTQ-for-LLaMa/tree/cuda)

[![Star History Chart](https://api.star-history.com/svg?repos=PanQiwei/AutoGPTQ&type=Date)](https://star-history.com/#PanQiWei/AutoGPTQ&Date)
[![Star History Chart](https://api.star-history.com/svg?repos=PanQiwei/AutoGPTQ&type=Date)](https://star-history.com/#PanQiWei/AutoGPTQ&Date)
2 changes: 1 addition & 1 deletion auto_gptq/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
__version__ = "0.5.0.dev0"
__version__ = "0.6.0.dev0"
from .modeling import BaseQuantizeConfig
from .modeling import AutoGPTQForCausalLM
from .utils.peft_utils import get_gptq_peft_model
Expand Down
2 changes: 1 addition & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
os.environ["CXX"] = "g++"

common_setup_kwargs = {
"version": "0.5.0.dev0",
"version": "0.6.0.dev0",
"name": "auto_gptq",
"author": "PanQiWei",
"description": "An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.",
Expand Down

0 comments on commit 99b277a

Please sign in to comment.