GPTQModel v1.7.0
What's Changed
⚡backend.MLX
added for runtime-conversion and execution of GPTQ models on Apple's MLX
framework on Apple Silicon (M1+). ⚡ Exports of gptq models to mlx also now possible. We have added mlx exported models to huggingface.co/ModelCloud.
⚡ lm_head quantization now fully support by GPTQModel without external pkg dependency.
🐛 Fixed setup.py
not correctly detecting incompatible setuptools
/wheel
pkgs.
- [CI] run tests with linux tag by @CSY-ModelCloud in #1067
- Add backend.MLX by @LRL-ModelCloud in #1061
- add mlx generate test by @CL-ModelCloud in #1069
- [CI] upload source in build step by @CSY-ModelCloud in #1070
- code review by @CL-ModelCloud in #1072
- [CI] install mlx by @CSY-ModelCloud in #1071
- Add option to quantize
lm_head
by @ZX-ModelCloud in #1037 - fix test_packing by @LRL-ModelCloud in #1073
- [CI] add mlx test by @CSY-ModelCloud in #1074
- [CI] fix ci relase env name by @CSY-ModelCloud in #1078
- update mlx test by @CSY-ModelCloud in #1079
- convert to mlx support desc_act true by @LRL-ModelCloud in #1082
- [CI] add extra-index-url for pip install by @CSY-ModelCloud in #1083
- catch module error for setup.py by @CSY-ModelCloud in #1084
Full Changelog: v1.6.1...v1.7.0