Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement HQQ quantization #677

Merged
merged 31 commits into from
Aug 16, 2024
Merged

Implement HQQ quantization #677

merged 31 commits into from
Aug 16, 2024

Conversation

EricLBuehler
Copy link
Owner

@EricLBuehler EricLBuehler commented Aug 11, 2024

This adds support for HQQ quantization in 8, 4, 3, 2, and 1 bit.

Still need to add:

  • CPU dequant with simd accel, Metal dequant
  • Quantization optimizer

Copy link

github-actions bot commented Aug 11, 2024

Code Metrics Report
  ===============================================================================
 Language            Files        Lines         Code     Comments       Blanks
===============================================================================
 C Header                2           35           28            0            7
 Dockerfile              1           34           25            0            9
 Happy                   1          442          369            0           73
 JSON                   11          102          101            0            1
 Python                 45         1993         1695           62          236
 TOML                   20          606          535           11           60
-------------------------------------------------------------------------------
 Jupyter Notebooks       4            0            0            0            0
 |- Markdown             2           77           32           31           14
 |- Python               2          196          169            1           26
 (Total)                            273          201           32           40
-------------------------------------------------------------------------------
 Markdown               27         1894            0         1427          467
 |- BASH                 5          101           98            0            3
 |- JSON                 1           12           12            0            0
 |- Python               5           92           82            0           10
 |- Rust                 6          408          365           19           24
 |- TOML                 2           75           63            0           12
 (Total)                           2582          620         1446          516
-------------------------------------------------------------------------------
 Rust                  184        57550        52275         1007         4268
 |- Markdown            94          873           13          810           50
 (Total)                          58423        52288         1817         4318
===============================================================================
 Total                 296        62656        55028         2507         5121
===============================================================================
  

@EricLBuehler EricLBuehler merged commit fd90d9a into master Aug 16, 2024
17 checks passed
@EricLBuehler EricLBuehler deleted the hqq branch August 16, 2024 00:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant