Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement GPTQ quantization #467

Merged
merged 65 commits into from
Aug 9, 2024
Merged

Implement GPTQ quantization #467

merged 65 commits into from
Aug 9, 2024

Commits on Jun 19, 2024

  1. Add the kernels

    @EricLBuehler
    EricLBuehler committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    3c313e2 View commit details
    Browse the repository at this point in the history

  2. Remove include of aten or torch

    @EricLBuehler
    EricLBuehler committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    a908822 View commit details
    Browse the repository at this point in the history

Commits on Jun 22, 2024

  1. Merge branch 'master' into gptq

    @EricLBuehler
    EricLBuehler committed Jun 22, 2024
    Configuration menu
    Copy the full SHA
    c0e0492 View commit details
    Browse the repository at this point in the history

Commits on Jun 23, 2024

  1. Merge branch 'master' into gptq

    @EricLBuehler
    EricLBuehler committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    24b32ae View commit details
    Browse the repository at this point in the history

Commits on Jun 24, 2024

  1. Add the ffi bindings

    @EricLBuehler
    EricLBuehler committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    a0674bc View commit details
    Browse the repository at this point in the history

  2. Sketch the forward method

    @EricLBuehler
    EricLBuehler committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    c988419 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'master' into gptq

    @EricLBuehler
    EricLBuehler committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    041c00f View commit details
    Browse the repository at this point in the history

Commits on Jun 25, 2024

  1. Handle input and output reshapes

    @EricLBuehler
    EricLBuehler committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    420d497 View commit details
    Browse the repository at this point in the history

  2. Add some features

    @EricLBuehler
    EricLBuehler committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    fd33c32 View commit details
    Browse the repository at this point in the history

  3. Improve compat of build.rs

    @EricLBuehler
    EricLBuehler committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    93bd2dc View commit details
    Browse the repository at this point in the history

  4. Fix workspace dep

    @EricLBuehler
    EricLBuehler committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    0f718a5 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'master' into gptq

    @EricLBuehler
    EricLBuehler committed Jun 25, 2024
    Configuration menu
    Copy the full SHA
    821ca1e View commit details
    Browse the repository at this point in the history

Commits on Jun 26, 2024

  1. Merge branch 'master' into gptq

    @EricLBuehler
    EricLBuehler committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    adfbf4b View commit details
    Browse the repository at this point in the history

Commits on Jun 29, 2024

  1. Merge branch 'master' into gptq

    @EricLBuehler
    EricLBuehler committed Jun 29, 2024
    Configuration menu
    Copy the full SHA
    636523d View commit details
    Browse the repository at this point in the history

Commits on Jun 30, 2024

  1. Merge branch 'master' into gptq

    @EricLBuehler
    EricLBuehler committed Jun 30, 2024
    Configuration menu
    Copy the full SHA
    6316412 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into gptq

    @EricLBuehler
    EricLBuehler committed Jun 30, 2024
    Configuration menu
    Copy the full SHA
    7fcee44 View commit details
    Browse the repository at this point in the history

Commits on Jul 1, 2024

  1. Merge branch 'master' into gptq

    @EricLBuehler
    EricLBuehler committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    d5b1b28 View commit details
    Browse the repository at this point in the history

Commits on Jul 3, 2024

  1. Merge branch 'master' into gptq

    @EricLBuehler
    EricLBuehler committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    c4cd676 View commit details
    Browse the repository at this point in the history

  2. Finish merge

    @EricLBuehler
    EricLBuehler committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    865dfe1 View commit details
    Browse the repository at this point in the history

  3. Fixes

    @EricLBuehler
    EricLBuehler committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    942d971 View commit details
    Browse the repository at this point in the history

  4. Finish gptq gemm and add trait

    @EricLBuehler
    EricLBuehler committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    a1ae220 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'master' into gptq

    @EricLBuehler
    EricLBuehler committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    625f186 View commit details
    Browse the repository at this point in the history

  6. Add the cuda gptq matmul stub

    @EricLBuehler
    EricLBuehler committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    95a4468 View commit details
    Browse the repository at this point in the history

  7. Remove default feature

    @EricLBuehler
    EricLBuehler committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    4d2d743 View commit details
    Browse the repository at this point in the history

  8. Correct conditional comp

    @EricLBuehler
    EricLBuehler committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    f300f96 View commit details
    Browse the repository at this point in the history

Commits on Jul 4, 2024

  1. Add gguf qmatmul quantized support

    @EricLBuehler
    EricLBuehler committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    ab6fb3c View commit details
    Browse the repository at this point in the history

  2. Implement matmul with qmethod in qllama

    @EricLBuehler
    EricLBuehler committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    04dbbd6 View commit details
    Browse the repository at this point in the history

  3. Update readme of mistralrs quant

    @EricLBuehler
    EricLBuehler committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    cbbda7a View commit details
    Browse the repository at this point in the history

  4. int* to int64* in q_gemm.cu

    @EricLBuehler
    EricLBuehler committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    b7bf6c1 View commit details
    Browse the repository at this point in the history

  5. Add model and pipeline

    @EricLBuehler
    EricLBuehler committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    bad3ad5 View commit details
    Browse the repository at this point in the history

  6. Add gptq loader selector

    @EricLBuehler
    EricLBuehler committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    c1eaf6b View commit details
    Browse the repository at this point in the history

  7. Rename quantized_config -> quantization_config

    @EricLBuehler
    EricLBuehler committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    7c30b88 View commit details
    Browse the repository at this point in the history

  8. Fix g_idx shape

    @EricLBuehler
    EricLBuehler committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    132196f View commit details
    Browse the repository at this point in the history

  9. Ensure WNA16

    @EricLBuehler
    EricLBuehler committed Jul 4, 2024
    Configuration menu
    Copy the full SHA
    748af66 View commit details
    Browse the repository at this point in the history

Commits on Jul 5, 2024

  1. Broadcast add

    @EricLBuehler
    EricLBuehler committed Jul 5, 2024
    Configuration menu
    Copy the full SHA
    f564303 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into gptq

    @EricLBuehler
    EricLBuehler committed Jul 5, 2024
    Configuration menu
    Copy the full SHA
    7d9727c View commit details
    Browse the repository at this point in the history

  3. Format

    @EricLBuehler
    EricLBuehler committed Jul 5, 2024
    Configuration menu
    Copy the full SHA
    b9b4cea View commit details
    Browse the repository at this point in the history

  4. Merge branch 'master' into gptq

    @EricLBuehler
    EricLBuehler committed Jul 5, 2024
    Configuration menu
    Copy the full SHA
    ca23890 View commit details
    Browse the repository at this point in the history

  5. int64_t* -> int* rollback

    @EricLBuehler
    EricLBuehler committed Jul 5, 2024
    Configuration menu
    Copy the full SHA
    b3e798b View commit details
    Browse the repository at this point in the history

  6. Prep for correct types

    @EricLBuehler
    EricLBuehler committed Jul 5, 2024
    Configuration menu
    Copy the full SHA
    d860a9a View commit details
    Browse the repository at this point in the history

Commits on Jul 15, 2024

  1. Merge branch 'master' into gptq

    @EricLBuehler
    EricLBuehler committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    5fc7940 View commit details
    Browse the repository at this point in the history

  2. Finish merge

    @EricLBuehler
    EricLBuehler committed Jul 15, 2024
    Configuration menu
    Copy the full SHA
    cdee88c View commit details
    Browse the repository at this point in the history

Commits on Jul 22, 2024

  1. Merge branch 'master' into gptq

    @EricLBuehler
    EricLBuehler committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    dcc5b1c View commit details
    Browse the repository at this point in the history

  2. Complete merge

    @EricLBuehler
    EricLBuehler committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    88a080e View commit details
    Browse the repository at this point in the history

Commits on Jul 24, 2024

  1. Merge branch 'master' into gptq

    @EricLBuehler
    EricLBuehler committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    ea6a6ee View commit details
    Browse the repository at this point in the history

  2. Update cargo lock

    @EricLBuehler
    EricLBuehler committed Jul 24, 2024
    Configuration menu
    Copy the full SHA
    37521e5 View commit details
    Browse the repository at this point in the history

Commits on Jul 26, 2024

  1. Merge branch 'master' into gptq

    @EricLBuehler
    EricLBuehler committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    21015f6 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into gptq

    @EricLBuehler
    EricLBuehler committed Jul 26, 2024
    Configuration menu
    Copy the full SHA
    c152035 View commit details
    Browse the repository at this point in the history

Commits on Aug 8, 2024

  1. Merge from master

    @EricLBuehler
    EricLBuehler committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    1f21ef8 View commit details
    Browse the repository at this point in the history

  2. Integrate with new i32 type

    @EricLBuehler
    EricLBuehler committed Aug 8, 2024
    Configuration menu
    Copy the full SHA
    02fdd53 View commit details
    Browse the repository at this point in the history

Commits on Aug 9, 2024

  1. Fixes

    @EricLBuehler
    EricLBuehler committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    72c5752 View commit details
    Browse the repository at this point in the history

  2. More progress

    @EricLBuehler
    EricLBuehler committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    14552b0 View commit details
    Browse the repository at this point in the history

  3. It doesnt crash

    @EricLBuehler
    EricLBuehler committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    8298a5d View commit details
    Browse the repository at this point in the history

  4. Oops

    @EricLBuehler
    EricLBuehler committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    32e579f View commit details
    Browse the repository at this point in the history

  5. It works!

    @EricLBuehler
    EricLBuehler committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    08485db View commit details
    Browse the repository at this point in the history

  6. Remove some todos

    @EricLBuehler
    EricLBuehler committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    5d56f47 View commit details
    Browse the repository at this point in the history

  7. Testing isq support

    @EricLBuehler
    EricLBuehler committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    ca7d31a View commit details
    Browse the repository at this point in the history

  8. Add to all non adapter models

    @EricLBuehler
    EricLBuehler committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    4efa533 View commit details
    Browse the repository at this point in the history

  9. Clippy

    @EricLBuehler
    EricLBuehler committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    49618dd View commit details
    Browse the repository at this point in the history

  10. Avoid reallocating

    @EricLBuehler
    EricLBuehler committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    0ace364 View commit details
    Browse the repository at this point in the history

  11. Add support for gptq to adapter models

    @EricLBuehler
    EricLBuehler committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    079cd05 View commit details
    Browse the repository at this point in the history

  12. Add docs and logging

    @EricLBuehler
    EricLBuehler committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    f72f19b View commit details
    Browse the repository at this point in the history

  13. Update docs

    @EricLBuehler
    EricLBuehler committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    e65774c View commit details
    Browse the repository at this point in the history

  14. Clippy

    @EricLBuehler
    EricLBuehler committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    f5206a0 View commit details
    Browse the repository at this point in the history

  15. Remove a todo

    @EricLBuehler
    EricLBuehler committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    61a8530 View commit details
    Browse the repository at this point in the history