Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Llamacpp with cpp backend #2527

Closed
wants to merge 11 commits into from
Closed

Llamacpp with cpp backend #2527

wants to merge 11 commits into from

Commits on Aug 16, 2023

  1. Version1 of llm inference with cpp backend 

    Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
    @shrinath-suresh
    shrinath-suresh committed Aug 16, 2023
    Configuration menu
    Copy the full SHA
    ee07277 View commit details
    Browse the repository at this point in the history

Commits on Aug 17, 2023

  1. Updating llm handler - loadmodel, preprocess, inference methods 

    Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
    @shrinath-suresh
    shrinath-suresh committed Aug 17, 2023
    Configuration menu
    Copy the full SHA
    8addde3 View commit details
    Browse the repository at this point in the history

Commits on Aug 21, 2023

  1. Fixed infinite lock by adding request ids to the preprocess method 

    Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
    @shrinath-suresh
    shrinath-suresh committed Aug 21, 2023
    Configuration menu
    Copy the full SHA
    002e221 View commit details
    Browse the repository at this point in the history

  2. Adding test script for finding tokens per second llama-7b-chat and gg… 

    …ml version

Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
    @shrinath-suresh
    shrinath-suresh committed Aug 21, 2023
    Configuration menu
    Copy the full SHA
    f351d1d View commit details
    Browse the repository at this point in the history

Commits on Sep 13, 2023

  1. GGUF Compatibility 

    Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
    @shrinath-suresh
    shrinath-suresh committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    e3a753c View commit details
    Browse the repository at this point in the history

  2. Fixing unit tests 

    Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
    @shrinath-suresh
    shrinath-suresh committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    c4ab8a1 View commit details
    Browse the repository at this point in the history

  3. Fix typo 

    Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
    @shrinath-suresh
    shrinath-suresh committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    8cdda73 View commit details
    Browse the repository at this point in the history

  4. Using folly to read config path 

    Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
    @shrinath-suresh
    shrinath-suresh committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    026b836 View commit details
    Browse the repository at this point in the history

  5. Removing debug couts 

    Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
    @shrinath-suresh
    shrinath-suresh committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    c95a576 View commit details
    Browse the repository at this point in the history

Commits on Sep 15, 2023

  1. Processing all the items in the batch 

    Signed-off-by: Shrinath Suresh <shrinath@ideas2it.com>
    @shrinath-suresh
    shrinath-suresh committed Sep 15, 2023
    Configuration menu
    Copy the full SHA
    eda470f View commit details
    Browse the repository at this point in the history

Commits on Jan 12, 2024

  1. Adopted llama.cpp api changes

    @mreso
    mreso committed Jan 12, 2024
    Configuration menu
    Copy the full SHA
    28c9b02 View commit details
    Browse the repository at this point in the history