abetlen llama-cpp-python · Discussions · GitHub

Sort by: Latest activity

Categories

Discussions

You must be logged in to vote

Huggingface GGUF Editor

CISC started Sep 1, 2024 in Show and tell

1
You must be logged in to vote

Use model for multiple user sessions

riebers-m started Sep 4, 2024 in General

6
You must be logged in to vote

How do I specify the chat format for the phi3.5 model? It seems like no one knows

suhyun01150 asked Sep 4, 2024 in Q&A · Unanswered

0
You must be logged in to vote

How to allow parallel inference requests

webhaypac asked Sep 3, 2024 in Q&A · Unanswered

0
You must be logged in to vote

Copilot support still working?

raghur asked Dec 21, 2023 in Q&A · Closed · Unanswered

4
You must be logged in to vote

ROPE failing for llama 3.1 GGUF using llamacpp python versions >0.2.84

shaunck96 started Aug 28, 2024 in General

0
You must be logged in to vote

GPU gets used and BLAS parameter is 1 when I download the CUDA supported llamacpp python version from this url (https://jllllll.github.io/llama-cpp-python-cuBLAS-wheels/AVX2/cu117/llama-cpp-python/)

shaunck96 started Aug 27, 2024 in General

0
You must be logged in to vote

Critical Vulnerability CVE-2024-34359

shaunck96 started Aug 26, 2024 in General

0
You must be logged in to vote

Example with stream = True?

matthiasgeihs asked Jun 5, 2023 in Q&A · Answered

4
You must be logged in to vote

Does chat_format need to be explicitly specified when working with chat completions?

laelhalawani asked Aug 25, 2024 in Q&A · Unanswered

0
You must be logged in to vote

How can I run 8bit models?

kadirnar started Jul 30, 2024 in General

1
You must be logged in to vote

How to make it work with CUDA-support?

itinance asked Aug 18, 2024 in Q&A · Unanswered

1
You must be logged in to vote

How to choose tokens manually instead of solely relying on the PRNG to pick them based on the random seed?

42e6 asked Aug 12, 2024 in Q&A · Unanswered

2
You must be logged in to vote

What need to be done to do function calling with a custom GGUF model?

hpx502766238 asked Aug 16, 2024 in Q&A · Unanswered

0
You must be logged in to vote

How to run model https://huggingface.co/FiditeNemini/Llama-3.1-Unhinged-Vision-8B-GGUF

Vorobey1234 asked Aug 15, 2024 in Q&A · Closed · Unanswered

0
You must be logged in to vote

Support for -ctk and -ctv Flags

Spider-netizen asked Aug 12, 2024 in Q&A · Unanswered

0
You must be logged in to vote

Unable to load llama 3.1 8 B. Please help me

rites1095 started Jul 31, 2024 in General

1
You must be logged in to vote

Format output as json

talhalatifkhan asked Aug 15, 2023 in Q&A · Unanswered

5
You must be logged in to vote

What need to be done to enable automatic function calling & "auto" tool choice?

samuelint asked Jul 23, 2024 in Q&A · Unanswered

2
You must be logged in to vote

[GUIDE] How to ACTUALLY build with cuBLAS support on windows

tk-master started Nov 4, 2023 in General

13
You must be logged in to vote

Building Dockerfile with CUDA support and smaller image.

jflachman asked Jul 19, 2024 in Q&A · Answered

2
You must be logged in to vote

Add support for Mistral NeMo

SeBL4RD started Jul 20, 2024 in General

0
You must be logged in to vote

Does OpenAI Compatible Web Server for Llama server support cuda?

RakshitAralimatti asked Jul 16, 2024 in Q&A · Unanswered

0
You must be logged in to vote

T5 architecture support

mtsdurica started Jul 14, 2024 in Ideas

0
You must be logged in to vote

Flask Api for LLM inference. How to make this system concurrent.

rites1095 started Jul 12, 2024 in General

0