Run LocalAI with pre-installed llama.cpp

[ebudmada]

Hi, Thank for your works guys.

I believe this feature is not possible, but i guess it would be more simple for LocalAI to use the existent llama.cpp install on the current system, and add the feature of local API.

This would solve a lots of requests and bug in the issues section.

Let me know your thoughts about this, have a nice day!

[muunkky]

I have already downloaded an LLM file but I don't know how to reference it in PRELOAD_MODELS. It only seems to want URL values to download new ones (sorta defeats the purpose)

[Aisuko]

Hi, guys. Welcome.

For @ebudmada. Thanks for your idea. It is really cool. However, we have the repo go-llama.cpp. It is resposible for bounding llama.cpp. We need to make sure the features are tested before released. And some features need the specifc version of llama.cpp. So, I believe the bouding is the most reliable solution now.

For @muunkky. Have you try You can set the PRELOAD_MODELS environment variable in https://localai.io/models/#image-generation-stable-diffusion. Maybe it is helpful. And do not forget to check the model status.