Add support for Ollama LLM

[ygalblum]

Allow using Ollama as the LLM

[Fau57]

AGREED!

[oatmealm]

I'm trying this branch ... its seems to to want to build lllamacpp...

settings-ollama.yaml

ollama:
  model: llama2:latest

PGPT_PROFILES=ollama make run

  File "/home/user/src/privateGPT/private_gpt/components/llm/llm_component.py", line 38, in __init__
    self.llm = LlamaCPP(
               ^^^^^^^^^
  File "/home/user/.cache/pypoetry/virtualenvs/private-gpt-o6kY6e_Z-py3.11/lib/python3.11/site-packages/llama_index/llms/llama_cpp.py", line 119, in __init__
    raise ValueError(
ValueError: Provided model path does not exist. Please check the path or provide a model_url to download.

[ygalblum]

@oatmealm
IIUC according to https://github.com/imartinez/privateGPT/blob/main/private_gpt/settings/settings_loader.py#L36 setting PGPT_PROFILES instruct the code to load additional settings-{profile} .yml files. But, the base settings.yml is always loaded.

So, I think that you also need to set the llm mode:

llm:
  mode: ollama

[imartinez]

Amazing! Could you update the documentation so people can figure out how to use this?

https://docs.privategpt.dev/manual/advanced-setup/llm-backends

Just update the markdown in fern/docs/pages/manual/llms.mdx

[ygalblum]

Could you update the documentation so people can figure out how to use this?

Done

[github-actions]

Published docs preview URL: https://privategpt-preview-07644e6b-6bd8-4356-8df9-0943032914bd.docs.buildwithfern.com

[icsy7867]

This is great! Ollama now supports the openai api format. I have just been running privategpt using the "openailike" which has been working fine.

[skywalker123p]

How can I combine embeddings when using Ollama LLM? Please give example of configuration and how to run it? thanks