Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ollama support #1028

Closed
BodhiHu opened this issue Sep 10, 2023 · 2 comments · Fixed by #2628
Closed

Ollama support #1028

BodhiHu opened this issue Sep 10, 2023 · 2 comments · Fixed by #2628
Labels
area/backends enhancement New feature or request roadmap up for grabs Tickets that no-one is currently working on

Comments

@BodhiHu
Copy link

BodhiHu commented Sep 10, 2023

Is your feature request related to a problem? Please describe.

Hello, I tried ollama on my macbook and got pretty good performance compared to running LocalAI with llama-stable directly(which consumes lots of CPU and not using GPU at all):

Screenshot 2023-08-24 at 18 07 54

While Ollama will use the GPU and so saves CPU, but unfortunately ollama did not have OpenAI like API.

Describe the solution you'd like
Add support for ollama.

Describe alternatives you've considered

Had not found one proper.

Additional context

Thanks a lot :D

@BodhiHu BodhiHu added the enhancement New feature or request label Sep 10, 2023
@mudler
Copy link
Owner

mudler commented Sep 10, 2023

While I can understand the request of another backend, ollama isn't really adding any features that LocalAI hasn't already.

Did you followed the docs to use GPU? https://localai.io/basics/getting_started/#cuda for metal
https://localai.io/basics/build/#metal-apple-silicon

Ollama isn't in the roadmap, however feel free to take a stab at it.

@mudler mudler added the roadmap label Nov 13, 2023
@mudler
Copy link
Owner

mudler commented Nov 13, 2023

come up again on X, up for grabs :)

@mudler mudler removed their assignment Nov 13, 2023
@mudler mudler added the up for grabs Tickets that no-one is currently working on label Nov 13, 2023
mudler added a commit that referenced this issue Jun 21, 2024
Fixes: #2527
Fixes: #1028

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
mudler added a commit that referenced this issue Jun 21, 2024
Fixes: #2527
Fixes: #1028

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
mudler added a commit that referenced this issue Jun 21, 2024
Fixes: #2527
Fixes: #1028

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
mudler added a commit that referenced this issue Jun 21, 2024
Fixes: #2527
Fixes: #1028

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
@mudler mudler closed this as completed in f569237 Jun 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/backends enhancement New feature or request roadmap up for grabs Tickets that no-one is currently working on
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants