Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Request] Allow custom openai endpoint #1424

Closed
lenaxia opened this issue Dec 19, 2023 · 3 comments · Fixed by #1413
Closed

[Request] Allow custom openai endpoint #1424

lenaxia opened this issue Dec 19, 2023 · 3 comments · Fixed by #1413
Labels
stale

Comments

@lenaxia
Copy link

lenaxia commented Dec 19, 2023

Due to changes in PrivateGPT, openai replacements no longer work as we cannot define custom openai endpoints.

LocalAI is an example: https://github.com/mudler/LocalAI/tree/master/examples/privateGPT

It is a drop in replacement for ChatGPT Openai with compatible REST APIs. Being able to define custom openai endpoints in the settings files would enable us to use localai.
@creasydude
Copy link

any updates?

@mchill
Copy link
Contributor

mchill commented Dec 21, 2023

This is pretty easy to implement with OpenAILike. I tested it briefly at first with LocalAI, and then with vLLM. I also need this feature, so I'll get around to opening a PR if no one else picks it up.

mchill added a commit to mchill/privateGPT that referenced this issue Dec 22, 2023
@mchill
feat(llm): add openailike llm mode
59e7bd5 
This mode behaves the same as the openai mode, except that it allows setting custom models not
supported by OpenAI. It can be used with any tool that serves models from an OpenAI compatible API.

Implements zylon-ai#1424
@mchill mchill mentioned this issue Dec 22, 2023
Merged
imartinez pushed a commit that referenced this issue Dec 26, 2023
@mchill
feat(llm): Add openailike llm mode (#1447)
2d27a9f 
This mode behaves the same as the openai mode, except that it allows setting custom models not
supported by OpenAI. It can be used with any tool that serves models from an OpenAI compatible API.

Implements #1424
@github-actions github-actions bot mentioned this issue Dec 26, 2023
Merged
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 6, 2024

Stale issue

@github-actions github-actions bot added the stale label Jan 6, 2024
@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Feb 1, 2024
simonbermudez pushed a commit to simonbermudez/saimon that referenced this issue Feb 24, 2024
@mchill @simonbermudez
feat(llm): Add openailike llm mode (zylon-ai#1447)
ae8cbfd 
This mode behaves the same as the openai mode, except that it allows setting custom models not
supported by OpenAI. It can be used with any tool that serves models from an OpenAI compatible API.

Implements zylon-ai#1424
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
stale
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

chore(main): release 0.3.0 zylon-ai/private-gpt
3 participants
@lenaxia @mchill @creasydude