-
Notifications
You must be signed in to change notification settings - Fork 69
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Putting Generic Authentication flow to the Core #694
Putting Generic Authentication flow to the Core #694
Conversation
@geoand @sberyozkin, I am putting this as a draft here in reference to #626 , is this what you were looking for? I am going to update the documentation once we agree that this is what's needed. I opted to keep the usage of the I welcome every proposal / change to this logic. |
Hi @csotiriou Thanks for starting looking into it. Yes, I think this is what we thought about with Georgios.
+1. I'll do the same for Vertex Gemini and subsequently we can do it for all other remote model providers as necessary. Thanks |
Very nice, thanks! The formatter is complaining:
Furthermore, please rebase the PR onto |
core/runtime/src/main/java/io/quarkiverse/langchain4j/runtime/auth/ModelAuthProvider.java
Outdated
Show resolved
Hide resolved
7d23f5d
to
832ab48
Compare
@geoand @sberyozkin thanks, I just corrected the documentation, since atm OpenAI is the only module that is using it, I suppose we don't need to move it elsewhere. If we want to move it elsewhere, I wonder where this might be (a new section, perhaps?) |
Thanks @csotiriou Having model provider sections like it is done for OpenAI is good IMHO, perhaps, once I have it supported for one more model provider, Vertex Gemini, it would make sense to have a higher level short section on |
Hey folks, sorry for the delay, I was travelling last week and a ton of stuff had piled up! I will review shortly |
...oyment/src/main/java/io/quarkiverse/langchain4j/openai/deployment/OpenAiCommonProcessor.java
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks a lot! I added a small comment
e13e938
to
3c9bbb2
Compare
putting the resolve methods in the interface, not in the embedded Resolver class updating documentation formatting added forgotten unremovable beans configuration Update docs/modules/ROOT/pages/openai.adoc Co-authored-by: Georgios Andrianakis <geoand@gmail.com>
a6318c1
to
90bccba
Compare
Thanks a lot for this! @sberyozkin do you want to test it locally to make sure if fits your needs? |
Hi @geoand @csotiriou , sure, give me a few days please, I'll rework vertex gemini PR, cheers |
|
||
public OpenAIRestAPIFilter(AuthProvider authorizer) { | ||
public OpenAIRestAPIFilter(ModelAuthProvider authorizer) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IMHO it should be aligned with how it is done for Vertex AI Gemini TokenFilter
, to have ExecutorService
injected as well and then call a custom provider from this ExecutorService
as providers may want to use some blocking calls etc. Using ManagedExecutor
is best as it retains the original RequestScope
which lets users inject Quarkus API beans which require it, as proposed in #708.
We confirmed with Georgios that using ManagedExecutot helps in the secure-fraud-detection demo
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point!
import io.quarkiverse.langchain4j.ModelName; | ||
|
||
public interface ModelAuthProvider { | ||
String getAuthorization(Input input); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @csotiriou @geoand, only a couple of questions/suggestions for you to consider.
I was thinking, can we expect situations going forward where local models will require a secure token access, and where a token is passed, for example, as a property ?
I guess one can say that if Input
is null then do not return a Bearer
prefix in the response... This is related to the next question...
The other question, so ModelAuthProvider
is expected to return a complete HTTP Authorization
header value, including the scheme, Bearer <token>
, instead of only <token>
. In #708, moving to ModelAuthProvider
has an impact of having to update the test filter to include Bearer
- I don't think it is a breaking change though, since the current AuthProvider
is not public.
I wonder, should we expect providers returns the token only, and then filters, depending on a context (access to the remote or local model) will either add Bearer
or not.
IMHO it might be a bit better, and a little bit simpler for custom auth providers too...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think we need to worry about that now, it's very early days to be able to know exactly how systems using auth will evolve.
For now, I think the proposal in this PR is fine.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Implementation of request mentioned in #626