Does not support google gemini

[KotlinFactory]

Google Gemini Pro 1.5 (especially the new experimental version) is one of the top models in the the LLM leaderboard, showcasing its exceptional capabilities and potential.

The model performs well especially in non-english languages such as German or English.

However currently, Spring AI only supports accessing Google Gemini through Google Vertex AI. This requires users to manage authentication through Google Cloud and prevents the use of the standalone Gemini API.

Another user also brought this up in #1247

This leads to:

• High friction and decreased Adoption: There is a reason Google released AI Studio. Forcing users through Vertex AI creates unnecessary friction and complexity, especially for those unfamiliar with Google Cloud. This is a significant turnoff for developers evaluating Spring AI, potentially leading them to abandon the framework altogether in favor of alternatives that offer a more straightforward integration with the standalone Gemini API.

• Cumbersome prototyping: The standalone API allows for rapid prototyping and testing of ideas without the need for complex infrastructure setup. This is particularly valuable for new developers who want to quickly explore the capabilities of Gemini and Spring AI without getting bogged down in configuration and deployment hurdles.

The lack of support for the standalone Gemini API in Spring AI is a significant limitation and forces users to either abandon Spring AI or refactor their codebase to use the Google Cloud SDK directly.

Expected behavior

Spring AI should provide support for the standalone Google Gemini API, similar to how other language models are integrated. This would include:
Configuration options for specifying API keys and endpoints.
Abstractions similar to existing language model integrations to streamline usage and maintain consistency.
Documentation outlining how to configure and use the standalone Gemini API within the Spring AI framework.

Additional context

In the ReadMe of the project the relatively minor role of Java in the AI landscape is acknowledged.
Tools like LangChain have gained immense popularity by focusing on developer-friendliness and providing a smooth onboarding experience.
By removing barriers to entry and simplifying integrations, like enabling access to the standalone Google Gemini API, Spring AI can attract a wider audience and solidify its position as a leading framework for AI development (not just) in Java/ the JVM space.

[ziodave]

Let's see how it goes for you 😇

[youngmoneee]

I wanted to try it out as well. I’m just waiting for the branch to open. 👍

[tzolov]

@KotlinFactory , @ziodave, @youngmoneee the truth of the matter is, Google Gemini API does NOT provide Java Client.
So I guess you should raise an issue somewhere on their GitHub repose (good luck to find anything reliable there).

The other option is to try to build a low-level Spring AI Gemini client on top of the REST API, as we've done for many other providers including the , now deprecated, Google Vertex PaLM2
To commit time and resources for the later we need some re-assurance that this API won't disappear in next 6 months.

Honestly Google's Dev docs and API management is amongst the most confusing out there. Would appreciate if you can point me to reliable doc and code sources.

[youngmoneee]

@tzolov in situations where Google’s APIs and models frequently change or get deprecated, how about creating an adapter interface like GoogleAiApi and simply swapping out the implementation when needed?

This approach could allow us to call methods from various implementations like PaLM2 or Gemini through bridge methods. While it’s true that Google’s API endpoints often change, it seems like the functionality they offer doesn’t differ significantly apart from performance or naming variations.

Similarly, Hugging Face supports multiple implementations through a unified interface, and it seems to work well without major issues. Maybe we could take a similar approach?

[KotlinFactory]

@KotlinFactory , @ziodave, @youngmoneee the truth of the matter is, Google Gemini API does NOT provide Java Client. So I guess you should raise an issue somewhere on their GitHub repose (good luck to find anything reliable there).

The other option is to try to build a low-level Spring AI Gemini client on top of the REST API, as we've done for many other providers including the , now deprecated, Google Vertex PaLM2 To commit time and resources for the later we need some re-assurance that this API won't disappear in next 6 months.

Honestly Google's Dev docs and API management is amongst the most confusing out there. Would appreciate if you can point me to reliable doc and code sources.

I totally agree with you, that google tends to provide horrible documentation for their APIs.
Quite frankly I don't see the problem with using their Rest-API. It seems to be really straight forward to implement.
Their SDKs are usually not great either, so it might actually be a benefit not having to use their SDKs.

[tzolov]

@KotlinFactory, I'm not sure the REST API reference doc event exists: https://ai.google.dev/api ?

[KotlinFactory]

Hey @tzolov it surely does. I build my own gemini client a few month ago I believe. It was based on this incomplete documentation from their side.

curl \ -X POST https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-pro-exp-0801:generateContent?key=${API_KEY} \ -H 'Content-Type: application/json' \ -d @<(echo '{ "contents": [ { "role": "user", "parts": [ { "text": "INSERT_INPUT_HERE" } ] } ], "generationConfig": { "temperature": 1, "topK": 64, "topP": 0.95, "maxOutputTokens": 8192, "responseMimeType": "text/plain" } }')

[tzolov]

@KotlinFactory so, can you point me to this "incomplete" Google Gemini REST API documentation?

My point is that, if a billions worth company, with zillions employees can't put their ** together to add a simple API doc, why should anyone take this API seriously?
Without a "stable" API with Docs I'm afraid that any efforts to reverse engineer and maintain Google APIs are likely to face API deprecated in few months - as it already happen with our efforts on Google PaLM2.

[tzolov]

@KotlinFactory,
Having said this, I would be very interested to spend some time in exploring this. Unfortunately my time is stretch with some hight priority tasks.
If you have time to contribute (and hopefully maintain) Spring AI Gemini (REST-based) API, it would be highly appreciated ;)

[KotlinFactory]

@tzolov Yeah I've heard about and seen the poor documentation of google APIs in various instances. Its really annoying.

Regarding the documentation Ive just used the "get code" button.

Yes, I would love l be contributing some gemini client to Spring AI. Just one question, how feature complete does it need to be considered. I've already some simple Java based Rest client for the GeminiAPI.

[KotlinFactory]

I will try ask them for an openapi spec. Perhaps they will respond.

[youngmoneee]

Regarding the documentation Ive just used the "get code" button.
Yes, I would love l be contributing some gemini client to Spring AI. Just one question, how feature complete does it need to be considered. I've already some simple Java based Rest client for the GeminiAPI.

I will join when the branch opens.

[KotlinFactory]

Okay, apparently there at least was a more complete documentation at some point.

https://discuss.ai.google.dev/t/gemini-rest-documentation-disappeared/36825/2

[tzolov]

Thanks for digging further @KotlinFactory

I found this Migrate from Gemini on Google AI to Vertex AI documentation:
https://cloud.google.com/vertex-ai/generative-ai/docs/migrate/migrate-google-ai

Apparently, unlike Vertex AI, no Enterprise support nor MLOps for Google AI Gemini API is expected. Not very encouraging to commit to this API.

If I understand correct the primary reason for someone to consider Gemini API over Vertex AI API is the API key authentication former support and later lacks?

Otherwise, you can use any of the available models (including 1.5 pro & flash) with the Spring AI Vertex Gemini Chat model.
Any other differences?

[KotlinFactory]

Yeah thats the primary concern. I dont beliebe users should be forced to use GCP just to use gemini.

There is also some response from their side. It appears like we can obtain an openapi spec given the instructions.

See google-gemini/cookbook#261

[KotlinFactory]

@tzolov Perhaps it would be an idea to reuse some parts of the vertex.ai module? I think except for authentication its basically identical.

[markpollack]

Given the enterprise customer base we focus on, I don't see us having bandwidth to devote to this in the short term. Is'nt it really more of an issue using an API key than 'using GCP' @KotlinFactory (neat handle name btw)

[Columpio]

They have REST API defined here

[vanduc2514]

@KotlinFactory theoratically the vertex.ai module can be re-used and modify to alternate the REST call to the Gemini endpoint instead of going through GCP. However, the issue is that Google haven't release a SDK for Java yet (There is only Android SDK and it's only used for prototyping purpose ) and the vertex.ai library from Google that Spring AI is using does not support Gemini without GCP REST call. To make the change, Spring AI needs to re-implement this using classes in that module and I think it's not beneficial, this should be handled by Google instead.

Related issue #626

[romantal]

Here you can find Google Generative Language API OpenAPI Specification (Revision 20241016):
google-gemini-api-openapi-specification-revision-20241016

Some work had to be done to be able to generate open api specification from google discovery specification. You can find here how to do this:
google-generativelanguage-discovery-json-patcher-app

I tested this endpoint: "/v1beta/models/{model}:generateContent" it works fine.

[markpollack]

Revising this topic again. Mariusz has made an implementation and has previously contributed to Spring AI.

See https://github.com/didalgolab/spring-ai-gemini

@didalgolab is this something that you would like to contribute to Spring AI?

[coderphonui]

Love to have this.

[didalgolab]

@markpollack Thanks for the mention. It is a bit basic implementation, suiting my needs but not updated for a while. However, if there's actual interest in this, I can consider refining it and turning it into a contribution.

[tboeghk]

Just a heads up: You can connect Spring AI to model brokers like openrouter.ai and utilize their OpenAI compatible API.

That way I got Google Gemini to work using the Spring AI OpenAPI starter. There's just a (very) small configuration fix needed: #1522 (comment)

[didalgolab]

Google has also an experimental OpenAI-compatible REST API: https://ai.google.dev/gemini-api/docs/openai.