How to register a service for Dall-E 3 image generation?

[bredlej]

Hello,

I can't quite find in the documentation nor GitHub samples how to generate images using this extension.

What I'm trying to do is to write a service like this:

@RegisterAiService
public interface DallEService {
    @SystemMessage("""
            You're a food illustrator. I'll provide you with a recipe name, description and ingredients. Please draw simple picture of the dish. Don't write any text in the picture. Generate the picture and return a url to it.
            """)
    @UserMessage("Name: {{recipeName}} Description: {{recipeDescription}} Ingredients: {{ingredients}}")
    String generateIllustration(String recipeName, String recipeDescription, String ingredients);
}

Now when I use this service I can see in the logs that it does a POST request to /v1/chat/completions instead of /v1/images/generations:

2024-06-11 13:46:27,828 INFO  [io.qua.lan.ope.OpenAiRestApi$OpenAiClientLogger] (vert.x-eventloop-thread-2) Request:
- method: POST
- url: https://api.openai.com/v1/chat/completions
- headers: [Accept: application/json], [Authorization: Be...ii], [Content-Type: application/json], [User-Agent: langchain4j-openai], [content-length: 595]
- body: {
  "model" : "gpt-4o",
  "messages" : [ {
    "role" : "system",
    "content" : "You're a food illustrator. I'll provide you with a recipe name, description and ingredients. Please draw in the style of New York Times comics. Don't write any text in the picture. Generate the picture and return an url to it.\n"
  }, {
    "role" : "user",
    "content" : "Name: Sushi Bowl Description: A simple and refreshing sushi-inspired bowl. Ingredients: ogórki, ryż, sos sojowy, ocet ryżowy, cukier, sól"

[...]

Also the response says I'm using a wrong API key but it's actually the same I'm using for my usual chat requests.

2024-06-11 13:46:28,043 INFO  [io.qua.lan.ope.OpenAiRestApi$OpenAiClientLogger] (vert.x-eventloop-thread-2) Response:
- status code: 401
- headers: [Date: Tue, 11 Jun 2024 11:46:28 GMT], [Content-Type: application/json; charset=utf-8], [Content-Length: 307], [Connection: keep-alive], [vary: Origin], [x-request-id: req_1dac9bef147ccae216ecb748d552f55a], [strict-transport-security: max-age=15724800; includeSubDomains], [CF-Cache-Status: DYNAMIC], [Set-Cookie: __...ne], [Set-Cookie: _c...ne], [Server: cloudflare], [CF-RAY: 89215c5c5d7a92ad-FRA], [alt-svc: h3=":443"; ma=86400]
- body: {
    "error": {
        "message": "Incorrect API key provided: =sk-proj*********************************************p4ii. You can find your API key at https://platform.openai.com/account/api-keys.",
        "type": "invalid_request_error",
        "param": null,
        "code": "invalid_api_key"
    }
}

I saw that there's a config value quarkus.langchain4j.openai.image-model.model-name set by default to dall-e-3.

But how do I actually tell a service to use this, and generate images instead of text?

Thanks!

[geoand]

Hi, currently AiService can not use dev.langchain4j.model.image.ImageModel. Hopefully that will change soon, but until then you can use ImageModel programmatically.

[bredlej]

All right, no worries, thanks for your quick response!

[geoand]

YW!