[Improvement] Add support for o1-preview and o1-mini OpenAI models

[dougcunha]

OpenAI has offered API access to its o1-preview and o1-mini models for some time now. However, differences in API usage make the current implementation incompatible with these models, resulting in the error I've attached below.

Failed to generate commit message: System.Exception: AI service returns error code BadRequest
at SourceGit.Models.OpenAIService.Chat(String prompt, String question, CancellationToken cancellation) + 0x3f9
at SourceGit.Commands.GenerateCommitMessage.GenerateChangeSummary(Change change) + 0x90
at SourceGit.Commands.GenerateCommitMessage.Result() + 0x19a

[love-linger]

Seems like #630

[love-linger]

Can you test the OpenAI api from curl?

curl https://api.openai.com/v1/chat/completions \
    -H "Content-Type: application/json" \
    -H "Authorization: Bearer YOUR_API_KEY" \
    -d '{
        "model": "MODEL",
        "messages": [
            {
                "role": "system",
                "content": "xxxx"
            },
            {
                "role": "user",
                "content": "xxxxx"
            }
        ]
    }'

[dougcunha]

I apologize for the confusion. I received an email from OpenAI stating that I have access to the o1-mini, but a cURL request returned that it is not available for me yet.

You can delete this. Thank you.

{
    "error": {
        "message": "The model `MODEL` does not exist or you do not have access to it.",
        "type": "invalid_request_error",
        "param": null,
        "code": "model_not_found"
    }
}

[love-linger]

did you forget to replace the MODEL with the target model you want to test?

[love-linger]

PS. I've pushed a commit that outputs the response body if the AI service returns an error code.

[dougcunha]

My apologies. I did not use the correct model. A different error has now occurred.

{
  "error": {
    "message": "Unsupported value: 'messages[0].role' does not support 'system' with this model.",
    "type": "invalid_request_error",
    "param": "messages[0].role",
    "code": "unsupported_value"
  }
}

[love-linger]

Since the official OpenAI library is available for .NET 9, I'll try to refactor the AI integration.

[dougcunha]

I found this:

This error occurs because the o1 series models, including the o1-mini, do not support messages with the "system" role in the chat completions endpoint. Currently, these models only accept messages with "user" and "assistant" roles.

Differences from GPT-4 usage:

GPT-4 supports messages with the "system" role to define model behavior; o1 models do not currently offer this support.

Adapting your code:

To use the o1-mini model, remove messages with the "system" role from your request. If you need to provide specific instructions to the model, include them directly in the user message content.

{
  "model": "o1-mini",
  "messages": [
    {
      "role": "user",
      "content": "Você é um assistente que fornece respostas concisas. Minha pergunta é: [sua pergunta aqui]"
    }
  ]
}

Therefore, you integrate the desired instructions directly into the user's message, circumventing the current limitation of o1 models regarding the "system" role.

I agree with you, I think that using the official OpenAI library is the best approach.

[love-linger]

I've pushed a branch feat-openai. Can you test it for me?

• Using official OpenAI library instead of custom one.
• The server URL must be (or equivalent) to https://api.openai.com/v1 not https://api.openai.com/v1/chat/completions
• If your self-hosted AI service, such as Ollama do NOT requires an API Key, you can set it to any text but can not leaves empty.

PS. I might not reply immediately because it's already 10pm in China...

[dougcunha]

It's working just fine, but I notice a difference compared to the previous implementation.
Look for yourself.

[love-linger]

Unfortunately, I found that the OpenAI official library was not ready to publish with native aot mode...

I'll modify the previous implementation for this issue tomorrow

[dougcunha]

After playing a little bit with the current implementation using the OpenAI lib I got this error once.

Failed to generate commit message: System.Net.Http.HttpIOException: The response ended prematurely. (ResponseEnded)
at System.Net.Http.HttpConnection.FillAsync(Boolean async)
at System.Net.Http.HttpConnection.ChunkedEncodingReadStream.Read(Span1 buffer) at System.ClientModel.Internal.ReadTimeoutStream.Read(Byte[] buffer, Int32 offset, Int32 count) at System.IO.Stream.Read(Span1 buffer)
at System.Net.ServerSentEvents.SseParser1.FillLineBuffer() at System.Net.ServerSentEvents.SseParser1.Enumerate()+MoveNext()
at OpenAI.Chat.InternalStreamingChatCompletionUpdateCollection.StreamingChatUpdateEnumerator.MoveNext()
at OpenAI.Chat.InternalStreamingChatCompletionUpdateCollection.GetValuesFromPage(ClientResult page)+MoveNext()
at System.ClientModel.CollectionResult1.GetEnumerator()+MoveNext() at SourceGit.Models.OpenAIService.Chat(String prompt, String question, CancellationToken cancellation, Action1 onUpdate) in D:\projetos\sourcegitOfficial\src\Models\OpenAI.cs:line 112
at SourceGit.Commands.GenerateCommitMessage.GenerateChangeSummary(Change change, StringBuilder summary, StringBuilder body) in D:\projetos\sourcegitOfficial\src\Commands\GenerateCommitMessage.cs:line 76
at SourceGit.Commands.GenerateCommitMessage.Exec() in D:\projetos\sourcegitOfficial\src\Commands\GenerateCommitMessage.cs:line 50

[love-linger]

Done. You can download the latest CI build.

[love-linger]

I will follow up on the issue - openai/openai-dotnet#243

Once it is resolved, I will use OpenAI's official .NET library to replace the existing implementation