Support for Azure OpenAI API

[pabvald]

Added

• Support for Azure OpenAI API. Requires two environment variables to be st: AZURE_OPENAI_API_KEY and AZURE_OPENAI_ENDPOINT(i.e. https://<resource-name>.openai.azure.com).

[svilupp]

Thanks for the PR! I'll do a quick review -- Have you tested it? I think I see some syntax issues, so I doubt it would actually run.

[svilupp]

Have a look at the changes if it makes sense!

Once you make the necessary changes, please test it and request a new review :)

[svilupp]

Looks good to me -- in an ideal scenario, I'd prefer to resolve the OpenAI bug first and update this implementation, but are you in a rush to get going on your projects? I think we could merge it as is if you have immediate use cases.

[svilupp]

Before we merge, could you please confirm that you've tried both the chat & embedding functionality and that they work? (We no longer use Azure directly, so I cannot test it myself)

[codecov]

Codecov Report

Attention: Patch coverage is 40.00000% with 6 lines in your changes missing coverage. Please review.

Project coverage is 91.54%. Comparing base (76b528d) to head (a3784c8).

Files with missing lines	Patch %	Lines
src/llm_openai.jl	0.00%	6 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #212      +/-   ##
==========================================
- Coverage   91.66%   91.54%   -0.12%     
==========================================
  Files          46       46              
  Lines        4415     4425      +10     
==========================================
+ Hits         4047     4051       +4     
- Misses        368      374       +6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[pabvald]

@svilupp I have applied the changes. There's something, which I would like to know your opinion about:

The base url for an Azure LLM is as long as https://<resource-name>.openai.azure.com/openai/deployments/<model-name>. Right now, the user needs to set AZURE_OPENAI_HOST = "https://<resource-name>.openai.azure.com" and the /openai/deployments/<model-name> part is appended when creating the provider. I am wondering if maybe it would be more reasonable to set AZURE_OPENAI_HOST= "" and let PromptingTools.jl build the whole URL, since .openai.azure.com is as repetitive as `/openai/deployments/.

Also how about refactoring llm_openai.jl into a subfolder with several files? The file already has 1728 lines and a lot of logic...

[svilupp]

maybe it would be more reasonable to set AZURE_OPENAI_HOST= "" and let PromptingTools.jl build the whole URL, since .openai.azure.com is as repetitive as `/openai/deployments/.

Interesting thought. My immediate reaction would be: 1) how would you know the resource name? and 2) isn't it harder to debug/share with Python and other services if we do things under the hood?

Lastly, you say it's always that address pattern, but, actually, for corporates there can be a lot of customization in the paths if requested (esp. with proxying for measuring consumption, adding internal content filters, etc.).

All in all, I'd probably stick to separating HOST vs PATH and am not sure if investing in refactoring that would yield any large benefits? Isn't it better to spend time building something with the API?

Also how about refactoring llm_openai.jl into a subfolder with several files? The file already has 1728 lines and a lot of logic...
100% agree + the extra modules should go out etc. The whole package is a bit of dino...
I haven't done it yet, because of not having much time + waiting for new LLM usage patterns to emerge to force the refactor and to be also "value adding" (there will be a lot of releases in Q4).

[pabvald]

Before we merge, could you please confirm that you've tried both the chat & embedding functionality and that they work? (We no longer use Azure directly, so I cannot test it myself)

I have tested it and it works:

julia> msg = aigenerate(PT.AzureOpenAISchema(), PT.AITemplate(:AssistantAsk); ask="What is the capital of France?", model="gpt-4o-2")
[ Info: Tokens: 54 in 0.8 seconds
AIMessage("Paris.")

and

julia> text_to_embed = "The concept of artificial intelligence."
julia> msg = aiembed(PT.AzureOpenAISchema(), text_to_embed; model="text-embedding-ada-002")
[ Info: Tokens: 6 @ Cost: $0.0 in 0.4 seconds
PromptingTools.DataMessage(JSON3.Array{Float64, Vector{UInt8}, SubArray{UInt64, 1, Vector{UInt64}, Tuple{UnitRange{Int64}}, true}} of size (1536,))

[pabvald]

Lastly, you say it's always that address pattern, but, actually, for corporates there can be a lot of customization in the paths if requested (esp. with proxying for measuring consumption, adding internal content filters, etc.).

I guess... Then maybe it would be better make the user include openai/deployments in AZURE_OPENAI_HOST ?

All in all, I'd probably stick to separating HOST vs PATH

or how exactly should AZURE_OPENAI_HOST look like ?

---

am not sure if investing in refactoring that would yield any large benefits? Isn't it better to spend time building something with the API?

mmm I am talking about separating the different functions in several files, which is a 30-minute effort. For me having a file with so many lines made the implementation of the support for Azure OpenAI more difficult.

[pabvald]

Looks good to me -- in an ideal scenario, I'd prefer to resolve the OpenAI bug first and update this implementation, but are you in a rush to get going on your projects? I think we could merge it as is if you have immediate use cases.

I am actually in a rush to use the Azure OpenAI functionality.