How to provide custom endpoint for Azure OpenAi service? (gateway URL)

[egarkavy]

In my company we use gateway url to access Azure open AI service to do chat completions. It looks like this: https://xxx-xxx-openai.azure-api.net.
From your sourcecode I see that endpoint is set directly in the completeionWithRetry method

Is there any way to work with your Azure OpenAI class but having custom endpoint ?

[simpdanny]

Same issue here. I personally hope the endpoint could be modified in the form of
${this.azureOpenApiBasePath}/openai/deployments/${azureOpenAIApiDeploymentName}.

But for this issue, you could somewhat override endpoint by providing basePath/baseOptions.

const model = new ChatOpenAI(
    { 
          openAIApiKey: xxx,
          modelName: <my_deployment>
    },
    new Configuration (
          basePath: "https://<my_endpoint_path>/openai/deployments/<my_deployment>",
          baseOptions: {
              'headers' : { 'api-key': xxx, 'Host': yyy },
              'params' : { 'api-version': zzz }
          }
    )
)

[nfcampos]

Yes, this would be a good change to make

[egarkavy]

Just in case the only one option I found right now is to override private client prop of the base class. I basically copy pasted the code which instantiates it in the internal implementation

class AzureOpenAi extends ChatOpenAI {
  constructor(
    fields: AzureOpenAiConfig &
      Partial<OpenAIChatInput> &
      Partial<AzureOpenAIInput> &
      BaseChatModelParams & {
        concurrency?: number;
        cache?: boolean;
        openAIApiKey?: string;
      },
    configuration?: ConfigurationParameters,
  ) {
    super(
      {
        ...fields,
      },
      configuration,
    );

    const endpoint = `https://${fields.baseUrl}/openai/deployments/${fields.azureOpenAIApiDeploymentName}`;
 
    // this is actually an internal code but with custom overrides
    const clientConfig = new Configuration({
      ...(this as any).clientConfig,
      basePath: endpoint,
      baseOptions: {
        timeout: this.timeout,
        ...(this as any).clientConfig.baseOptions,
      },
    });
    (this as any).client = new OpenAIApi(clientConfig);
  }
}

[govind-kumarr]

Yes, this would be a good change to make

can you assign this to me

[chelouche9]

@nfcampos if this problem is still not taken care of I can do it. Please let me know :)

[govind-kumarr]

@nfcampos if this problem is still not taken care of I can do it. Please let me know :)

yes go ahead

[chelouche9]

By the way, I think there are two ways to solve it.

1. Adding a new property to the class (azureOpenAIApiDomainName) that has a default value of "openai.azure.com"
2. Adding a custom domain property that overrides the url concatenation entirely.

@nfcampos what do you think?

[holwerda]

By the way, I think there are two ways to solve it.

1. Adding a new property to the class (azureOpenAIApiDomainName) that has a default value of "openai.azure.com"
2. Adding a custom domain property that overrides the url concatenation entirely.

@nfcampos what do you think?

That would be awesome to allow a custom domain IMO

[clemenspeters]

Thanks for raising this issue @egarkavy 👏 👍

I created a PR to suggest a solution for this. ✅
What's the fastest way to get this reviewed and merged? 🚀
Can I support this process in any way? 😃

CC: @nfcampos @chelouche9 @holwerda @govind-kumarr