Concurrent Requests

[yguedidi]

Hello!
How it would be possible to do concurrent requests to OpenAI API?
In some cases it happen that we need to run multiple time the same prompt and it would be great to parallelize that.
Thanks

[yguedidi]

Any news?

[CViniciusSDias]

I was just about to start this discussion as well. My case is different. We have different prompts that need to be executed and running them concurrently would be of great help.

Currently we are considering not using this client and calling OpenAI API directly using concurrent requests with Guzzle, but that's gonna be a big hustle.

[yguedidi]

Currently we are considering not using this client and calling OpenAI API directly using concurrent requests with Guzzle, but that's gonna be a big hustle.

I'm planning the same thing in coming days, but with Symfony HttpClient 😅

[ondraasek]

any news?

[schelv]

I've managed to get concurrent requests to the OpenAI API working by using the async and await methods from the Amp package.
In addition the OpenAiClient needs to be configured to use an async HTTP client:

use Amp\Http\Client\HttpClientBuilder;
use Amp\Http\Client\Psr7\PsrAdapter;
use Amp\Http\Client\Psr7\PsrHttpClient;
use Laminas\Diactoros\RequestFactory;
use Laminas\Diactoros\ResponseFactory;
use OpenAI;

$asyncPsrHttpClient = new PsrHTTPClient(HttpClientBuilder::buildDefault(),
    new PsrAdapter(new RequestFactory(), new ResponseFactory()));

$asyncOpenAIClient = OpenAI::factory()
    ->withApiKey(config('apiKey'))
    ->withHttpHeader('OpenAI-Beta', 'assistants=v2')
    ->withHttpClient($asyncPsrHttpClient)
    ->make();

It does not work with streams however, but there is a withStreamHandler method on the factory that might be used to fix this.