ReRun Document processing

[KSemenenko]

Context / Scenario

I process 500 page pdf, and somwere in the middle of process I reach TooManyRequset for Azure OpenAI.
and import is failed.

But I need a way to continue it. maybe halp of parts is already embedded, so I want to continue.

The problem

Now I can't process document, and I can't conintue from places/step I stopped.

Proposed solution

Option to conintue / rerun document processing.
so I can do like this:

var info = await memory.GetDocumentStatusAsync(fileInfo.DocumentId, "default");
if (info.Completed == info.Failed == false)
{
    var result = await memory.ImportDocumentAsync(new Document(info.DocumentId),  "default", info.RemainingSteps);
}

but i don't think it's good idea.

Importance

would be great to have

[johnnyreilly]

We've also struggled with this. When using KernelMemory.ImportDocumentAsync with a very large PDF, it will eventually error out with:

2024-02-01T20:06:55.767601865Z        ---> Azure.RequestFailedException: You are sending too many requests. Please try again later.
2024-02-01T20:06:55.767607622Z       Status: 503 (Service Unavailable)
2024-02-01T20:06:55.767611742Z       
2024-02-01T20:06:55.767616947Z       Content:
2024-02-01T20:06:55.767621400Z       {"error":{"code":"","message":"You are sending too many requests. Please try again later."}}

It would be tremendous if there was a way to gracefully handle this and also to achieve the goal of processing large documents. I'm not certain what the shape of that should be but I suspect it's a common need.

[KSemenenko]

@johnnyreilly I found how to use retry logic for thous cases

[johnnyreilly]

What did you do?

[KSemenenko]

First you have setup this
microsoft/semantic-kernel#4744 (comment)
You can use poly.

So, and then you can create http client and pass it as parameter into builder, where you adding embedding service and OpenAI service, last parameter is http client.