Frequent timeouts

[mobilinked]

Description of the bug:

I am using following the code to access Gemini Chat:

let model = GenerativeModel(
       name: "gemini-1.5-pro-latest",
       apiKey: APIKey.default,
       requestOptions: RequestOptions(timeout: XXX, apiVersion: "v1beta")
)

let history = [
       ModelContent(role: "model", parts: "**prompt**")
]
        
let chat = model.startChat(history: history)
let response = try await chat.sendMessage("a short message")

When the prompt size approaches 15k:

• The code frequently encounters timeout errors.
• However, when utilizing the same prompt in AI Studio, responses are significantly faster.

I have to set the timeout a larger value, for example 120 to dismiss the errors. But still there are many timeout errors.

For smaller prompt sizes, timeouts do not occur.

Actual vs expected behavior:

I suspect there may be an issue with either my code or the system itself, as the API calls seem to be executing slower than expected.

Any other information you'd like to share?

No response

[andrewheard]

Hi @mobilinked, just to clarify, by 15k do you mean 15KB or 15 thousand characters (or another size metric)?

[mobilinked]

Hi @andrewheard ,

After reviewing my code and comparing it with AI Studio, I realize the error was on my end. It seems I need to adjust the timeout value based on the content length or consider using a streaming API to alleviate concerns about waiting times.

I will close this issue now. Thank you for your understanding. Below is the information for your reference.

---

I meant that the count of the characters is about 15,000.

I'm currently developing a command-line application to translate webpages from English into various languages. The application utilizes system instructions that include a basic prompt outlining roles and workflow. These instructions are combined with the content pulled from a webpage. The base size of these instructions is around 2,500 characters, while the total length varies from 2,500 to 30,000 characters depending on the webpage content.

I've increased the timeout parameter to 240 seconds, which has significantly reduced most timeout issues.

Each webpage's content is loaded from a file and merged with the basic prompt in the system instructions. The AI then executes defined steps using the sendMessage function. Please review the attached logs, as the response times are considerably slower than expected.

System Instruction Character Count: 12123
Step 1 ...
Execution duration: 74.61162400245667 seconds
Step 2 ...
Execution duration: 76.62761402130127 seconds
Step 3 ...
Execution duration: 73.82129907608032 seconds

System Instruction Character Count: 9147
Step 1 ...
Execution duration: 50.07697904109955 seconds
Step 2 ...
Execution duration: 48.9211859703064 seconds
Step 3 ...
Execution duration: 45.519572019577026 seconds

System Instruction Character Count: 2895
Step 1 ...
Execution duration: 4.812904953956604 seconds
Step 2 ...
Execution duration: 4.421628952026367 seconds
Step 3 ...
Execution duration: 3.6941369771957397 seconds