Non-informative error.

[kverty]

Description

I clnstantly get com.aallam.openai.api.exception.GenericIOException: Failed to parse HTTP response: unexpected EOF exception and don't know what is wrong.

Steps to Reproduce

1. I create quite big ChatCompletionRequest (I don't get this error on short requests, when arrayOfValues.size == 170 for example)
2. Send it by Chat.chatCompletion(), the suspend one.
3. Get this error.

Environment

• openai-kotlin version: 3.4.1
• Kotlin version: 1.9.10
• OS: Windows 11

Additional Info

ChatCompletionRequest

{"model":"gpt-3.5-turbo-16k","messages":[{"role":"system","content":"1-th context:\nYour task is to return a new single column based on the user�s prompt and information from the existing columns.\nHere are existing columns in JSON format:\n{"name":"actor_3_name","type":"String","arrayOfValues":["Rupert Grint","Ramon Rodriguez","Common","Tom McCarthy","Gary Oldman","Jess Harnell","Stephen Colbert","Leo Howard","Steve Coogan","Dominic Monaghan","Armin Mueller-Stahl","Niecy Nash","Bruce Greenwood","Stephen McHattie","Anika Noni Rose","Stephen Graham","Michael Rispoli","Max Records","Bobb'e J. Thompson","Tom McCarthy","Al Roker","Kelly Keaton","Robert Maillet","Zoe Kazan","Boris Kodjoe","Andrea Di Stefano","Rie Miyazawa","Yvette Nicole Brown","Lou Taylor Pucci","Maude Apatow","Christoph Waltz","Jesse McCartney","Rupert Evans","Dee Bradley Baker","Tasha Smith","Judith Hill","John Hodgman","Xander Berkeley","Leleti Khumalo","Michael Weston","Denis O'Hare","James Corden","Rachael Harris","Kevin Brown","Richard Schiff","Michael Kelly","Taylor Lautner","Chandler Canterbury","Tom Everett Scott","Rain","Armin Mueller-Stahl","Alison Lohman","Josie Ho","Tom Sturridge","Tim Meadows","Slash","Andr? Dussollier","Craig T. Nelson","Sachiko Ishida","Vanessa Ferlito","Jane Curtin","Thomas Lennon","Shantel VanSanten","Helen McCrory","Patrick Fugit","Eddie Rouse","Michelle Nolden","Eric Winter","Mariah Carey","Aleksey Serebryakov","Mike Epps","Kim Dickens","Ann-Margret","Steven Mackintosh","Thomas Kretschmann","Shawn Doyle","Beverly D'Angelo","Jason Cope","Moises Arias","Chris Lowell","Candice Bergen","Reggie Lee","Tom Kane","Delphine Chan?ac","Randall Duk Kim","Verne Troyer","Kad Merad","Demetri Martin","Jeffrey Markle","Vanesa Tomasino","Allen Covert","Bailee Madison","Mariah Carey","Chris Elliott","Celia Weston","Robert Duvall","Shawn Hatosy","J?r?me Deschamps","Stephen Root","Derek Graf","Marie Gillain","Gaye Brown","Nicholas Tse","Thomas F. Wilson","Carmen Ejogo","Aryana Engineer","Skyler Samuels","Tiffany Lamb","Jake Short","Bernie Sanders","Lorna Raver","David Carradine","Nick Gomez","Brendan Sexton III","Ryan Donowho","Hope Olaide Wilson","Martin Starr","Dan Bakkedahl","Frances Conroy","Megan Mullally","Shawn Roberts","Fred Willard","Anne-Marie Duff","Aden Young","Leonie Benesch","Shawn Yue","Derek Luke","Armando Riesco","Rachel Dratch","Rachel Brosnahan","Cynthia Stevenson","Olly Alexander","Joan Allen","Kyung-gu Sol","Alia Shawkat","Margot Kidder","Martha MacIsaac","Beth Grant","Bobby Coleman","Conleth Hill","Yasuaki Kurata","Edi Gathegi","Marley Shelton","Edouard Baer","Diora Baird","Natalie Lisinska","Ethan Suplee","Annie Golden","Rumer Willis","Richard Schiff","Bruce McGill","Alicja Bachleda","Dahong Ni","Ryu Kohata","Emma Lung","Charlotte Sullivan","George Newbern","Hakeem Kae-Kazim","Charles Napier","Alba Rohrwacher","Robert Pralgo","Henry Simmons","Ted Danson","Aaron Yoo","Anthony Edwards","Glenn Plummer","Jason London","Wendie Malick","H?ctor Jim?nez","Shaun Evans","John Henshaw","?milie Dequenne","Jonathan Tucker","Debrianna Mansini","Samuel Roukin","Harry Connick Jr.","Clifton Collins Jr.","Diego Velazquez","Evan McGuire","Ellie Kendrick","Clifton Collins Jr.","Haley Bennett","Sean Harris","Zooey Deschanel","Peter Breitmayer","Bill Cobbs","Keri Lynn Pratt","Jennifer Butler","Carla Gallo","Derek Wayne Johnson","Michael McElhatton","Jessica Mauboy","Cara Seymour","Eamonn Walker","Maurice Ro?ves","Dominik Garc?a-Lorido","Raquel Alessi","Robert Wahlberg","John Bell","Andrea Ros","Wood Harris","Benedict Wong","Shenae Grimes-Beech","Katie Leclerc","Birol ?nel","Charlotte Rampling","Chandler Canterbury","Paresh Rawal","Anna Katharina Schwabroh","Kathleen Munroe","Victoria Thaine","Nicholas Gonzalez","Charlotte Sullivan","Stephen Henderson","Guillermo Francella","Jere Burns","Amy Irving","Mike Epps","Jason Maza","Noah Segan","Jim Norton","Talia Shire","Chris Kattan","Salvatore Santone","Frank Vincent","Zheng Xu","Margot Kidder","Ranin Karim","Romany Malco","Paul Adelstein","Silvia Colloca","Anne-Marie Duff","Dahong Ni","Christopher Atkins","Gauhar Khan","Tom McCarthy","Dee Wallace","Hallee Hirsh","Alicia Ziegler","Kevin Hamedani","Chrystee Pharris","Joe Cassidy","Ane Dahl Torp","Rachel Brosnahan","Michele Morrow","Drew Sidora","Elton John","D. David Morin","Lynn Redgrave","Sissy Petropoulou","Darren Kendrick","Pollyanna McIntosh","Jessica Duffy","Simon Baron-Cohen","Martha MacIsaac","Jordan Scovel","David Schaal","Michael Par?","Ebon Moss-Bachrach"]}\nReturn a new column in string type in the same JSON format as above with result values according to the user�s request all in one line, do not pretty-print it. Always return the same number of rows as in existing columns.\nNever give any verbal comments. ","name":null,"function_call":null},{"role":"user","content":"extract second name from actor_3_name","name":null,"function_call":null}],"temperature":0.7,"top_p":null,"n":null,"stop":null,"max_tokens":6000,"presence_penalty":null,"frequency_penalty":null,"logit_bias":null,"user":null,"functions":null,"function_call":null}

If this is unusable I am happy to provide ChatCompletionRequest in some other shape or form.

I simply call val completion = openAI.chatCompletion(chatCompletionRequest)
where openAI defined like this:

 private val openAI = run {
        val config = OpenAIConfig(
            token,
            organization = organization,
            logging = LoggingConfig(logger = Empty),
        )
        OpenAI(config)
    }

pretty straightforward

Full error text

com.aallam.openai.api.exception.GenericIOException: Failed to parse HTTP response: unexpected EOF
at com.aallam.openai.client.internal.http.HttpTransport.handleException(HttpTransport.kt:51)
at com.aallam.openai.client.internal.http.HttpTransport.perform(HttpTransport.kt:23)
at com.aallam.openai.client.internal.http.HttpTransport$perform$1.invokeSuspend(HttpTransport.kt)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
at kotlinx.coroutines.EventLoop.processUnconfinedEvent(EventLoop.common.kt:68)
at kotlinx.coroutines.internal.DispatchedContinuation.resumeWith(DispatchedContinuation.kt:347)
at io.ktor.util.pipeline.SuspendFunctionGun.resumeRootWith(SuspendFunctionGun.kt:138)
at io.ktor.util.pipeline.SuspendFunctionGun.access$resumeRootWith(SuspendFunctionGun.kt:11)
at io.ktor.util.pipeline.SuspendFunctionGun$continuation$1.resumeWith(SuspendFunctionGun.kt:55)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:569)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.ktor.server.netty.EventLoopGroupProxy$Companion.create$lambda$1$lambda$0(NettyApplicationEngine.kt:296)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.io.EOFException: Failed to parse HTTP response: unexpected EOF
at io.ktor.client.engine.cio.UtilsKt$readResponse$2.invokeSuspend(utils.kt:162)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
at kotlinx.coroutines.internal.LimitedDispatcher$Worker.run(LimitedDispatcher.kt:115)
at kotlinx.coroutines.scheduling.TaskImpl.run(Tasks.kt:103)
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:584)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:793)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:697)
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:684)

[aallam]

This appears to be an error from the underlying HTTP client. Could you please share which one you were using? As a workaround, have you tried using a different HTTP client?

[kverty]

I don't think this error happens in my code. You see, I put two logs this way:

logger.info {
    "GPT Completion request: ${json.encodeToString(ChatCompletionRequest.serializer(), chatCompletionRequest)}"
}
val completion = openAI.chatCompletion(chatCompletionRequest)
logger.info {
    "GPT Completion response: ${json.encodeToString(ChatCompletion.serializer(), completion)}"
}

Only the first one was printed. So error happens inside openAI.chatCompletion(chatCompletionRequest) call.

I see that httpClient is created here from config.

public fun OpenAI(config: OpenAIConfig): OpenAI {
    val httpClient = createHttpClient(config)
    val transport = HttpTransport(httpClient)
    return OpenAIApi(transport)
}

In my config I fill out only 3 fields: token, organization and logging. All the others are default. And it worked until now.
Upd. Maybe not exactly "now", I don't really know when it started to fail:)
Do you think I should specify proxy argument of config?

[aallam]

I was actually referring to Ktor's HTTP client engines (such as okhttp or apache), which one did you use in your gradle dependencies ?

[kverty]

In jvmMain we used CIO.
You use my gradle dependencies when creating httpClient by createHttpClient(config), not just config?

[aallam]

Yes, the engine you've chosen is used by the underlying Ktor client. The error is more likely related to the CIO engine. For JVM, I would recommend trying with OkHttp instead.

[kverty]

Fixed by doing

 private val openAI = run {
        val config = OpenAIConfig(
            token,
            organization = organization,
            logging = LoggingConfig(logger = Empty),
            timeout = Timeout(15.minutes, 20.minutes, 25.minutes)
        )
        OpenAI(config)
    }