[feature request] don't stream response

[zegerius]

The current implementation of --no-stream makes sense to some degree, i.e. the OpenAI response is not streamed. However, it doesn't make sense to render the 'ticker' that way - we have the full response in memory anyway.

It would also make sense to be able to toggle this dynamically:

.set stream true
.set stream false

The current --no-stream can be maintained and just set the config value (config.stream) to false.

I took a look at render/mod.rs:render_stream and render/repl.rs:repl_render_stream but don't have enough Rust experience to idiomatically solve this. My hunch is that there are two types of receivers ReplyStreamEvent and ReplyEvent. The writer is then either streamed or instant.

[sigoden]

The --no-stream only works in command mode.

In chat mode, the response always streamed.

GPT replys much faster in stream. I don't think anyone wants to use no-stream response in chat mode.

The OpenAI response can be stream or non-stream, They are different.

[kwekewk]

Bumping this. Now, there quite a lot different openai api compatible providers and some of which don't offer/support stream response