-
Notifications
You must be signed in to change notification settings - Fork 83
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Streaming chat in DevUI #442
Comments
@jmartisk do you think this is possible without too much work? |
What would that mean exactly, that the response from the model is a Multi and we would forward the events from the Multi to the UI as they arrive, over a websocket? |
Yes |
Yeah that shouldn't be very hard. I can look into it, just not sure when, things are piling up now :) |
Yeah, not a priority for sure, it would just be nice for demos |
I can give it a shot. |
I have something is working. However, it will only work when rag is disabled (and it's true by default). Should we add a message / alert to let the user know when streaming chat is not enabled and why it's not enabled? |
I think that's a good idea. I am pretty sure that noone is going to look at the browser console though :) |
Would it be possible to use a
StreamingChatModel
in the DevUI if it's accessible?This would allow to stream the response back
The text was updated successfully, but these errors were encountered: