Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: streaming issue fixed for build versions #1006

Merged

Conversation

thecodacus
Copy link
Collaborator

Fix Chat Streaming in Production Environment

Summary

Fixed real-time message streaming in production by correctly configuring Content-Type headers for Server-Sent Events (SSE).

Problem Description

The chat interface was experiencing inconsistent behavior between development and production environments:

  • Development (wrangler dev): Messages streamed correctly in real-time
  • Production: Messages only appeared after the entire stream completed
  • User experience was degraded due to lack of real-time updates
  • Console logs showed chunks arriving all at once instead of streaming

Root Cause Analysis

Server response headers were incorrectly configured:

// Previous configuration
return new Response(stream.readable, {
  status: 200,
  headers: {
    contentType: 'text/plain; charset=utf-8'  // Incorrect content type
  },
});

This caused:

  • Browsers and proxies to buffer the response
  • Stream chunks to be accumulated before delivery
  • Loss of real-time streaming capability

Solution

Updated response headers to properly indicate Server-Sent Events:

return new Response(stream.readable, {
  status: 200,
  headers: {
    'Content-Type': 'text/event-stream',    // Correct SSE content type
    'Connection': 'keep-alive'              // Maintain connection
  },
});

Technical Details

  • Server-Sent Events (SSE) protocol is used for real-time streaming
  • text/event-stream Content-Type signals that the response should be processed as a stream
  • Browser automatically handles chunking and parsing of SSE data
  • No changes required to client-side stream handling code

Testing Performed

  • Verified streaming in development environment (wrangler dev)
  • Verified streaming in production environment
  • Confirmed message chunks appear in real-time
  • Tested with varying message lengths
  • Verified console logs show proper chunk timing
  • Checked browser network tab for correct headers

Impact

  • Real-time message updates now work consistently across all environments
  • Improved user experience with immediate feedback
  • No buffering or delay in message display
  • Reduced memory usage by processing chunks as they arrive

Rollback Plan

If issues are discovered, we can revert to the previous content-type by changing the header back to text/plain; charset=utf-8.

@thecodacus thecodacus added this to the v0.0.6 milestone Jan 5, 2025
@thecodacus thecodacus merged commit 78eb3a5 into stackblitz-labs:main Jan 6, 2025
3 checks passed
@johnykes
Copy link

johnykes commented Jan 8, 2025

still not working

JJ-Dynamite pushed a commit to val-x/valenClient that referenced this pull request Jan 9, 2025
* fix: streaming issue fixed for build versions

* added keep-alive header
@thecodacus
Copy link
Collaborator Author

thecodacus commented Jan 10, 2025

still not working

what is your setup and issue ?
can you try the latest docker image

@johnykes
Copy link

still not working

what is your setup and issue ? can you try the latest docker image
Coolify. Tried using Docker, also using dev / prod commands directly

@thecodacus
Copy link
Collaborator Author

sorry I mean to say can you try this command
docker run ghcr.io/stackblitz-labs/bolt.diy:42030e1

@johnykes
Copy link

sorry I mean to say can you try this command
docker run ghcr.io/stackblitz-labs/bolt.diy:42030e1

thanks, but I have some custom edits:)

@johnykes
Copy link

but even with the 2 headers mentioned in #1060

responseHeaders.set('Cross-Origin-Embedder-Policy', 'require-corp');
responseHeaders.set('Cross-Origin-Opener-Policy', 'same-origin');

the problm persists for me.

It also uses https using CloudFlare tunnels.

timoa pushed a commit to timoa/bolt.diy that referenced this pull request Jan 21, 2025
* fix: streaming issue fixed for build versions

* added keep-alive header
JJ-Dynamite pushed a commit to val-x/valenClient that referenced this pull request Jan 29, 2025
* fix: streaming issue fixed for build versions

* added keep-alive header
@1394862250
Copy link

may you can try append one host ,just like 192.168.1.9 myblot.local,you need use .local that is importent
if you use powershell on window , you can try this conmend
Add-Content -Path "C:\Windows\System32\drivers\etc\hosts" -Value "192.168.1.9 myblot.local"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants