Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide a way to flush compression stream on demand #3599

Closed
mtharrison opened this issue Oct 10, 2017 · 5 comments
Closed

Provide a way to flush compression stream on demand #3599

mtharrison opened this issue Oct 10, 2017 · 5 comments
Assignees
Labels
feature New functionality or improvement
Milestone

Comments

@mtharrison
Copy link
Contributor

It's useful to be able to tell the payload compression stream to flush its content, for example when wanting the browser to render the HTML document <head> and download the associated resources while you're still generating or writing the document body.

Express's compression middleware achieves this by offering a res.flush() method. I think something similar would be useful in hapi for better control of streaming compressed content.

Streaming compressed payload with hapi

hapi

Streaming compressed payload with express+compression

express

@kanongil
Copy link
Contributor

Interesting. I was not aware that this is possible using compressed streams.

@kanongil
Copy link
Contributor

@mtharrison How would an api user access the flush() method in Hapi? The stream is not available until after the response has started transmission.

@hueniverse
Copy link
Contributor

The only way I think we can support something like this is with a new response api that lets you define blocks:

const handler = (request, h) => {
   return h.response().block('This\n').block('is\n').block('being\n').block('streamed!');
};

And then if compression is involved, we flush it after writing each block.

However, since we do not write to the response stream until the handler is done, the only way this works well is by crafting a response stream and returning that, and doing some of the handler work inside the stream data read events. In which case, this API is useless and we need to expose the flush action as something we hand over the readable stream at pipe time.

Hmm.

@cvbuelow
Copy link

I have been playing around with this and was able to get it to work by exposing the compressor on the response (last line in this snippet of transmit.js).

let compressor = null;
if (encoding &&
    length !== 0 &&
    response._isPayloadSupported()) {

    delete response.headers['content-length'];
    response._header('content-encoding', encoding);

    compressor = (encoding === 'gzip' ? Zlib.createGzip() : Zlib.createDeflate());
}
response.compressor = compressor;  // added this line

I'm sure you can think of a less hacky way to do this but it then allows me to do

const handler = (request, reply) => {
  const stream = new Stream.PassThrough();
  reply(null, stream).type('text/html').code(200);
  stream.write('<html><head><script src="https://unpkg.com/react@15.3.1/dist/react.min.js"></script>');

  setTimeout(() => {
    // `compressor` won't be available till next tick
    request.response.compressor.flush()
    setTimeout(() => {
      stream.write('</head><body>hello</body></html>');
      stream.end();
    }, 2000);
  });
}

@ryanwholey
Copy link

Got this working in hapi 17.8.5 with the new setCompressor changes added in c49b9c8

handler(request, h) {

    class ResponseStream extends Stream.PassThrough {
        setCompressor(compressor) {
            this._compressor = compressor;
        }
    }

    const stream = new ResponseStream()
    stream.write('<!DOCTYPE html><html><head></head><body>hello')

    setTimeout(() => {
        stream._compressor.flush()

        setTimeout(() => {
            stream.write(' world</body></html>')
            stream.end()
        }, 1000)
    })

   return h.response(stream)
   .type('text/html')
}

So awesome! Thanks @hueniverse!

@Marsup Marsup added feature New functionality or improvement and removed request labels Sep 20, 2019
@lock lock bot locked as resolved and limited conversation to collaborators Mar 21, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
feature New functionality or improvement
Projects
None yet
Development

No branches or pull requests

6 participants