Expose HTTP/2 Streams via Standardized API

[benjchristensen]

HTTP/2 streams already provide the needed bidirectional byte streams we are seeking to achieve. It seems superfluous to layer yet another protocol on top of them with extra framing when we could use HTTP/2 directly if we just had standardized APIs.

In fact, targeting HTTP/2 streams directly is exactly what I'm doing with Proxygen (where we can control both client and server on mobile devices) via APIs we expose. We then layer our application protocol (ReactiveSocket https://github.com/ReactiveSocket/reactivesocket) over HTTP/2.

Is there a reason to not pursue a standard API for this rather than another framing layer?

[wenbozhu]

Re: fetch (the only standard API for web), the same API has to work for HTTP/1.1 too; and HTTP is a byte-stream protocol (v.s. a message-frame protocol).

Do you have a spec on how you plan to map reactivesocket over http/2?

If http/2 is only for multiplexing, then you don't really need a low-level API. E.g. WiSH doesn't need access to any low-level framing either.

If http/2 framing is used for message-level framing for reactivesocket, then the head-of-line blocking issue remains.

If reactivesocket has its framing, then WiSH is not useful either (even if it's part of the web API).

[benjchristensen]

ReactiveSocket frames are written to HTTP/2 DATA frames. We can have a single ReactiveSocket frame within a single DATA frame, or it can be split across many. That can be a configuration choice for max size of a DATA frame.

The reasons for using HTTP/2 for transport rather than TCP are primarily about:

• sharing the TCP connection with other HTTP traffic (images, video, GETs) and wanting to allow HTTP/2 flow control to manage the multiplexed streams
• preference for using HTTP/2 for routing from edge

[benjchristensen]

If http/2 framing is used for message-level framing for reactivesocket, then the head-of-line blocking issue remains.

What head-of-line blocking would occur?

HTTP/2 multiplexes the various H2 streams. ReactiveSocket is layered on a single H2 stream. The H2 DATA frames are kept small so they interleaved and are managed by H2. ReactiveSocket itself then manages it's own flow.

[benjchristensen]

If reactivesocket has its framing, then WiSH is not useful either (even if it's part of the web API).

Regarding framing, correct, but still useful in standardizing bidirectional binary streams. Framing is not the needed functionality, it is bidirectional binary streams. Whether it is framed or not is not relevant.

If the binary streams are framed (as WebSockets are) then we just treat it as "framed TCP" and incorporate the application protocol frames (ReactiveSocket) inside the transport frames. ReactiveSocket allows the frame length it its frame header to be optional (https://github.com/ReactiveSocket/reactivesocket/blob/master/Protocol.md#frame-header-format) so that it can more efficiently target transports like WebSockets or WiSH that already have framing.

[benjchristensen]

Regarding HTTP being a byte-stream protocol ... yes, but not standardized as bi-directional (full duplex), otherwise we could just use HTTP directly.

As discussed in whatwg/fetch#229, HTTP/2 support full-duplex streaming, but unfortunately this is not exposed in a standardized manner. If it was, then applications could build everything they want directly on top of HTTP/2.

[wenbozhu]

#5 (comment)

For a L7 protocol with its own framing, then WiSH is indeed redundant.

[wenbozhu]

#5 (comment)

If a single RS "message" can be sent over multiple H2 data frames, then HOL blocking is not an issue.

IMO, HTTP/2 to RS is just a multiplexed transport. WebSocket (WS) or WiSH framing is important to RS because of the Web API and its delivery semantics.

What I don't understand is why we need an HTTP/2 extension for RS if the latter has its own frames.

[wenbozhu]

#5 (comment)

So we are really just talking about exposing bidi semantics of HTTP, which is mostly a runtime and API issue.
#5 (comment)

HTTP is a byte-stream protocol, so bidi communication for HTTP doesn't involve messages. And HTTP/2 DATA frames (similar to HTTP/1.1 T-E chunks) are not really for framing messages.

Once Fetch/streams is available in browsers, as an optimization to WS, then I think RS over HTTP/2 is as straightforward as RS over TCP (byte-stream). Unlike in the WS case, I don't see any need to map RS (or any L7 protocol) frames to HTTP/2 DATA frames, even just for optimization purposes.

BTW, curl, Jetty, squid all support earlier responses long before there is HTTP/2 ;)

[benjchristensen]

What I don't understand is why we need an HTTP/2 extension for RS if the latter has its own frames.

We don't. It's being discussed primarily for two reasons:

1. Eliminate a layer of framing. (Add the RS behavior as H2 extension rather than embedded within H2 DATA frames on a single H2 stream)
2. Make each RS stream be an H2 stream for better multiplexed interop with other H2 streams.

[benjchristensen]

So we are really just talking about exposing bidi semantics of HTTP, which is mostly a runtime and API issue.

That to me is the bare minimum that ...

• should be the easiest to get agreement on and standardization of.
• would allow any application protocol to be achieved

That said, if we could agree to higher level semantics, I'm open to that. I expect though that the lowest common denominator of bidi streams is going to be the easiest. For example, I don't expect agreement on all the behaviors chosen for ReactiveSocket (message oriented flow control, resumability, etc), but if we could agree to the most important elements we need in a higher level protocol, I'd love to pursue that.

[benjchristensen]

BTW, curl, Jetty, squid all support earlier responses long before there is HTTP/2 ;)

Server-side has never been the issue, it's browsers and standard client-side HTTP libraries used in mobile clients. If HTTP clients exposed the necessary bidi APIs then we would be free to build anything we want.

[benjchristensen]

HTTP is a byte-stream protocol, so bidi communication for HTTP doesn't involve messages.

Agreed.

And HTTP/2 DATA frames (similar to HTTP/1.1 T-E chunks) are not really for framing messages.

Agreed.

Once Fetch/streams is available in browsers, as an optimization to WS, then I think RS over HTTP/2 is as straightforward as RS over TCP (byte-stream).

If the API becomes adopted, it may very well be. So far though I have yet to see what the APIs are. For example, I only see reference to streaming responses here: https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API and https://fetch.spec.whatwg.org/ hasn't been loading all day for me to confirm :-/ (Will need to look again later)

Unlike in the WS case, I don't see any need to map RS (or any L7 protocol) frames to HTTP/2 DATA frames, even just for optimization purposes.

I don't understand this comment. What are you suggesting they be mapped to when done in a browser?

[benjchristensen]

If we had bidi byte streams standardized in all browsers and HTTP libraries, what is the need for WebSockets, WiSH, etc? I don't see a significant value in the message framing, as that alone is not sufficient for applications to use them.

For example, ReactiveSocket started out originally targeting WebSockets.

[tyoshino]

If the API becomes adopted, it may very well be. So far though I have yet to see what the APIs are. For example, I only see reference to streaming responses here: https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API and https://fetch.spec.whatwg.org/ hasn't been loading all day for me to confirm :-/ (Will need to look again later)

There're some active work for enabling streamed uploading but they're scattered. E.g.:

• Uploading: how to handle redirects yutakahirano/fetch-with-streams#66
• Uploading: Provide a way to not set Content-Length yutakahirano/fetch-with-streams#65
• Developer-controlled streams for request/response whatwg/fetch#88

Streamed uploading has been on our road map but we were busy for the ServiceWorker to controlled page streaming feature recently.

[tyoshino]

Hmm, so, Ben, it looks for RS:

• Bidi byte stream over HTTP is good enough
• but for efficiency, as you said in Expose HTTP/2 Streams via Standardized API #5 (comment) Review of WiSH #2 (comment), you want to rely on HTTP2 framing level boundary information if possible while addressing HOL issue by guaranteeing message granularity at application level as you explained in Expose HTTP/2 Streams via Standardized API #5 (comment)?

ReactiveSocket frames are written to HTTP/2 DATA frames. We can have a single ReactiveSocket frame within a single DATA frame, or it can be split across many. That can be a configuration choice for max size of a DATA frame.

How were you planning to address this requirement? Just by the F flag of the RS protocol or by using some of the HTTP2 level field?

[wenbozhu]

#5 (comment)

Re: eliminate a layer of framing

I agreed that it is a gain to have a single framing, as long as large RS messages can be double-framed.

The issue with such an extension is that it breaks HTTP/2, i.e. hard to claim the second goal at the same time. Before all proxies are aware of such an extension, DATA frames can be arbitrarily reframed by proxies for I/O optimizing, buffering, prioritizing reasons. The fact that RS frames are to multiplexed with other HTTP traffic makes such an extension not really an extension to HTTP/2 but a forked protocol IMO.

With HTTP/3 coming (QUIC working group just formed), which will merge HTTP/2 data frames with a transport frames (conceptually?), the issue of double framing becomes a moot point.

For proxygen and closed deployment, a well-known header to advise to proxies that DATA frames are reused for message framing may just be sufficient. And the header can be published as a common practice via an informational spec.

[wenbozhu]

#5 (comment)

Bidi byte-stream: it's coming (independently of HTTP/2).

OpenJDK 9: http://mail.openjdk.java.net/pipermail/net-dev/2016-August/010187.html
whatwg Fetch/streams + transform functions
Node: node streams

Bidi frame-level stream:
Node: session-level streams being propsed

In reviewing the OpenJDK API, I asked about a low-level API too for exposing frames. Will keep you posted.

[wenbozhu]

#5 (comment)

Ignore the last comment. I should have been more specific.

[wenbozhu]

#5 (comment)

Indeed. Once bidi byte-streams are standardized everywhere, L7 bidi protocols can be easily built.

Re: WiSH/WS

For browsers, WiSH could be more efficient if supported natively than manually decoding/encoding a byte-stream using fetch/stream+ transform functions. The other goal is to migrate existing WS traffic to HTTP/2 without rewriting the JS app.

There are simpler use cases WiSH (similar to server-sent-events) could be directly useful. And a protocol like WebChannel could use WiSH as an abstraction to create a wire-agnostic solution too.

PS. we really need use something like slacks for design discussion ;)

[wenbozhu]

To summarize my opinions:

1. A low-level HTTP/2 frame-level API is unlikely to happen on most platforms
2. A bidi byte-stream API for HTTP (HTTP/2 or not) is happening, if not on iOS ?
3. For 2) I'd like to see reactive streams (Java Flow API) to be the standard
4. Reusing HTTP/2 data frames for L7 frames is better off being treated as an implementation detail for a closed environment. The actual gain is to be measured, but I think CPU efficiency will be greatly improved.
5. HTTP/* is not really a high-performance wire protocol. You either care about the Web for which the proposed extension is not really a good thing for Web; or optimize for a closed environment for which any native solution may do better.

[benjchristensen]

1. A low-level HTTP/2 frame-level API is unlikely to happen on most platforms

Then I'll stop pursuing that.

1. A bidi byte-stream API for HTTP (HTTP/2 or not) is happening, if not on iOS ?

Then this seems to be what we should throw our effort behind. How can we ensure it happens satisfactorily?

For 2) I'd like to see reactive streams (Java Flow API) to be the standard

Agreed. How can we influence this?

Reusing HTTP/2 data frames for L7 frames is better off being treated as an implementation detail for a closed environment. The actual gain is to be measured, but I think CPU efficiency will be greatly improved.

I'm fine treating this as an optimization within closed environments (which for us is iOS, Android, and server-to-server).

HTTP/* is not really a high-performance wire protocol. You either care about the Web for which the proposed extension is not really a good thing for Web; or optimize for a closed environment for which any native solution may do better.

This is a pretty strong reason for why we have defined an application protocol that can work over multiple transports so that we can degrade efficiency for browsers, but pursue efficiency gains everywhere else.

[benjchristensen]

For browsers, WiSH could be more efficient if supported natively than manually decoding/encoding a byte-stream using fetch/stream+ transform functions. The other goal is to migrate existing WS traffic to HTTP/2 without rewriting the JS app.

There are simpler use cases WiSH (similar to server-sent-events) could be directly useful. And a protocol like WebChannel could use WiSH as an abstraction to create a wire-agnostic solution too.

I would definitely like an updated server-sent-events solution. Ideally it would support binary encoding (so we don't have to base-64 encode), and would support application level flow control like the Reactive Streams / Java Flow API, as that is what made it difficult for me to use SSE in years past (I would fill buffers in intermediate proxies and fall 10s of seconds behind).

For WebSockets, why wouldn't the bidi byte streams be sufficient once that exists everywhere? Is it just the framing? If so, is WiSH/WebSockets2 really about providing bidi binary framed streams in a way that fixes the issues that have prevented adoption of WebSockets?

[wenbozhu]

#5 (comment)

1. iOS is the main platform which I have not visibility to. As a matter of fact, even basic server-client-streaming has issues (e.g. unexpected buffering). Do you have any contact you and/or I can have a discussion with? I can share a doc on the current streaming behavior too.
2. reactive streams. The JS spec is not finalized. Are you actively involved?
   https://github.com/reactive-streams/reactive-streams-js/

And then we should discuss with the Node/JS community. And I can help.

[wenbozhu]

#5 (comment)

Once fetch/streams are available everywhere, there is no need to use websockets. This is why we created this bidi-web site in the first place.

Yes, WiSH or WS over HTTP/2 is all about binary framing + API binding (message based delivery).

What I like to explore further is:

1. Does it make sense to have a transport-agnostic framing for something like RS or WebChannel?
2. If 1) is true, maybe we could standardize the RS framing or WiSH or whatever.

I am actually not too keen on mapping the WS framing as-is to WiSH. It's a good starting point.

[benjchristensen]

Do you have any contact you and/or I can have a discussion with?

I know someone at Apple who might be able to get to the right person. I'll ask.

reactive streams. The JS spec is not finalized. Are you actively involved?

I'm not as I don't do much JS. But I know Ben Lesh and Jafar Husain who are heavily involved in RxJS so could become involved if they aren't already.

[wenbozhu]

Thanks! (apple contact).

Re: reactive streams / JS. I'll get in touch with them about the JS spec..

I meant to ask if you are involved in developing the overall Rx spec.

Is reactive-sockets exposing reactive-streams as the API, or with extensions too (for advanced semantics etc)?

[benjchristensen]

I meant to ask if you are involved in developing the overall Rx spec.

There is no single Rx spec across languages and platforms, but they all try to maintain the same basic contract: onNext* (onError | onComplete)?

I led RxJava for ~3 years but a year ago handed that off to others (David Karnok in particular) who is leading RxJava v2 now. I remained involved in defining the design of it though: https://github.com/ReactiveX/RxJava/blob/2.x/DESIGN.md

I was also involved in the definition of Reactive Streams: https://github.com/reactive-streams/reactive-streams-jvm/blob/master/README.md#specification It stabilized when we hit 1.0 so I haven't spent much time on it since.

Is reactive-sockets exposing reactive-streams as the API, or with extensions too (for advanced semantics etc)?

The Java version exposes ReactiveStreams APIs. In C++ we are still iterating on what those would look like, but the goal is to define the equivalent Publisher/Subscriber/Subscription concepts. Though in C++ we'll probably have both eager (without the Publisher) and lazy (with the Publisher) APIs.

[benjchristensen]

1. Does it make sense to have a transport-agnostic framing for something like RS or WebChannel?

The only reason I can see needing this is HTTP. Otherwise all other transports of interest expose byte streams (TCP, Aeron, QUIC, WebSockets, etc).

And the only reason to use HTTP is because of browsers or internet infrastructure. So to me it's not so important to be transport agnostic, but to just allow things like RS and WebChannel over HTTP. HTTP itself will worry about TCP vs QUIC, etc I would think.

1. If 1) is true, maybe we could standardize the RS framing or WiSH or whatever.

What is the value of adding framing on top of byte streams if things like RS and WebChannel can easily add their own framing? Is it to take care of fragmentation and reassembly on behalf of application protocols to address HOL blocking so they don't each need to solve that?

My perspective is that if we want to pursue something higher than byte streams, we should capture more value than just the framing, but that starts to become more difficult to get agreement on (such as end-to-end ReactiveStreams flow control semantics).

[wenbozhu]

Wrt a common framing for streaming messages over byte-stream transport protocols (HTTP being one of them), it's exploratory and the main use case for now is to migrate WS traffic to HTTP/2.

HOL blocking is a concern of the underlying transport protocol. This is really just about defining a "simple" reusable framing format.

With a common framing format, it is possible to use an extensible format to carry "control messages", e.g. for resumability,, multi-homing support etc. Ironically I haven't found a good/efficient binary format for control messages. The only viable option seems to be CBOR.

[benjchristensen]

CBOR is the one @tmontgomery pointed me at as well and I'm pretty happy with it as a generic default.

Wrt a common framing for streaming messages over byte-stream transport protocols (HTTP being one of them), it's exploratory and the main use case for now is to migrate WS traffic to HTTP/2.

Thanks for clarifying.

This is really just about defining a "simple" reusable framing format.

Is this a goal just for HTTP, or a simple reusable framing format that is also usable on TCP? Seems that it should work for targeting any byte stream if it were to exist.

[wenbozhu]

Correct. WiSH is just a MIME type (that works over any byte-stream protocols).