draft-yoshino-wish.xml

<?xml version="1.0" encoding="UTF-8" ?>
<?rfc notedraftinprogress="yes" ?>
<?rfc rfcprocack="yes" ?>
<?rfc symrefs="yes" ?>
<?rfc toc="yes" ?>
<rfc
   category="std"
   docName="draft-yoshino-wish-03"
   ipr="trust200902">
  <front>
    <title>WiSH: A General Purpose Message Framing over Byte-Stream Oriented Wire Protocols (HTTP) </title>
    <author
       initials="T.Y."
       surname="Yoshino"
       fullname="Takeshi Yoshino">
      <organization>Google, Inc.</organization>
      <address>
        <email>tyoshino@google.com</email>
      </address>
    </author>
    <author
       initials="W.Z."
       surname="Zhu"
       fullname="Wenbo Zhu">
      <organization>Google, Inc.</organization>
      <address>
        <email>wenboz@google.com</email>
      </address>
    </author>
    <date month="May" year="2017"/>
    <abstract>
      <t>
        This document defines a general purpose message framing named WiSH which supports bi-directional (bidi) message-based communication over byte-stream oriented protocols such as HTTP (in its standard semantics).
        WiSH is designed to be compatible with WebSocket.
        WiSH can be viewed as a binary and bidi alternative to the framing defined for the server-sent events (SSE, EventSource) Web API <xref target="SSE" />.
      </t>
    </abstract>
  </front>

  <middle>
    <section title="Introduction">
      <t>
        The WebSocket protocol was proposed to provide native client-server bidi messaging for the Web.
        It has been implemented and deployed widely, but there are still missing semantics and features.
        See <xref target="BidiwebSurvey" />.
      </t>

      <t>
        WiSH is a general purpose message framing for use over the standard HTTP semantics to provide bidi messaging semantics.
        WiSH stands for Web in Strict HTTP.
      </t>

      <t>
        The communication protocol providing the standard HTTP semantics can be HTTP/1.1 <xref target="RFC7231" />, HTTP/2 <xref target="RFC7540" />, HTTP/2 + QUIC <xref target="QUIC" />, or any future protocols.
        Wire protocol features such as multiplexing, session priority, etc. are provided by the underlying protocol <xref target="TransportAbstraction" />.
        Unlike HTTP/2, HTTP/1.1 doesn't specify if earlier 2xx responses are allowed <xref target="RFC7540" />.
        Therefore, when HTTP/1.1 is used as the underlying protocol, full-duplex communication may be broken if the client, server or any intermediary chooses to buffer or reject earlier 2xx responses.
        Since intermediaries may buffer response bodies, bidi communication over WiSH may experience extra latency compared to WebSocket.
        When HTTPS is used, response body buffering by intermediaries is less likely to happen.
      </t>

      <t>
        The wire protocol features of WebSocket, such as handshake or control messages, are all dropped.
        WiSH respects the semantics of the underlying protocol (as opposed to turning it to a transport protocol).
        The concept of fragmentation is retained for enabling starting message transmission before determining the final length of the message.
      </t>

      <t>
        Application-level protocols may use WiSH as the framing protocol to support bidi communication over HTTP and for Web and Internet clients.
      </t>
    </section>

    <section title="Background">
      <t>
        There has been several attempts to improve bidi message-based communication on the Web.
      </t>

      <t>
        The server-sent events (SSE) <xref target="SSE" /> realized message-based communication in the server-to-client direction, by introducing a new Web API and a special message framing format while using HTTP as the wire protocol.
        Except for the issue of possible buffering by intermediaries, SSE works well with existing intermediaries and frameworks that support HTTP.
      </t>

      <t>
        WebSocket realized bidi message-based communication by introducing both a new Web API and a new wire protocol.
        Because the wire protocol is incompatible with HTTP, intermediaries and frameworks have to be upgraded to understand the wire protocol to support WebSocket.
      </t>

      <t>
        In parallel to the development of WebSocket, HTTP has been greatly improved with HTTP/2.
        There are more improvements upcoming to the HTTP e.g. QUIC.
      </t>

      <t>
        It's desirable that normal HTTP traffic and bidi message-based communication on the Web share further evolution to reduce cost of development and standardization.
        Bidi message-based communication on the Web should be multiplexed with normal HTTP traffic and should benefit from future transport-level improvements such as QUIC.
      </t>

      <t>
        WiSH is designed based on the above analysis.
        Use of the standard HTTP semantics as-is reduces cost and makes the Web simpler.
      </t>
    </section>

    <section title="Conformance Requirements and Terminology">
      <t>
        The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in <xref target="RFC2119" />.
      </t>

      <t>
        Requirements phrased in the imperative as part of algorithms (such as "strip any leading space characters" or "return false and abort these steps") are to be interpreted with the meaning of the key word ("MUST", "SHOULD", "MAY", etc.) used in introducing the algorithm.
      </t>

      <t>
        Conformance requirements phrased as algorithms or specific steps can be implemented in any manner, so long as the end result is equivalent.
        In particular, the algorithms defined in this specification are intended to be easy to understand and are not intended to be performant.
      </t>
    </section>

    <section title="WiSH Protocol">
      <t>
        WiSH frames messages over an HTTP request body or response body using the framing defined in <xref target="framing" />.
      </t>

      <t>
        The <spanx style="verb">Content-Type</spanx> header value of the underlying HTTP request or response message for which WiSH is used MUST be <spanx style="verb">application/web-stream</spanx>.
      </t>
    </section>

    <section title="Framing" anchor="framing">
      <figure>
        <artwork>
 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-------+-+-------------+-------------------------------+
|F|C|0|0|opcode |0|Payload      |Extended payload length        |
|I|M| | |4 bit  | |length       |16 bit if payload length is 126|
|N|P| | |       | |7 bit        |64 bit if payload length is 127|
+-+-+-+-+-------+-+-------------+ - - - - - - - - - - - - - - - +
|                                                               |
+ - - - - - - - - - - - - - - - +-------------------------------+
|                               |Payload Data                   |
+-------------------------------+ - - - - - - - - - - - - - - - +
|                                                               |
+---------------------------------------------------------------+
        </artwork>
      </figure>

      <t>
        WiSH is compatible with the framing of the WebSocket protocol <xref target="RFC6455" />.
      </t>

      <t>
        The opcode field indicates how to interpret the payload data field.
        WiSH uses the following opcodes.

        <list style="symbols">
          <t>
            %x0 denotes a continuation frame
          </t>
          <t>
            %x1 denotes the initial frame of a text message
          </t>
          <t>
            %x2 denotes the initial frame of a binary message
          </t>
          <t>
            %x3 denotes the initial frame of a text metadata message
          </t>
          <t>
            %x4 denotes the initial frame of a binary metadata message
          </t>
        </list>

        Any values not listed here are reserved.
      </t>

      <t>
        The FIN bit together with the continuation frame opcode, payload length and extended payload length work in the same way as the WebSocket protocol to represent messages.
        The fragmentation mechanism allows for flushing part of a large message payload without waiting for the total size of the message to be determined.
      </t>

      <t>
        The CMP bit indicates whether the message is compressed.
        The CMP bit of the first frame MUST be set to 1 when compression is enabled for the message.
        Otherwise, it MUST be set to 0.
        The CMP bit of non-first frames MUST be always set to 0.
      </t>

      <t>
        The message type distinction by the opcode field (text and binary) is kept to allow better Web support.
      </t>

      <t>
        The two metadata opcodes can be used for exchanging metadata e.g. using messages with opcode set to %x3 with metadata encoded in JSON in its payload field.
      </t>

      <t>
        The status code and status reason defined in the WebSocket protocol are dropped.
      </t>

      <t>
        The ping and pong control message of the WebSocket protocol are dropped.
        If such a feature is needed, it should be provided by underlying protocols.
      </t>

      <t>
        The permessage-deflate extension <xref target="RFC7692" /> is defined for the WebSocket protocol, to add a compression mechanism to it.
        The permessage-deflate extension can be applied to WiSH.
        The details are to be specified.
      </t>

      <t>
        What contents are exchanged and in what encoding they are exchanged over WiSH are to be defined by the application layer.
      </t>
    </section>

    <section title="Using WiSH over HTTP" anchor="http">
      <t>
        The standard HTTP (REST) semantics should be followed, especially the choice of the HTTP method.
        Some HTTP semantics may not be applicable, e.g. the <spanx style="verb">Cache-Control</spanx> header, when the body is streamed.
        However, such limitation is not specific to WiSH.
      </t>
    </section>

    <section title="WebSocket Compatibility Consideration">
      <section title="Valid UTF-8 Requirement">
        <t>
          In RFC6455, endpoints are required to _Fail the WebSocket Connection_ when they find that the byte stream in a text message is not a valid UTF-8 stream.
          To conform to the requirement, RFC6455 server frameworks check UTF-8 validness.
          The contents of text messages of WiSH also MUST be a valid UTF-8 stream.
          However, WiSH endpoints are not required to check UTF-8 validness.
          This provides more flexibility to server development.
          For example, a server may choose to check UTF-8 validness inside a JSON parser.
        </t>
      </section>
    </section>

    <section title="Acknowledgements">
      <t>
        Thank you to the following people for giving feedback to the document: Ben Christensen, Costin Manolache, Kari Hurtta, Loïc Hoguin, Roberto Peon, Van Catha.
      </t>
    </section>
  </middle>

  <back>
    <references title="Normative References">
      <?rfc include="reference.RFC.2119" ?>
      <?rfc include="reference.RFC.6455" ?>
      <?rfc include="reference.RFC.7231" ?>
      <?rfc include="reference.RFC.7540" ?>
      <?rfc include="reference.RFC.7692" ?>
    </references>

    <references title="Non-normative References">
      <reference
          anchor="SSE"
          target="https://html.spec.whatwg.org/multipage/comms.html#server-sent-events">
        <front>
          <title>HTML Living Standard - Server-sent events</title>
          <author>
            <organization>WHATWG</organization>
          </author>
          <date month="May" year="2017" />
        </front>
      </reference>

      <reference
          anchor="BidiwebSurvey"
          target="https://github.com/bidiweb/bidiweb-semantics/blob/master/SurveyOfProtocolGaps.md">
        <front>
          <title>Non Request-Response Communication over the Web, and What's Missing</title>
          <author initials="T." surname="Yoshino" fullname="Takeshi Yoshino"></author>
          <author initials="W." surname="Zhu" fullname="Wenbo Zhu"></author>
          <date month="January" year="2014" />
        </front>
      </reference>

      <reference
          anchor="TransportAbstraction"
          target="https://github.com/bidiweb/http-transport-abstraction">
        <front>
          <title>http-transport-abstraction</title>
          <author initials="W." surname="Zhu" fullname="Wenbo Zhu"></author>
          <date month="July" year="2016" />
        </front>
      </reference>

      <reference anchor="QUIC">
        <front>
          <title>QUIC: A UDP-Based Secure and Reliable Transport for HTTP/2</title>
          <author initials="R." surname="Hamilton" fullname="Ryan Hamilton"></author>
          <author initials="J." surname="Iyengar" fullname="Janardhan Iyengar"></author>
          <author initials="I." surname="Swett" fullname="Ian Swett"></author>
          <author initials="A." surname="Wilk" fullname="Alyssa Wilk"></author>
          <date month="July" year="2016" />
        </front>
      </reference>
    </references>
  </back>
</rfc>