Rewrite to use Web IDL, and generally modernize

[domenic]

Closes #963.

Normative changes to widely-implemented features, roughly in order of most disruptive to least-disruptive:

• For the queuing strategy classes, their size and highWaterMark properties are now getters on the prototype, instead of data properties on the prototype and instance (respectively). Closes Specifying QueuingStrategies in WebIDL #1005. In particular this means that attempts to set either of them post-creation will throw a TypeError. Chromium already ships these semantics.

• Functions which take a dictionary no longer accept non-objects.

• For the queuing strategy classes, their highWaterMark property will no longer return a non-number from their highWaterMark properties, if one was passed to the constructor. Instead, NaN will be returned.

• All methods and accessors are now enumerable, per Web IDL defaults, instead of non-enumerable, per ECMAScript defaults.

• All classes are now exposed globally. Formerly, ReadableStreamDefaultReader, ReadableStreamBYOBReader, ReadableStreamDefaultController, ReadableByteStreamController, WritableStreamDefaultWriter, WritableStreamDefaultController, and TransformStreamDefaultController were not exposed. Closes Should ReadableStreamDefaultReader be exposed? #586.

• All classes now have [Symbol.toStringTag] properties. Closes Question: Should Streams have the prototype for Symbol.toStringTag and toString declared? #952.

• Some functions have changed their length property value.

• Some exceptions are thrown earlier, at argument-conversion time.

• Property lookup in options arguments now happens earlier, at argument-conversion time, and in alphabetical order, per dictionary rules.

Normative changes to unimplemented features:

• ReadableStream's getIterator() method has been renamed to values() as part of adopting Web IDL's infrastructure for async iterators.

• The byobRequest property on ReadableByteStreamController now returns null when there is no BYOB request, instead of returning undefined.

• The view property on ReadableStreamBYOBRequest now returns null when the view cannot be written into, instead of returning undefined.

• Various byte-stream-related APIs that used to specifically prohibit detached buffers now check for zero-length views or buffers, which is a more general category.

• The async iterator's next() and return() methods now behave more like async generators, e.g. returning promises fulfilled with { value: undefined, done: true } after return()ing the iterator, instead of returning a rejected promise.

Editorial changes:

• All APIs are specified to using Web IDL now, instead of using a modified version of the ECMAScript specification conventions. We continue using abstract operations and completion records for now, and we have to drop down to the ECMAScript level in a couple places (notably for dealing with %ObjectPrototype% vs. null-prototype iteration result objects, and transferring array buffers). But overall this removes a lot of type-checking and conversion boilerplate from the specification. Closes Switching to Web IDL #963. Closes Tracking issue for Web IDL async iterator support #1017. See Editorial: use dictionaries for { value, done } and eliminate "forAuthorCode"? #1036 for further followup on the iteration result objects.

• Individual abstract operations, constructors, methods, and properties no longer have their own heading. They are instead lumped together in sections. Closes Make the table of contents more useful/less dense #885.

• The constructors, methods, and properties are now documented in a per-class block, using the usual WHATWG "domintro" style. Closes Consider better styling for API intro notes #907.

• Abstract operations are now consistently alphabetized within their section. Closes Are abstract operations supposed to be in alphabetical order? #684.

• By using Bikeshed's <div algorithm> feature, we now get automatic identifier highlighting. Closes Add identifier highlighting similar to the ES spec #687.

• Switched to 100-character line limits, 1-space indents, and omitting end tags, per WHATWG conventions.

• Removed usage of emu-algify in favor of using some more of Bikeshed's built-in features, plus manually annotating a few things.

• Switched to concise Bikeshed linking syntax, e.g. [=term=] and [$AbstractOp$].

• Eliminated a number of utility abstract operations, especially around calling functions, by better using Web IDL.

Other bug fixes:

• Web IDL makes constructor behavior clear, so this closes Constructor specification is unclear #965.

---

Note to reviewers: this isn't really a fun thing to review. I started by trying to do this incrementally, e.g. removing emu-algify first, then reformatting to WHATWG conventions/modern Bikeshed syntax, then Web IDL-ifying, to make it more reviewable. But this way turned out to be a lot easier for me, and I don't think this would have gotten done if I'd taken the harder route.

The best review strategy I can think of is just to scroll around randomly in the output preview and see if anything catches your eye.

---

TODOs:

• Add back async iterators, after Add async value iterators and async iterator arguments webidl#808 arrives.
• Update the reference implementation. Probably use webidl2js. (In progress)
• See what tests break. Update them all. (In progress)
• Add idlharness tests.
• Add tests for the per-global-ness of the queuing strategy functions.
• (Optional) Consider removing the hand-curated IDs for non-exported abstract operations. They are painful to maintain and don't buy us much.
• (Optional) Consider renaming internal slot names to be less long-winded now that they are no longer used for brand checks
• (Optional) Consider renaming internal slot names to be uppercase to solve Use [[UpperCamelCase]] in internal slots #825.

---

Here's the pull request template, which we should update as we complete the above list of TODOs. I think in particular asking for implementer interest will be more feasible after we have the test changes up, although the "normative changes" list above is reasonable by itself.

• At least two implementers are interested (and none opposed):
  • Chrome
  • Firefox
• Tests are written and can be reviewed and commented upon at:
  • Streams: update tests for Web IDL conversion web-platform-tests/wpt#22982
• Implementation bugs are filed:
  • Chrome: https://bugs.chromium.org/p/chromium/issues/detail?id=1093862
  • Firefox: https://bugzilla.mozilla.org/show_bug.cgi?id=1645093
  • Safari: https://bugs.webkit.org/show_bug.cgi?id=213078

(See WHATWG Working Mode: Changes for more details.)

---

Preview | Diff

[ricea]

Wow. It might take me a little time to review this.

I assume we don't have to mention minor normative changes like the order of lookup of dictionary fields changing?

[domenic]

I'd love to have the list of normative changes as exhaustive as possible, at least to the extent of including general points like that. (It seems similar to my "Some exceptions are thrown earlier, at argument-conversion time".) I'll edit the OP to include that, and if you can think of any other such changes, just let me know.

[ricea]

Possible breaking change: the first argument of the constructors of ReadableStream, WritableStream and TransformStream is now compulsory.

Methods which take a dictionary will be stricter about what types they accept. For example readable.getReader(true) will no longer work.

An additional irrelevant change for queuing strategies: currently new CountQueuingStrategy({highWaterMark: 'sandwich'}).highWaterMark returns "sandwich". After this change, it will return NaN.

[domenic]

Possible breaking change: the first argument of the constructors of ReadableStream, WritableStream and TransformStream is now compulsory.

Note that this is relatively easy to reverse, and maybe we should do so to minimize unnecessary breaking changes. On the other hand, I have a hard time imagining code usefully using no-argument versions of these...

Oh wait, no-arg TransformStream is actually really important. OK, well, we should definitely allow that. I think I'll just allow it for all of them.

[ricea]

Reference implementation changes look good so far.

[domenic]

The reference implemenatation is mostly ported. I want to wait for the following webidl2js PRs to land and then reassess the test pass rate:

• Ensure promise-typed operations and attributes return promises jsdom/webidl2js#200
• Fix treatment of non-strings as enumerations jsdom/webidl2js#198
• Implement support for callback functions jsdom/webidl2js#194

[MattiasBuelens]

Went through the new spec text. Incredible work! I'm glad to see that almost all of the manual type checks have now been obsoleted by WebIDL. 👏

I'll go through the reference implementation another time. 😅

[ricea]

I've so far reviewed the preview up to the end of ReadableStream. It's very high quality. I haven't checked that all the links go to the right places. I think we might need to trust in Bikeshed for that.

[ricea]

Typo: manipualtion

[ricea]

The preview lgtm.

I'm going to review the reference implementation changes next. After that I will do a light review of the markup.