feat: implement websockets

[KhafraDev]

Fixes #932
Refs: nodejs/node#19308

• Initial handshake
• Integrating with fetch (await fetch('wss://whatever'))
• Setting websocket state
• Implementing WebSocket events
• Implementing WebSocket getters
• Sending data to the server & buffering edit: since the socket is a duplex, buffering should be handled already for us
• pings and pongs edit: Expose ping/pong mechanism for user integration whatwg/websockets#10
• CloseEvent, MessageEvent, and ErrorEvent
• web-platform-tests
• diagnostics channel support (opening connection, closing, receiving a message)
• permessage-deflate support
• feat: implement websockets #1795 (comment)

---

To test it out:

import { WebSocket } from './lib/websocket/websocket.js'

const ws = new WebSocket('wss://echo.websocket.events/')

ws.addEventListener('message', (event) => {
  console.log(event.data) // "echo.websocket.events sponsored by Lob.com"

  ws.send('my message')
  
  setTimeout(() => ws.close(), 2000)
})

ws.addEventListener('close', ({ code }) => {
  console.log(code) // 1005
})

[codecov-commenter]

Codecov Report

Base: 90.24% // Head: 82.93% // Decreases project coverage by -7.31% ⚠️

Coverage data is based on head (73128fd) compared to base (f38fbdc).
Patch coverage: 15.86% of modified lines in pull request are covered.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1795      +/-   ##
==========================================
- Coverage   90.24%   82.93%   -7.32%     
==========================================
  Files          58       66       +8     
  Lines        5179     5755     +576     
==========================================
+ Hits         4674     4773      +99     
- Misses        505      982     +477     

Impacted Files	Coverage Δ
lib/websocket/receiver.js	5.82% <5.82%> (ø)
lib/websocket/frame.js	10.00% <10.00%> (ø)
lib/websocket/util.js	10.81% <10.81%> (ø)
lib/websocket/websocket.js	11.35% <11.35%> (ø)
lib/websocket/connection.js	17.89% <17.89%> (ø)
lib/websocket/events.js	22.03% <22.03%> (ø)
lib/fetch/index.js	82.12% <47.05%> (-1.39%)	⬇️
lib/fetch/webidl.js	92.92% <60.00%> (-0.92%)	⬇️
index.js	100.00% <100.00%> (ø)
lib/websocket/constants.js	100.00% <100.00%> (ø)
... and 11 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

[lpinca]

If you have concurrent writes this might change the order of messages:

websocket.send(blob);
websocket.send(string);

The string message is sent before the blob.

[KhafraDev]

I don't know of any good way of fixing this without needing a Promise for every send.. Preferably node would allow writing a blob to a writable/duplex stream.

But then there would be issues masking the body/creating a frame. I guess the only other solution is adding a queue, but that's better for another time.

[lpinca]

This is the reason why there is no Blob support in ws. This and because the user can already do

websocket.send(await blob.arrayBuffer());

I agree with this ¹

Non-goals

• Support Blob chunks. The old WebSocket API defaults to receiving messages as Blobs; however, creating and reading Blobs is more costly than creating and reading ArrayBuffers. In practice, even though it requires explicitly setting binaryType, 97% of messages are received as ArrayBuffers. On the send side, sending Blobs adds considerable complexity to the implementation because the contents are not available synchronously. Since less than 4% of sent messages are Blobs it is better to avoid this complexity where we can.

Footnotes

1. https://github.com/ricea/websocketstream-explainer/tree/832f88d7f8eaecbe51da318d802951a3f8aadb2f#non-goals ↩

[KhafraDev]

Yes, when I was writing out my comment is kinda just clicked - "oh... this is why ws doesn't support blobs", lol. I still think undici should implement it for the sake of complying with the spec.

When I get around to implementing websocketstream this shouldn't be a problem at least!

[lpinca]

Well, ws already uses a queue (only when permessage-deflate is enabled and negotiated since compression is asynchronous) so support for Blob could be added, but I think it does not worth the effort. I think that Blob is of little use in Node.js if not natively supported by streams.

[jimmywarting]

hmm, but send(await blob.arrayBuffer()) would require reading the hole blob into memory before sending...
sure you could do something like:

for await (const chunk of blob.stream()) {
  websocket.send(chunk)
}

But then you would get multiple messages and you would have to assemble them yourself on the receiving side as well 😞

if you can accept blob response types then it could as well be piped to a tmp location on the disc and wouldn't hold up any memory when receiving blobs

are there any limit on how large a message can be?

[jimmywarting]

Ah, just went into the github repo and learned about a new WebSocketStream forgot that it existed, only head about it when it was in early proposal. Well, then you could instead maybe do: blob.stream().pipeTo(websocket)?

Should we implement a globalThis.WebSocketStream also?

[lpinca]

But then you would get multiple messages and you would have to assemble them yourself on the receiving side as well 😞

That's up to you. With ws you could send each chunk as a message fragment and receive the whole thing as a single message on the receiving end.

for await (const chunk of blob.stream()) {
  websocket.send(chunk, { fin: false}); // For the sake of simplicity backpressure is not handled.
}
websocket.send(Buffer.alloc(0), { fin: true });

if you can accept blobs then it could as well be piped to a tmp location on the disc and wouldn't hold up any memory when receiving blobs

AFAIK the whole message is buffered in memory before the 'message' event is emitted. That message is then returned to the user as a Blob.

[mcollina]

good work here, apart from the outstanding comments this is ready for me

[KhafraDev]

Any last minute issues to fix? I'd like to merge this by tonight and subsequently fix/add the remaining features over the next few days.

My next PRs (if this is merged, otherwise they'll be here):

• pings and pongs (very easy to add, I want to add tests though)
• tests (WPTs are "fine", but very lacking -- only 220 tests total)
• concurrent sends with a blob and string/typed array (we need to resolve the blob asynchronously, causing it to be sent later than other things)
• permessage-deflate support (likely not for a while)

Regarding the concurrent sends with a blob, we'd need core support for writing blobs to a writable stream. Then we could do:

const frame = new WebsocketFrameSend(Buffer.allocUnsafe(0))
const buffer = frame.createFrame(opcodes.TEXT) // or binary

socket.cork()
socket.write(buffer)
socket.write(blob)
socket.uncork()

socket.write(string)

[lpinca]

You have to mask the blob data so support for Blob in Writable.prototype.write() is not sufficient.

[KhafraDev]

Right, completely forgot about masking. There would need to be a way of resolving blobs synchronously, which I doubt has much hope of landing in core.

[lpinca]

There would need to be a way of resolving blobs synchronously, which I doubt has much hope of landing in core.

When permessage-deflate will be implemented, if compression will be done asynchronously (there is currently no support for sync deflate streams in Node.js core) you will have the same problem. You will then need a queue for compressed messages which you can also reuse for blobs.

FWIW you can also chose to never compress messages and still be RFC compliant. This is another area where the WHATWG specification falls short. It's all or nothing, there is no way to specify which messages to compress via websocket.send().

[KhafraDev]

Also found a way to read blobs synchronously, just took a deep dive into node internals. Obviously not going to implement it because it's a terrible hack lol.

const blob = new Blob(['hello world'])

const handle = Reflect.ownKeys(blob)[0] // kHandle

console.log(blob[handle].toArrayBuffer())