You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The kernel-to-subprocess protocol uses netstrings to frame each message. In #1797 I discovered that the library we were using for this fails to handle partial chunks in the stream decoder, which breaks the protocol any time the delivery message is larger than the OS kernel's pipe buffer (typically about 8kb, but it varies by OS, which is why the test passed on Linux but failed on macOS).
I haven't been able to find any correct libraries that offer a Stream interface, so I have to write my own.
Description of the Design
Four functions:
encode: takes a Buffer, returns a Buffer
decode: takes a Buffer, returns a list of Buffers (for the completed netstrings) plus a leftover Buffer
createWriter: creates a Transform stream that takes Buffers (each of which is a single payload) and emits Buffers (to be concatenated out to a byte pipe)
createReader: creates a Transform stream that takes Buffers (which are chunks that do not necessarily align to netstring boundaries, but when all are received and concatenated, will form one or more netstrings) and emits Buffers (one payload each)
Security Considerations
The usual memory-consumption attack exists (the sender just writes 99999.. forever, and the recipient keeps buffering the supposed length prefix forever in hopes of seeing the termination colon). We control both sides of the pipe, however, so I'm not going to try to address it.
Test Plan
Unit tests.
The text was updated successfully, but these errors were encountered:
What is the Problem Being Solved?
The kernel-to-subprocess protocol uses netstrings to frame each message. In #1797 I discovered that the library we were using for this fails to handle partial chunks in the stream decoder, which breaks the protocol any time the delivery message is larger than the OS kernel's pipe buffer (typically about 8kb, but it varies by OS, which is why the test passed on Linux but failed on macOS).
I haven't been able to find any correct libraries that offer a Stream interface, so I have to write my own.
Description of the Design
Four functions:
encode
: takes a Buffer, returns a Bufferdecode
: takes a Buffer, returns a list of Buffers (for the completed netstrings) plus aleftover
BuffercreateWriter
: creates aTransform
stream that takes Buffers (each of which is a single payload) and emits Buffers (to be concatenated out to a byte pipe)createReader
: creates aTransform
stream that takes Buffers (which are chunks that do not necessarily align to netstring boundaries, but when all are received and concatenated, will form one or more netstrings) and emits Buffers (one payload each)Security Considerations
The usual memory-consumption attack exists (the sender just writes
99999..
forever, and the recipient keeps buffering the supposed length prefix forever in hopes of seeing the termination colon). We control both sides of the pipe, however, so I'm not going to try to address it.Test Plan
Unit tests.
The text was updated successfully, but these errors were encountered: