Webserial must open a large size buffer to transmit data

[demonguy]

Let say I need to transfer 1MB data to my device. And the protocol between device and host is "sending 4096 bytes data, device return OK, and then send another 4096 data, repeat it until all data sent"

The persudo code is like:

await device.open({baudRate: 115200, flowControl: "hardware", bufferSize: 8192});
reader = await device.readable.getReader();
writer = await device.writable.getWriter();

while (true) {
    reader.read();
    .......prepare 4096 chunk.........
    writer.write(chunk);
}

Since i only send 4096 bytes everytime. I think 8192 bytes buffer is enough.
However i found only bufferSize: 1024*1024 works (e.g. the total size i need to transfer). If i set it to 8192 bytes, the device will send me an error when i sent third 4096 packet. (Seems the device think i'm done sending data.)

Did i miss anything on that? What if i need to transfer 1GB file?

[reillyeon]

The bufferSize parameter controls the size of the buffer the browser uses to queue writes to and hold data read from the device. It's the buffer used when calling the read(2) and write(2) system calls (or their platform equivalents) when the data is passed to and from the operating system. You don't need to make the buffer large enough to hold the entire file. If the chunk you pass to write() is larger than the available space in the buffer it will write what it can and try again when more space is available. In your case in particular I would never expect more than 4096 bytes in the buffer at a time, assuming the device really does wait until it has received an entire packet before acknowledging it.

What you might be running into is #123, where because at most bufferSize bytes will be passed to the operating system at once the chunks passed to write() may be broken up on different byte boundaries than the chunks passed to the operating system. Some USB devices emulating a serial port seem to make assumptions about how data is arranged in the USB packets, even though that makes no sense from a serial device perspective. The solution in that case is to set bufferSize larger than a packet and close the WritableStream between every packet in order to force the browser and operating system to send all the data to the device, like this:

await device.open({baudRate: 115200, flowControl: "hardware", bufferSize: 8192});
reader = await device.readable.getReader();

while (true) {
    let { value, done } = await reader.read();
    .......prepare 4096 chunk.........
    writer = await device.writable.getWriter();
    writer.write(chunk);
    await writer.close();
}

[demonguy]

This works.
However i think there is potential performance issue. Just like WebUsb "WICG/webusb#209"
Though it's unlikely to solve on the serial layer.
Maybe later, WebSerial specification could add an option, telling the port if it should ensure every write not to be borken into pieces

[demonguy]

Well. Thanks for you reply, i think we're good on windows.
Now the last problem we could not solve is "#151".

[reillyeon]

There is a performance penalty but it is unavoidable in order to work around this particularly picky device behavior. The firmware should be fixed to not make invalid assumptions about how the driver constructs USB packets from the data provided by the application.

[demonguy]

Yeah. You're right.
But i think there are a lot of usb-serial devices driver will do it. Because it's "easy to program".