-
Notifications
You must be signed in to change notification settings - Fork 29.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Delayed setEncoding on Readable causes multibyte characters to break #27932
Labels
Comments
addaleax
added a commit
to addaleax/node
that referenced
this issue
May 28, 2019
Convert already-stored chunks when `.setEncoding()` is called so that subsequent `data` events will receive decoded strings, as they expect. Fixes: nodejs#27932
3 tasks
Hi, thanks for quick reaction. I've noticed that pull request has label for Node v10.x, but it is possible that this problem exists on other versions too - on one server we have Node v8.16.0 which has this issue too, so it might exists even on v12.x. |
targos
pushed a commit
that referenced
this issue
May 31, 2019
Convert already-stored chunks when `.setEncoding()` is called so that subsequent `data` events will receive decoded strings, as they expect. Fixes: #27932 PR-URL: #27936 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: Rich Trott <rtrott@gmail.com>
BethGriggs
pushed a commit
that referenced
this issue
Jul 9, 2019
Convert already-stored chunks when `.setEncoding()` is called so that subsequent `data` events will receive decoded strings, as they expect. Fixes: #27932 PR-URL: #27936 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: Rich Trott <rtrott@gmail.com>
BethGriggs
pushed a commit
that referenced
this issue
Jul 17, 2019
Convert already-stored chunks when `.setEncoding()` is called so that subsequent `data` events will receive decoded strings, as they expect. Fixes: #27932 PR-URL: #27936 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de> Reviewed-By: Rich Trott <rtrott@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
4.15.0-50-generic #54~16.04.1-Ubuntu x86_64 GNU/Linux
Server
Client
When setEncoding('utf8') is used on Readable stream (eg. IncomingMessage) with some delay, although no data are read, then already buffered chunks are read as buffer which breaks multibyte characters.
If encoding is set immediately, eg. on http 'request' event, than it works properly, however http frameworks (Express, Fastify, Koa) allows including asynchronous middlewares in processing pipeline before body parsing occurs, which delays setEncoding call and thus breaks multibyte characters.
Included code simulates this delay as two first emitted 'data' chunks are buffers which in standard body processing breaks characters if chunk contains only some bytes of unicode character. In case of http body parsing middlewares this also breaks content-length check.
Example body processing middleware from Fastify v1.14.6
The text was updated successfully, but these errors were encountered: