New Buffer API backport to v4.x (staging branch) #7562

ChALkeR · 2016-07-06T12:32:10Z

Moved from #7475, retargeted to v4.x-staging branch.

Checklist

make -j4 test (UNIX), or vcbuild test nosign (Windows) passes
tests and/or benchmarks are included
documentation is changed or added
commit message follows commit guidelines

Affected core subsystem(s)

buffer

Description of change

This backports the new Buffer.alloc(), Buffer.allocUnsafe(), Buffer.from(), and Buffer.allocUnsafeSlow() APIs for v4.

Note: some backported tests are disabled, but those are not related to the new API.

Important: Buffer.from(arrayBuffer[, byteOffset [, length]]) (as in 6.x) was not backported, I used Buffer.from(arrayBuffer) instead, that helped to significantly reduce the code changes.

Note that --zero-fill-buffers is not included here (see #5745 for that), also the encoding parameter for .fill() is not included.

Existing code changes are minor here, this is not invasive.

Only two changes in existing code:

.fill('') now zero-fills,
One call to Buffer(string, encoding) is replaced with Buffer.from.

There is a typecheck in that code path, so the behaviour is identical. In fact it could be reverted back to Buffer(string, encoding) or even changed to fromString(string, encoding) — that would just affect the double checks.

All the other changes are additive, i.e. introduce new methods and don't change the existing ones.

ChALkeR · 2016-07-06T12:35:50Z

/cc @thealphanerd @nodejs/lts

Commit messages amended to include LGTMs and correct links, one comment removed that @trevnorris noted in #7475 (comment).

Should be good to land now.

rvagg · 2016-07-07T11:27:54Z

doc/api/buffer.md

+
+* [`Buffer.from(array)`][buffer_from_array] returns a new `Buffer` containing
+  a *copy* of the provided octets.
+* [`Buffer.from(arrayBuffer[, byteOffset [, length]])`][buffer_from_arraybuf]


Needs to be changed to remove arg 2 and 3?

Argh, missed that. Will fix soon, thanks!

rvagg · 2016-07-07T11:36:16Z

some backported tests are disabled, but those are not related to the new API.

I can't see any disabled tests, do you mean they are simply not included or am I not seeing them?

CI @ https://ci.nodejs.org/job/node-test-commit/4003/

Linter is failing on one of the tests, https://ci.nodejs.org/job/node-test-linter/3196/console

ChALkeR · 2016-07-07T11:38:20Z

@rvagg

I can't see any disabled tests, do you mean they are simply not included or am I not seeing them?

Grep for // fails on 4.x in the test/parallel/test-buffer-alloc.js file

Linter is failing on one of the tests

Hm. I guess I somehow missed that. Will fix, thanks!

rvagg · 2016-07-07T11:40:16Z

ahh, not whole tests, individual assertions, gotcha 👍

Treat negative length arguments to `Buffer()`/`allocUnsafe()` as if they were zero so the allocation does not affect the pool’s offset. Fixes: nodejs#7047 Refs: nodejs#7051 Refs: nodejs#7221 Refs: nodejs#7475 PR-URL: nodejs#7562 Reviewed-By: Trevor Norris <trev.norris@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Nikolai Vavilov <vvnicholas@gmail.com>

This backports the new `Buffer.alloc()`, `Buffer.allocUnsafe()`, `Buffer.from()`, and `Buffer.allocUnsafeSlow()` APIs for v4. Some backported tests are disabled, but those are not related to the new API. Note that `Buffer.from(arrayBuffer[, byteOffset [, length]])` is not supported in v4.x, only `Buffer.from(arrayBuffer)` is. Refs: nodejs#4682 Refs: nodejs#5833 Refs: nodejs#7475 PR-URL: nodejs#7562 Reviewed-By: Trevor Norris <trev.norris@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Nikolai Vavilov <vvnicholas@gmail.com>

ChALkeR · 2016-07-08T03:36:40Z

@rvagg Fixed! =)

ChALkeR · 2016-07-08T03:38:42Z

New CI: https://ci.nodejs.org/job/node-test-pull-request/3212/

rvagg · 2016-07-08T05:40:07Z

beautiful

lgtm

jasnell · 2016-07-08T05:52:14Z

LGTM

ChALkeR · 2016-07-08T12:06:57Z

@thealphanerd @nodejs/lts, should I land this?

Treat negative length arguments to `Buffer()`/`allocUnsafe()` as if they were zero so the allocation does not affect the pool’s offset. Fixes: #7047 Refs: #7051 Refs: #7221 Refs: #7475 PR-URL: #7562 Reviewed-By: Trevor Norris <trev.norris@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Nikolai Vavilov <vvnicholas@gmail.com>

This backports the new `Buffer.alloc()`, `Buffer.allocUnsafe()`, `Buffer.from()`, and `Buffer.allocUnsafeSlow()` APIs for v4. Some backported tests are disabled, but those are not related to the new API. Note that `Buffer.from(arrayBuffer[, byteOffset [, length]])` is not supported in v4.x, only `Buffer.from(arrayBuffer)` is. Refs: #4682 Refs: #5833 Refs: #7475 PR-URL: #7562 Reviewed-By: Trevor Norris <trev.norris@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Nikolai Vavilov <vvnicholas@gmail.com>

MylesBorins · 2016-07-08T14:49:21Z

landed in 350e8c6...34776ec

ChALkeR · 2016-07-08T17:09:26Z

@thealphanerd Thanks!

Treat negative length arguments to `Buffer()`/`allocUnsafe()` as if they were zero so the allocation does not affect the pool’s offset. Fixes: #7047 Refs: #7051 Refs: #7221 Refs: #7475 PR-URL: #7562 Reviewed-By: Trevor Norris <trev.norris@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Nikolai Vavilov <vvnicholas@gmail.com>

This backports the new `Buffer.alloc()`, `Buffer.allocUnsafe()`, `Buffer.from()`, and `Buffer.allocUnsafeSlow()` APIs for v4. Some backported tests are disabled, but those are not related to the new API. Note that `Buffer.from(arrayBuffer[, byteOffset [, length]])` is not supported in v4.x, only `Buffer.from(arrayBuffer)` is. Refs: #4682 Refs: #5833 Refs: #7475 PR-URL: #7562 Reviewed-By: Trevor Norris <trev.norris@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Nikolai Vavilov <vvnicholas@gmail.com>

Treat negative length arguments to `Buffer()`/`allocUnsafe()` as if they were zero so the allocation does not affect the pool’s offset. Fixes: #7047 Refs: #7051 Refs: #7221 Refs: #7475 PR-URL: #7562 Reviewed-By: Trevor Norris <trev.norris@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Nikolai Vavilov <vvnicholas@gmail.com>

This backports the new `Buffer.alloc()`, `Buffer.allocUnsafe()`, `Buffer.from()`, and `Buffer.allocUnsafeSlow()` APIs for v4. Some backported tests are disabled, but those are not related to the new API. Note that `Buffer.from(arrayBuffer[, byteOffset [, length]])` is not supported in v4.x, only `Buffer.from(arrayBuffer)` is. Refs: #4682 Refs: #5833 Refs: #7475 PR-URL: #7562 Reviewed-By: Trevor Norris <trev.norris@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Nikolai Vavilov <vvnicholas@gmail.com>

Notable Changes: This list is not yet complete. Please comment in the thread with commits you think should be included. Descriptions will also be updated in a later release candidate. Semver Minor: * buffer: * backport new buffer constructor APIs to v4.x (Сковорода Никита Андреевич) #7562 * ignore negative allocation lengths (Anna Henningsen) #7562 * backport --zero-fill-buffers cli option (James M Snell) #5745 * build: * add Intel Vtune profiling support (Chunyang Dai) #5527 * repl: * copying tabs shouldn't trigger completion (Eugene Obrezkov) #5958 * src: * add node::FreeEnvironment public API (Cheng Zhao) #3098 * test: * run v8 tests from node tree (Bryon Leung) #4704 * V8: * backport 9c927d0f01 from V8 upstream (Myles Borins) #7451 * cherry-pick 68e89fb from v8's upstream (Fedor Indutny) #3779 Semver Patch: * **libuv**: * upgrade libuv to 1.9.1 (Saúl Ibarra Corretgé) #6796 * upgrade libuv to 1.9.0 (Saúl Ibarra Corretgé) #5994

Notable Changes: This list is not yet complete. Please comment in the thread with commits you think should be included. Descriptions will also be updated in a later release candidate. Semver Minor: * buffer: * backport new buffer constructor APIs to v4.x (Сковорода Никита Андреевич) #7562 * ignore negative allocation lengths (Anna Henningsen) #7562 * backport --zero-fill-buffers cli option (James M Snell) #5745 * build: * add Intel Vtune profiling support (Chunyang Dai) #5527 * repl: * copying tabs shouldn't trigger completion (Eugene Obrezkov) #5958 * src: * add node::FreeEnvironment public API (Cheng Zhao) #3098 * test: * run v8 tests from node tree (Bryon Leung) #4704 * V8: * Add post portem data to imrpove object inspection and function's context variables inspection (Fedor Indutny) #3779 Semver Patch: * **libuv**: * upgrade libuv to 1.9.1 (Saúl Ibarra Corretgé) #6796 * upgrade libuv to 1.9.0 (Saúl Ibarra Corretgé) #5994

Treat negative length arguments to `Buffer()`/`allocUnsafe()` as if they were zero so the allocation does not affect the pool’s offset. Fixes: #7047 Refs: #7051 Refs: #7221 Refs: #7475 PR-URL: #7562 Reviewed-By: Trevor Norris <trev.norris@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Nikolai Vavilov <vvnicholas@gmail.com>

This backports the new `Buffer.alloc()`, `Buffer.allocUnsafe()`, `Buffer.from()`, and `Buffer.allocUnsafeSlow()` APIs for v4. Some backported tests are disabled, but those are not related to the new API. Note that `Buffer.from(arrayBuffer[, byteOffset [, length]])` is not supported in v4.x, only `Buffer.from(arrayBuffer)` is. Refs: #4682 Refs: #5833 Refs: #7475 PR-URL: #7562 Reviewed-By: Trevor Norris <trev.norris@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Nikolai Vavilov <vvnicholas@gmail.com>

Treat negative length arguments to `Buffer()`/`allocUnsafe()` as if they were zero so the allocation does not affect the pool’s offset. Fixes: #7047 Refs: #7051 Refs: #7221 Refs: #7475 PR-URL: #7562 Reviewed-By: Trevor Norris <trev.norris@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Nikolai Vavilov <vvnicholas@gmail.com>

This backports the new `Buffer.alloc()`, `Buffer.allocUnsafe()`, `Buffer.from()`, and `Buffer.allocUnsafeSlow()` APIs for v4. Some backported tests are disabled, but those are not related to the new API. Note that `Buffer.from(arrayBuffer[, byteOffset [, length]])` is not supported in v4.x, only `Buffer.from(arrayBuffer)` is. Refs: #4682 Refs: #5833 Refs: #7475 PR-URL: #7562 Reviewed-By: Trevor Norris <trev.norris@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Nikolai Vavilov <vvnicholas@gmail.com>

Notable Changes: This list is not yet complete. Please comment in the thread with commits you think should be included. Descriptions will also be updated in a later release candidate. Semver Minor: * buffer: * backport new buffer constructor APIs to v4.x (Сковорода Никита Андреевич) #7562 * backport --zero-fill-buffers cli option (James M Snell) #5745 * build: * add Intel Vtune profiling support (Chunyang Dai) #5527 * repl: * copying tabs shouldn't trigger completion (Eugene Obrezkov) #5958 * src: * add node::FreeEnvironment public API (Cheng Zhao) #3098 * test: * run v8 tests from node tree (Bryon Leung) #4704 * V8: * Add post portem data to imrpove object inspection and function's context variables inspection (Fedor Indutny) #3779 Semver Patch: * **buffer**: * ignore negative allocation lengths (Anna Henningsen) #7562 * **libuv**: * upgrade libuv to 1.9.1 (Saúl Ibarra Corretgé) #6796 * upgrade libuv to 1.9.0 (Saúl Ibarra Corretgé) #5994 * **npm**: * upgrade to 2.15.9 (Kat Marchán) #7692

Notable Changes: Semver Minor: * buffer: * backport new buffer constructor APIs to v4.x (Сковорода Никита Андреевич) #7562 * backport --zero-fill-buffers cli option (James M Snell) #5745 * build: * add Intel Vtune profiling support (Chunyang Dai) #5527 * repl: * copying tabs shouldn't trigger completion (Eugene Obrezkov) #5958 * src: * add node::FreeEnvironment public API (Cheng Zhao) #3098 * test: * run v8 tests from node tree (Bryon Leung) #4704 * V8: * Add post mortem data to improve object inspection and function's context variables inspection (Fedor Indutny) #3779 Semver Patch: * **buffer**: * ignore negative allocation lengths (Anna Henningsen) #7562 * **crypto**: * update root certificates (Ben Noordhuis) #7363 * **libuv**: * upgrade libuv to 1.9.1 (Saúl Ibarra Corretgé) #6796 * upgrade libuv to 1.9.0 (Saúl Ibarra Corretgé) #5994 * **npm**: * upgrade to 2.15.9 (Kat Marchán) #7692

Add version detection and use deprecated `new Buffer(<string>)`, if the new Buffer API isn't supported by runtime. See nodejs/node#7562.

ChALkeR added buffer Issues and PRs related to the buffer subsystem. semver-minor PRs that contain new features and should be released in the next minor version. labels Jul 6, 2016

ChALkeR force-pushed the buffer-v4.x-staging branch from 05d35ba to 308d9ba Compare July 6, 2016 12:33

ChALkeR mentioned this pull request Jul 6, 2016

New Buffer API backport to v4.x #7475

Closed

4 tasks

mscdex added the v4.x label Jul 6, 2016

rvagg reviewed Jul 7, 2016
View reviewed changes

addaleax and others added 2 commits July 8, 2016 06:36

ChALkeR force-pushed the buffer-v4.x-staging branch from 308d9ba to 1b39ebc Compare July 8, 2016 03:36

MylesBorins closed this Jul 8, 2016

seishun mentioned this pull request Jul 8, 2016

buffer: runtime-deprecate Buffer constructor #7152

Closed

2 tasks

MylesBorins mentioned this pull request Jul 12, 2016

Semver-Minor commits in v4.x-staging nodejs/Release#120

Closed

MylesBorins mentioned this pull request Jul 12, 2016

v4.5.0 proposal #7688

Merged

ChALkeR mentioned this pull request Jul 29, 2016

Document process for Semver Major Changes and Deprecation Policy #7912

Closed

cxam mentioned this pull request Aug 2, 2017

Handle Buffer.from TypeError on node versions prior to 4.5.x stripe/stripe-node#365

Merged

SheetJSDev mentioned this pull request Jun 21, 2018

typeerror in node version 4 and lower verison SheetJS/sheetjs#1150

Closed

phureewat29 mentioned this pull request Oct 25, 2018

Replace deprecated Buffer constructor omise/omise-node#99

Merged

lukastribus mentioned this pull request May 13, 2019

ValueError: could not convert string to float: b'undefined' Anorov/cloudflare-scrape#241

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New Buffer API backport to v4.x (staging branch) #7562

New Buffer API backport to v4.x (staging branch) #7562

ChALkeR commented Jul 6, 2016

ChALkeR commented Jul 6, 2016 •

edited

Loading

rvagg Jul 7, 2016

ChALkeR Jul 7, 2016

ChALkeR Jul 8, 2016

rvagg commented Jul 7, 2016

ChALkeR commented Jul 7, 2016 •

edited

Loading

rvagg commented Jul 7, 2016

ChALkeR commented Jul 8, 2016

ChALkeR commented Jul 8, 2016

rvagg commented Jul 8, 2016

jasnell commented Jul 8, 2016

ChALkeR commented Jul 8, 2016 •

edited

Loading

MylesBorins commented Jul 8, 2016

ChALkeR commented Jul 8, 2016

New Buffer API backport to v4.x (staging branch) #7562

New Buffer API backport to v4.x (staging branch) #7562

Conversation

ChALkeR commented Jul 6, 2016

Checklist

Affected core subsystem(s)

Description of change

ChALkeR commented Jul 6, 2016 • edited Loading

rvagg Jul 7, 2016

Choose a reason for hiding this comment

ChALkeR Jul 7, 2016

Choose a reason for hiding this comment

ChALkeR Jul 8, 2016

Choose a reason for hiding this comment

rvagg commented Jul 7, 2016

ChALkeR commented Jul 7, 2016 • edited Loading

rvagg commented Jul 7, 2016

ChALkeR commented Jul 8, 2016

ChALkeR commented Jul 8, 2016

rvagg commented Jul 8, 2016

jasnell commented Jul 8, 2016

ChALkeR commented Jul 8, 2016 • edited Loading

MylesBorins commented Jul 8, 2016

ChALkeR commented Jul 8, 2016

ChALkeR commented Jul 6, 2016 •

edited

Loading

ChALkeR commented Jul 7, 2016 •

edited

Loading

ChALkeR commented Jul 8, 2016 •

edited

Loading