From 2ae98ce7cb95dee79e839afb2ee4c7d77d7670cd Mon Sep 17 00:00:00 2001 From: cjihrig Date: Sun, 11 Mar 2018 10:33:42 -0400 Subject: [PATCH 001/129] lib: define printErr() in script string MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This commit moves the printErr() function, used by the tick profiler processer, into the code string passed to vm.runInThisContext(). PR-URL: https://github.com/nodejs/node/pull/19285 Fixes: https://github.com/nodejs/node/issues/19260 Reviewed-By: Michaël Zasso Reviewed-By: Gus Caplan Reviewed-By: Daniel Bevenius Reviewed-By: James M Snell Reviewed-By: Tobias Nießen Reviewed-By: Ruben Bridgewater --- lib/internal/v8_prof_processor.js | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/lib/internal/v8_prof_processor.js b/lib/internal/v8_prof_processor.js index 7592253060294c..4799a88f69ca54 100644 --- a/lib/internal/v8_prof_processor.js +++ b/lib/internal/v8_prof_processor.js @@ -21,11 +21,6 @@ scriptFiles.forEach(function(s) { script += process.binding('natives')[s] + '\n'; }); -// eslint-disable-next-line no-unused-vars -function printErr(err) { - console.error(err); -} - const tickArguments = []; if (process.platform === 'darwin') { tickArguments.push('--mac'); @@ -36,6 +31,7 @@ tickArguments.push.apply(tickArguments, process.argv.slice(1)); script = `(function(module, require) { arguments = ${JSON.stringify(tickArguments)}; function write (s) { process.stdout.write(s) } + function printErr(err) { console.error(err); } ${script} })`; vm.runInThisContext(script)(module, require); From b0d6067d874be735a8b3018e2b66752bd2628584 Mon Sep 17 00:00:00 2001 From: Myles Borins Date: Tue, 11 Sep 2018 01:34:26 -0400 Subject: [PATCH 002/129] doc: fix 8.12.0 changelog MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit PR-URL: https://github.com/nodejs/node/pull/22803 Reviewed-By: Michaël Zasso Reviewed-By: Trivikram Kamat Reviewed-By: Luigi Pinca Reviewed-By: Colin Ihrig Reviewed-By: Tobias Nießen Reviewed-By: Richard Lau --- doc/changelogs/CHANGELOG_V8.md | 37 ++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/doc/changelogs/CHANGELOG_V8.md b/doc/changelogs/CHANGELOG_V8.md index 0c53fa5dec40d7..fdd5d4271bc057 100644 --- a/doc/changelogs/CHANGELOG_V8.md +++ b/doc/changelogs/CHANGELOG_V8.md @@ -62,6 +62,43 @@ will be supported actively until April 2019 and maintained until December 2019. ### Notable Changes +* **async_hooks**: + - rename PromiseWrap.parentId (Ali Ijaz Sheikh) [#18633](https://github.com/nodejs/node/pull/18633) + - remove runtime deprecation (Ali Ijaz Sheikh) [#19517](https://github.com/nodejs/node/pull/19517) + - deprecate unsafe emit{Before,After} (Ali Ijaz Sheikh) [#18513](https://github.com/nodejs/node/pull/18513) +* **cluster**: + - add cwd to cluster.settings (cjihrig) [#18399](https://github.com/nodejs/node/pull/18399) + - support windowsHide option for workers (Todd Wong) [#17412](https://github.com/nodejs/node/pull/17412) +* **crypto**: + - allow passing null as IV unless required (Tobias Nießen) [#18644](https://github.com/nodejs/node/pull/18644) +* **deps**: + - upgrade npm to 6.4.1 (Kat Marchán) [#22591](https://github.com/nodejs/node/pull/22591) + - upgrade libuv to 1.19.2 (cjihrig) [#18918](https://github.com/nodejs/node/pull/18918) + - Upgrade node-inspect to 1.11.5 (Jan Krems) [#21055](https://github.com/nodejs/node/pull/21055) +* **fs,net**: + - support as and as+ flags in stringToFlags() (Sarat Addepalli) [#18801](https://github.com/nodejs/node/pull/18801) + - emit 'ready' for fs streams and sockets (Sameer Srivastava) [#19408](https://github.com/nodejs/node/pull/19408) +* **http, http2**: + - add options to http.createServer() (Peter Marton) [#15752](https://github.com/nodejs/node/pull/15752)- + - add 103 Early Hints status code (Yosuke Furukawa) [#16644](https://github.com/nodejs/node/pull/16644) + - add http fallback options to .createServer (Peter Marton) [#15752](https://github.com/nodejs/node/pull/15752) +* **n-api**: + - take n-api out of experimental (Michael Dawson) [#19262](https://github.com/nodejs/node/pull/19262) +* **perf_hooks**: + - add warning when too many entries in the timeline (James M Snell) [#18087](https://github.com/nodejs/node/pull/18087) +* **src**: + - add public API for managing NodePlatform (Cheng Zhao) [#16981](https://github.com/nodejs/node/pull/16981) + - allow --perf-(basic-)?prof in NODE\_OPTIONS (Leko) [#17600](https://github.com/nodejs/node/pull/17600) + - node internals' postmortem metadata (Matheus Marchini) [#14901](https://github.com/nodejs/node/pull/14901) +* **tls**: + - expose Finished messages in TLSSocket (Anton Salikhmetov) [#19102](https://github.com/nodejs/node/pull/19102) +* **trace_events**: + - add file pattern cli option (Andreas Madsen) [#18480](https://github.com/nodejs/node/pull/18480) +* **util**: + - implement util.getSystemErrorName() (Joyee Cheung) [#18186](https://github.com/nodejs/node/pull/18186) + +### Commits + * [[`b7f9334454`](https://github.com/nodejs/node/commit/b7f9334454)] - **(SEMVER-MINOR)** **async_hooks**: rename PromiseWrap.parentId (Ali Ijaz Sheikh) [#18633](https://github.com/nodejs/node/pull/18633) * [[`373f4d6225`](https://github.com/nodejs/node/commit/373f4d6225)] - **(SEMVER-MINOR)** **async_hooks**: remove runtime deprecation (Ali Ijaz Sheikh) [#19517](https://github.com/nodejs/node/pull/19517) * [[`daacff8584`](https://github.com/nodejs/node/commit/daacff8584)] - **(SEMVER-MINOR)** **async_hooks**: deprecate unsafe emit{Before,After} (Ali Ijaz Sheikh) [#18513](https://github.com/nodejs/node/pull/18513) From 0340dd8c8d0227f31bd93f32589c8929fe4d9045 Mon Sep 17 00:00:00 2001 From: Vse Mozhet Byt Date: Fri, 6 Apr 2018 18:28:20 +0300 Subject: [PATCH 003/129] doc: add and unify return statements in crypto.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Conform return statements to the style guide and tool parsers. Also bring back a description fragment that seems to be erroneously deleted in https://github.com/nodejs/node/commit/1e07acd476309e7ddc4981160b89731b61a31179 Backport-PR-URL: https://github.com/nodejs/node/pull/22870 PR-URL: https://github.com/nodejs/node/pull/19853 Reviewed-By: James M Snell Reviewed-By: Rich Trott Reviewed-By: Luigi Pinca Reviewed-By: Gibson Fahnestock Reviewed-By: Trivikram Kamat Reviewed-By: Ruben Bridgewater Reviewed-By: Tobias Nießen --- doc/api/crypto.md | 108 +++++++++++++++++++++++---------------- tools/doc/type-parser.js | 6 +++ 2 files changed, 69 insertions(+), 45 deletions(-) diff --git a/doc/api/crypto.md b/doc/api/crypto.md index 06e47d6d5c926c..d5ee1baab9ef1b 100644 --- a/doc/api/crypto.md +++ b/doc/api/crypto.md @@ -68,8 +68,8 @@ const cert2 = crypto.Certificate(); added: v0.11.8 --> - `spkac` {string | Buffer | TypedArray | DataView} -- Returns {Buffer} The challenge component of the `spkac` data structure, which -includes a public key and a challenge. +- Returns: {Buffer} The challenge component of the `spkac` data structure, which + includes a public key and a challenge. ```js const cert = require('crypto').Certificate(); @@ -84,8 +84,8 @@ console.log(challenge.toString('utf8')); added: v0.11.8 --> - `spkac` {string | Buffer | TypedArray | DataView} -- Returns {Buffer} The public key component of the `spkac` data structure, -which includes a public key and a challenge. +- Returns: {Buffer} The public key component of the `spkac` data structure, + which includes a public key and a challenge. ```js const cert = require('crypto').Certificate(); @@ -100,8 +100,8 @@ console.log(publicKey); added: v0.11.8 --> - `spkac` {Buffer | TypedArray | DataView} -- Returns {boolean} `true` if the given `spkac` data structure is valid, `false` -otherwise. +- Returns: {boolean} `true` if the given `spkac` data structure is valid, + `false` otherwise. ```js const cert = require('crypto').Certificate(); @@ -178,10 +178,10 @@ console.log(encrypted); added: v0.1.94 --> - `outputEncoding` {string} - -Returns any remaining enciphered contents. If `outputEncoding` -parameter is one of `'latin1'`, `'base64'` or `'hex'`, a string is returned. -If an `outputEncoding` is not provided, a [`Buffer`][] is returned. +- Returns: {Buffer | string} Any remaining enciphered contents. + If `outputEncoding` parameter is one of `'latin1'`, `'base64'` or `'hex'`, + a string is returned. If an `outputEncoding` is not provided, a [`Buffer`][] + is returned. Once the `cipher.final()` method has been called, the `Cipher` object can no longer be used to encrypt data. Attempts to call `cipher.final()` more than @@ -204,10 +204,10 @@ The `cipher.setAAD()` method must be called before [`cipher.update()`][]. - -When using an authenticated encryption mode (only `GCM` is currently -supported), the `cipher.getAuthTag()` method returns a [`Buffer`][] containing -the _authentication tag_ that has been computed from the given data. +- Returns: {Buffer} When using an authenticated encryption mode (only `GCM` is + currently supported), the `cipher.getAuthTag()` method returns a [`Buffer`][] + containing the _authentication tag_ that has been computed from the given + data. The `cipher.getAuthTag()` method should only be called after encryption has been completed using the [`cipher.final()`][] method. @@ -217,7 +217,7 @@ been completed using the [`cipher.final()`][] method. added: v0.7.1 --> - `autoPadding` {boolean} **Default:** `true` -- Returns the {Cipher} for method chaining. +- Returns: {Cipher} for method chaining. When using block encryption algorithms, the `Cipher` class will automatically add padding to the input data to the appropriate block size. To disable the @@ -242,6 +242,7 @@ changes: - `data` {string | Buffer | TypedArray | DataView} - `inputEncoding` {string} - `outputEncoding` {string} +- Returns: {Buffer | string} Updates the cipher with `data`. If the `inputEncoding` argument is given, its value must be one of `'utf8'`, `'ascii'`, or `'latin1'` and the `data` @@ -331,10 +332,10 @@ console.log(decrypted); added: v0.1.94 --> - `outputEncoding` {string} - -Returns any remaining deciphered contents. If `outputEncoding` -parameter is one of `'latin1'`, `'ascii'` or `'utf8'`, a string is returned. -If an `outputEncoding` is not provided, a [`Buffer`][] is returned. +- Returns: {Buffer | string} Any remaining deciphered contents. + If `outputEncoding` parameter is one of `'latin1'`, `'ascii'` or `'utf8'`, + a string is returned. If an `outputEncoding` is not provided, a [`Buffer`][] + is returned. Once the `decipher.final()` method has been called, the `Decipher` object can no longer be used to decrypt data. Attempts to call `decipher.final()` more @@ -349,7 +350,7 @@ changes: description: This method now returns a reference to `decipher`. --> - `buffer` {Buffer | TypedArray | DataView} -- Returns the {Cipher} for method chaining. +- Returns: {Cipher} for method chaining. When using an authenticated encryption mode (only `GCM` is currently supported), the `decipher.setAAD()` method sets the value used for the @@ -366,7 +367,7 @@ changes: description: This method now returns a reference to `decipher`. --> - `buffer` {Buffer | TypedArray | DataView} -- Returns the {Cipher} for method chaining. +- Returns: {Cipher} for method chaining. When using an authenticated encryption mode (only `GCM` is currently supported), the `decipher.setAuthTag()` method is used to pass in the @@ -390,7 +391,7 @@ The `decipher.setAuthTag()` method must be called before added: v0.7.1 --> - `autoPadding` {boolean} **Default:** `true` -- Returns the {Cipher} for method chaining. +- Returns: {Cipher} for method chaining. When data has been encrypted without standard block padding, calling `decipher.setAutoPadding(false)` will disable automatic padding to prevent @@ -413,6 +414,7 @@ changes: - `data` {string | Buffer | TypedArray | DataView} - `inputEncoding` {string} - `outputEncoding` {string} +- Returns: {Buffer | string} Updates the decipher with `data`. If the `inputEncoding` argument is given, its value must be one of `'latin1'`, `'base64'`, or `'hex'` and the `data` @@ -467,6 +469,7 @@ added: v0.5.0 - `otherPublicKey` {string | Buffer | TypedArray | DataView} - `inputEncoding` {string} - `outputEncoding` {string} +- Returns: {Buffer | string} Computes the shared secret using `otherPublicKey` as the other party's public key and returns the computed shared secret. The supplied @@ -484,6 +487,7 @@ If `outputEncoding` is given a string is returned; otherwise, a added: v0.5.0 --> - `encoding` {string} +- Returns: {Buffer | string} Generates private and public Diffie-Hellman key values, and returns the public key in the specified `encoding`. This key should be @@ -496,6 +500,7 @@ or `'base64'`. If `encoding` is provided a string is returned; otherwise a added: v0.5.0 --> - `encoding` {string} +- Returns: {Buffer | string} Returns the Diffie-Hellman generator in the specified `encoding`, which can be `'latin1'`, `'hex'`, or `'base64'`. If `encoding` is provided a string is @@ -506,6 +511,7 @@ returned; otherwise a [`Buffer`][] is returned. added: v0.5.0 --> - `encoding` {string} +- Returns: {Buffer | string} Returns the Diffie-Hellman prime in the specified `encoding`, which can be `'latin1'`, `'hex'`, or `'base64'`. If `encoding` is provided a string is @@ -516,6 +522,7 @@ returned; otherwise a [`Buffer`][] is returned. added: v0.5.0 --> - `encoding` {string} +- Returns: {Buffer | string} Returns the Diffie-Hellman private key in the specified `encoding`, which can be `'latin1'`, `'hex'`, or `'base64'`. If `encoding` is provided a @@ -526,6 +533,7 @@ string is returned; otherwise a [`Buffer`][] is returned. added: v0.5.0 --> - `encoding` {string} +- Returns: {Buffer | string} Returns the Diffie-Hellman public key in the specified `encoding`, which can be `'latin1'`, `'hex'`, or `'base64'`. If `encoding` is provided a @@ -613,6 +621,7 @@ changes: - `otherPublicKey` {string | Buffer | TypedArray | DataView} - `inputEncoding` {string} - `outputEncoding` {string} +- Returns: {Buffer | string} Computes the shared secret using `otherPublicKey` as the other party's public key and returns the computed shared secret. The supplied @@ -631,6 +640,7 @@ added: v0.11.14 --> - `encoding` {string} - `format` {string} **Default:** `uncompressed` +- Returns: {Buffer | string} Generates private and public EC Diffie-Hellman key values, and returns the public key in the specified `format` and `encoding`. This key should be @@ -649,10 +659,9 @@ is returned. added: v0.11.14 --> - `encoding` {string} - -Returns the EC Diffie-Hellman private key in the specified `encoding`, -which can be `'latin1'`, `'hex'`, or `'base64'`. If `encoding` is provided -a string is returned; otherwise a [`Buffer`][] is returned. +- Returns: {Buffer | string} The EC Diffie-Hellman private key in the specified + `encoding`, which can be `'latin1'`, `'hex'`, or `'base64'`. If `encoding` + is provided a string is returned; otherwise a [`Buffer`][] is returned. ### ecdh.getPublicKey([encoding][, format]) - `encoding` {string} - `format` {string} **Default:** `uncompressed` - -Returns the EC Diffie-Hellman public key in the specified `encoding` and -`format`. +- Returns: {Buffer | string} The EC Diffie-Hellman public key in the specified + `encoding` and `format`. The `format` argument specifies point encoding and can be `'compressed'` or `'uncompressed'`. If `format` is not specified the point will be returned in @@ -798,6 +806,7 @@ console.log(hash.digest('hex')); added: v0.1.92 --> - `encoding` {string} +- Returns: {Buffer | string} Calculates the digest of all of the data passed to be hashed (using the [`hash.update()`][] method). The `encoding` can be `'hex'`, `'latin1'` or @@ -889,6 +898,7 @@ console.log(hmac.digest('hex')); added: v0.1.94 --> - `encoding` {string} +- Returns: {Buffer | string} Calculates the HMAC digest of all of the data passed using [`hmac.update()`][]. The `encoding` can be `'hex'`, `'latin1'` or `'base64'`. If `encoding` is @@ -992,6 +1002,7 @@ changes: - `key` {string} - `passphrase` {string} - `outputFormat` {string} +- Returns: {Buffer | string} Calculates the signature on all the data passed through using either [`sign.update()`][] or [`sign.write()`][stream-writable-write]. @@ -1115,6 +1126,8 @@ changes: - `object` {string | Object} - `signature` {string | Buffer | TypedArray | DataView} - `signatureFormat` {string} +- Returns: {boolean} `true` or `false` depending on the validity of the + signature for the data and public key. Verifies the provided data using the given `object` and `signature`. The `object` argument can be either a string containing a PEM encoded object, @@ -1140,9 +1153,6 @@ If a `signatureFormat` is specified, the `signature` is expected to be a string; otherwise `signature` is expected to be a [`Buffer`][], `TypedArray`, or `DataView`. -Returns `true` or `false` depending on the validity of the signature for -the data and public key. - The `verify` object can not be used again after `verify.verify()` has been called. Multiple calls to `verify.verify()` will result in an error being thrown. @@ -1153,10 +1163,9 @@ thrown. - -Returns an object containing commonly used constants for crypto and security -related operations. The specific constants currently defined are described in -[Crypto Constants][]. +- Returns: {Object} An object containing commonly used constants for crypto and + security related operations. The specific constants currently defined are + described in [Crypto Constants][]. ### crypto.DEFAULT_ENCODING - `algorithm` {string} - `options` {Object} [`stream.transform` options][] +- Returns: {Hash} Creates and returns a `Hash` object that can be used to generate hash digests using the given `algorithm`. Optional `options` argument controls stream @@ -1413,6 +1427,7 @@ added: v0.1.94 - `algorithm` {string} - `key` {string | Buffer | TypedArray | DataView} - `options` {Object} [`stream.transform` options][] +- Returns: {Hmac} Creates and returns an `Hmac` object that uses the given `algorithm` and `key`. Optional `options` argument controls stream behavior. @@ -1450,6 +1465,7 @@ added: v0.1.92 --> - `algorithm` {string} - `options` {Object} [`stream.Writable` options][] +- Returns: {Sign} Creates and returns a `Sign` object that uses the given `algorithm`. Use [`crypto.getHashes()`][] to obtain an array of names of the available @@ -1462,6 +1478,7 @@ added: v0.1.92 --> - `algorithm` {string} - `options` {Object} [`stream.Writable` options][] +- Returns: {Verify} Creates and returns a `Verify` object that uses the given algorithm. Use [`crypto.getHashes()`][] to obtain an array of names of the available @@ -1472,8 +1489,8 @@ signing algorithms. Optional `options` argument controls the - -Returns an array with the names of the supported cipher algorithms. +- Returns: {string[]} An array with the names of the supported cipher + algorithms. Example: @@ -1486,8 +1503,7 @@ console.log(ciphers); // ['aes-128-cbc', 'aes-128-ccm', ...] - -Returns an array with the names of the supported elliptic curves. +- Returns: {string[]} An array with the names of the supported elliptic curves. Example: @@ -1501,6 +1517,7 @@ console.log(curves); // ['Oakley-EC2N-3', 'Oakley-EC2N-4', ...] added: v0.7.5 --> - `groupName` {string} +- Returns: {Object} Creates a predefined `DiffieHellman` key exchange object. The supported groups are: `'modp1'`, `'modp2'`, `'modp5'` (defined in @@ -1534,9 +1551,8 @@ console.log(aliceSecret === bobSecret); - -Returns an array of the names of the supported hash algorithms, -such as `RSA-SHA256`. +- Returns: {string[]} An array of the names of the supported hash algorithms, + such as `'RSA-SHA256'`. Example: @@ -1622,6 +1638,7 @@ changes: - `iterations` {number} - `keylen` {number} - `digest` {string} +- Returns: {Buffer} Provides a synchronous Password-Based Key Derivation Function 2 (PBKDF2) implementation. A selected HMAC digest algorithm specified by `digest` is @@ -1737,6 +1754,7 @@ added: v0.5.8 - `callback` {Function} - `err` {Error} - `buf` {Buffer} +- Returns: {Buffer} if the `callback` function is not provided. Generates cryptographically strong pseudo-random data. The `size` argument is a number indicating the number of bytes to generate. @@ -1789,11 +1807,10 @@ added: v7.10.0 * `buffer` {Buffer|Uint8Array} Must be supplied. * `offset` {number} **Default:** `0` * `size` {number} **Default:** `buffer.length - offset` +* Returns: {Buffer} Synchronous version of [`crypto.randomFill()`][]. -Returns `buffer` - ```js const buf = Buffer.alloc(10); console.log(crypto.randomFillSync(buf).toString('hex')); @@ -1885,6 +1902,7 @@ added: v6.6.0 --> - `a` {Buffer | TypedArray | DataView} - `b` {Buffer | TypedArray | DataView} +- Returns: {boolean} This function is based on a constant-time algorithm. Returns true if `a` is equal to `b`, without leaking timing information that diff --git a/tools/doc/type-parser.js b/tools/doc/type-parser.js index 39acd60a19d00d..f8d12ad11b99dc 100644 --- a/tools/doc/type-parser.js +++ b/tools/doc/type-parser.js @@ -40,6 +40,12 @@ const customTypesMap = { 'cluster.Worker': 'cluster.html#cluster_class_worker', + 'Cipher': 'crypto.html#crypto_class_cipher', + 'Decipher': 'crypto.html#crypto_class_decipher', + 'Hash': 'crypto.html#crypto_class_hash', + 'Hmac': 'crypto.html#crypto_class_hmac', + 'Sign': 'crypto.html#crypto_class_sign', + 'Verify': 'crypto.html#crypto_class_verify', 'crypto.constants': 'crypto.html#crypto_crypto_constants_1', 'dgram.Socket': 'dgram.html#dgram_class_dgram_socket', From f2158f30fbf0ca63886a0f8c22cbbf1b683035bd Mon Sep 17 00:00:00 2001 From: Rich Trott Date: Sat, 1 Sep 2018 22:47:37 -0700 Subject: [PATCH 004/129] test: improve assertion in test-inspector.js Remove an unecessary string literal from assert.strictEqual() call in test-inspector.js. The string literal is printed instead of the value that causes an error. Removing the string literal allows the value that caused the error to be printed. This improves the troubleshooting experience when the test fails due to that assertion. Backport-PR-URL: https://github.com/nodejs/node/pull/22888 PR-URL: https://github.com/nodejs/node/pull/22849 Reviewed-By: Teddy Katz Reviewed-By: Matteo Collina Reviewed-By: Anna Henningsen Reviewed-By: Trivikram Kamat Reviewed-By: Luigi Pinca Reviewed-By: Colin Ihrig --- test/sequential/test-inspector.js | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/test/sequential/test-inspector.js b/test/sequential/test-inspector.js index 50fcff2d425655..e7fec6f256399c 100644 --- a/test/sequential/test-inspector.js +++ b/test/sequential/test-inspector.js @@ -33,8 +33,7 @@ function checkBadPath(err) { } function checkException(message) { - assert.strictEqual(message.exceptionDetails, undefined, - 'An exception occurred during execution'); + assert.strictEqual(message.exceptionDetails, undefined); } function assertNoUrlsWhileConnected(response) { From e9416d4f678ac435f2234d94ede02daace29aa8e Mon Sep 17 00:00:00 2001 From: Rich Trott Date: Sun, 2 Sep 2018 10:55:33 -0700 Subject: [PATCH 005/129] test: simplify assertion in http2 tests In test-http2-timeout-large-write.js and test-http2-timeout-large-write-file.js: Use assert.ok() on a boolean that the test itself creates and sets, rather than assert.strictEqual(). This allows us to use a static message without running afoul of the upcoming "do not use string literals with assert.strictEqual()" lint rule. Backport-PR-URL: https://github.com/nodejs/node/pull/22888 PR-URL: https://github.com/nodejs/node/pull/22849 Reviewed-By: Teddy Katz Reviewed-By: Matteo Collina Reviewed-By: Anna Henningsen Reviewed-By: Trivikram Kamat Reviewed-By: Luigi Pinca Reviewed-By: Colin Ihrig --- test/sequential/test-http2-timeout-large-write-file.js | 2 +- test/sequential/test-http2-timeout-large-write.js | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/test/sequential/test-http2-timeout-large-write-file.js b/test/sequential/test-http2-timeout-large-write-file.js index 910e7a0fc497bd..bb366cfff04091 100644 --- a/test/sequential/test-http2-timeout-large-write-file.js +++ b/test/sequential/test-http2-timeout-large-write-file.js @@ -48,7 +48,7 @@ server.on('stream', common.mustCall((stream) => { })); server.setTimeout(serverTimeout); server.on('timeout', () => { - assert.strictEqual(didReceiveData, false, 'Should not timeout'); + assert.ok(!didReceiveData, 'Should not timeout'); }); server.listen(0, common.mustCall(() => { diff --git a/test/sequential/test-http2-timeout-large-write.js b/test/sequential/test-http2-timeout-large-write.js index a15fb46af6d28a..73114776df0a0e 100644 --- a/test/sequential/test-http2-timeout-large-write.js +++ b/test/sequential/test-http2-timeout-large-write.js @@ -40,13 +40,13 @@ server.on('stream', common.mustCall((stream) => { stream.write(content); stream.setTimeout(serverTimeout); stream.on('timeout', () => { - assert.strictEqual(didReceiveData, false, 'Should not timeout'); + assert.ok(!didReceiveData, 'Should not timeout'); }); stream.end(); })); server.setTimeout(serverTimeout); server.on('timeout', () => { - assert.strictEqual(didReceiveData, false, 'Should not timeout'); + assert.ok(!didReceiveData, 'Should not timeout'); }); server.listen(0, common.mustCall(() => { From 702d67f4c4e6988250dde078c348fe94a5e33038 Mon Sep 17 00:00:00 2001 From: Rich Trott Date: Wed, 12 Sep 2018 20:16:23 -0700 Subject: [PATCH 006/129] test: refactor flag check Refactor test-vm-run-in-new-context so that check for `--expose-gc` flag will not run afoul of an upcoming lint rule that checks that string literals are not used for the `message` argument of `assert.strictEqual()`. Backport-PR-URL: https://github.com/nodejs/node/pull/22888 PR-URL: https://github.com/nodejs/node/pull/22849 Reviewed-By: Teddy Katz Reviewed-By: Matteo Collina Reviewed-By: Anna Henningsen Reviewed-By: Trivikram Kamat Reviewed-By: Luigi Pinca Reviewed-By: Colin Ihrig --- test/parallel/test-vm-run-in-new-context.js | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/test/parallel/test-vm-run-in-new-context.js b/test/parallel/test-vm-run-in-new-context.js index 1edb061ea6a871..e506905c554c2c 100644 --- a/test/parallel/test-vm-run-in-new-context.js +++ b/test/parallel/test-vm-run-in-new-context.js @@ -26,8 +26,8 @@ const common = require('../common'); const assert = require('assert'); const vm = require('vm'); -assert.strictEqual(typeof global.gc, 'function', - 'Run this test with --expose-gc'); +if (typeof global.gc !== 'function') + assert.fail('Run this test with --expose-gc'); common.globalCheck = false; From 976d55f9e358da4e6ed6a6c78fe33da722d0c8e8 Mon Sep 17 00:00:00 2001 From: Rich Trott Date: Wed, 12 Sep 2018 20:24:28 -0700 Subject: [PATCH 007/129] test: remove string literal from assertion Remove string literal from `assert.strictEqual()` call `message` parameter and make it a comment above the assertion instead. Backport-PR-URL: https://github.com/nodejs/node/pull/22888 PR-URL: https://github.com/nodejs/node/pull/22849 Reviewed-By: Teddy Katz Reviewed-By: Matteo Collina Reviewed-By: Anna Henningsen Reviewed-By: Trivikram Kamat Reviewed-By: Luigi Pinca Reviewed-By: Colin Ihrig --- test/parallel/test-next-tick-domain.js | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/test/parallel/test-next-tick-domain.js b/test/parallel/test-next-tick-domain.js index 5c526197926df7..3e55ef3225fc40 100644 --- a/test/parallel/test-next-tick-domain.js +++ b/test/parallel/test-next-tick-domain.js @@ -27,5 +27,5 @@ const origNextTick = process.nextTick; require('domain'); -assert.strictEqual(origNextTick, process.nextTick, - 'Requiring domain should not change nextTick'); +// Requiring domain should not change nextTick. +assert.strictEqual(origNextTick, process.nextTick); From 52b21caff28a0792051dfa1bc537723f8acc0c95 Mon Sep 17 00:00:00 2001 From: Rich Trott Date: Thu, 13 Sep 2018 13:04:09 -0700 Subject: [PATCH 008/129] test: remove string literal from assertion Remove string literal as assertion message in call to assert.strictEqual() in test-dns-resolveany-bad-ancount. Backport-PR-URL: https://github.com/nodejs/node/pull/22888 PR-URL: https://github.com/nodejs/node/pull/22849 Reviewed-By: Teddy Katz Reviewed-By: Matteo Collina Reviewed-By: Anna Henningsen Reviewed-By: Trivikram Kamat Reviewed-By: Luigi Pinca Reviewed-By: Colin Ihrig --- test/parallel/test-dns-resolveany-bad-ancount.js | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/test/parallel/test-dns-resolveany-bad-ancount.js b/test/parallel/test-dns-resolveany-bad-ancount.js index 63ed1774b11933..82378b31bc3552 100644 --- a/test/parallel/test-dns-resolveany-bad-ancount.js +++ b/test/parallel/test-dns-resolveany-bad-ancount.js @@ -31,8 +31,8 @@ server.bind(0, common.mustCall(() => { assert.strictEqual(err.syscall, 'queryAny'); assert.strictEqual(err.hostname, 'example.org'); const descriptor = Object.getOwnPropertyDescriptor(err, 'message'); - assert.strictEqual(descriptor.enumerable, - false, 'The error message should be non-enumerable'); + // The error message should be non-enumerable. + assert.strictEqual(descriptor.enumerable, false); server.close(); })); })); From 8209ccb3132051ed2eaa25f010a96ee902396f04 Mon Sep 17 00:00:00 2001 From: Rich Trott Date: Thu, 13 Sep 2018 13:47:57 -0700 Subject: [PATCH 009/129] test: prepare test-assert for strictEqual linting Make minor modifications to test-assert.js to prepare it for linting rule that forbids the use of string literals for the third argument of assert.strictEqual(). Backport-PR-URL: https://github.com/nodejs/node/pull/22888 PR-URL: https://github.com/nodejs/node/pull/22849 Reviewed-By: Teddy Katz Reviewed-By: Matteo Collina Reviewed-By: Anna Henningsen Reviewed-By: Trivikram Kamat Reviewed-By: Luigi Pinca Reviewed-By: Colin Ihrig --- test/parallel/test-assert.js | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/test/parallel/test-assert.js b/test/parallel/test-assert.js index cdb824c3cffa8f..c73b49deb328cb 100644 --- a/test/parallel/test-assert.js +++ b/test/parallel/test-assert.js @@ -610,11 +610,11 @@ try { } try { - assert.strictEqual(1, 2, 'oh no'); + assert.strictEqual(1, 2, 'oh no'); // eslint-disable-line no-restricted-syntax } catch (e) { assert.strictEqual(e.message.split('\n')[0], 'oh no'); - assert.strictEqual(e.generatedMessage, false, - 'Message incorrectly marked as generated'); + // Message should not be marked as generated. + assert.strictEqual(e.generatedMessage, false); } { From f5985c734cf1072defc63a3b34cc37a299b7e391 Mon Sep 17 00:00:00 2001 From: Anna Henningsen Date: Sun, 25 Feb 2018 21:56:08 +0100 Subject: [PATCH 010/129] tls,http2: handle writes after SSL destroy more gracefully MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This might otherwise result in a hard crash when trying to write to a socket after a sudden disconnect. Note that the test here uses an aborted `h2load` run to create the failing requests; That’s far from ideal, but it provides a reasonably reliably reproduction at this point. Backport-PR-URL: https://github.com/nodejs/node/pull/22924 PR-URL: https://github.com/nodejs/node/pull/18987 Fixes: https://github.com/nodejs/node/issues/18973 Reviewed-By: James M Snell Reviewed-By: Matteo Collina --- src/tls_wrap.cc | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/src/tls_wrap.cc b/src/tls_wrap.cc index c3e6e699c9c22c..4c01f618a357ed 100644 --- a/src/tls_wrap.cc +++ b/src/tls_wrap.cc @@ -601,7 +601,12 @@ int TLSWrap::DoWrite(WriteWrap* w, size_t count, uv_stream_t* send_handle) { CHECK_EQ(send_handle, nullptr); - CHECK_NE(ssl_, nullptr); + + if (ssl_ == nullptr) { + ClearError(); + error_ = "Write after DestroySSL"; + return UV_EPROTO; + } bool empty = true; @@ -642,12 +647,6 @@ int TLSWrap::DoWrite(WriteWrap* w, return 0; } - if (ssl_ == nullptr) { - ClearError(); - error_ = "Write after DestroySSL"; - return UV_EPROTO; - } - crypto::MarkPopErrorOnReturn mark_pop_error_on_return; int written = 0; From d58867a6a7c9376fcbfecd94f96d36af3c787f74 Mon Sep 17 00:00:00 2001 From: Refael Ackermann Date: Mon, 13 Aug 2018 16:18:40 -0400 Subject: [PATCH 011/129] test: call gc() explicitly to avoid OOM PR-URL: https://github.com/nodejs/node/pull/22301 Refs: https://github.com/nodejs/reliability/issues/12 Refs: https://github.com/nodejs/node/issues/16354 Reviewed-By: Rich Trott Reviewed-By: Ruben Bridgewater Reviewed-By: Joyee Cheung Reviewed-By: Gus Caplan --- .../test-stringbytes-external-exceed-max-by-1-binary.js | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/test/addons/stringbytes-external-exceed-max/test-stringbytes-external-exceed-max-by-1-binary.js b/test/addons/stringbytes-external-exceed-max/test-stringbytes-external-exceed-max-by-1-binary.js index 996c01752da7c6..844ebb45bac612 100644 --- a/test/addons/stringbytes-external-exceed-max/test-stringbytes-external-exceed-max-by-1-binary.js +++ b/test/addons/stringbytes-external-exceed-max/test-stringbytes-external-exceed-max-by-1-binary.js @@ -1,3 +1,4 @@ +// Flags: --expose-gc 'use strict'; const common = require('../../common'); @@ -29,10 +30,13 @@ assert.throws(function() { buf.toString('latin1'); }, /"toString\(\)" failed/); +// FIXME: Free the memory early to avoid OOM. +// REF: https://github.com/nodejs/reliability/issues/12#issuecomment-412619655 +global.gc(); let maxString = buf.toString('latin1', 1); assert.strictEqual(maxString.length, kStringMaxLength); -// Free the memory early instead of at the end of the next assignment maxString = undefined; +global.gc(); maxString = buf.toString('latin1', 0, kStringMaxLength); assert.strictEqual(maxString.length, kStringMaxLength); From 9e2077afeee0b135fa6961a96314e00a95350ed0 Mon Sep 17 00:00:00 2001 From: Daniel Beckert Date: Wed, 15 Aug 2018 11:14:19 -0300 Subject: [PATCH 012/129] deps: backport 9a23bdd from upstream V8 Original commit message: [Isolate] Fix Isolate::PrintCurrentStackTrace for interpreted frames Previously we were getting the code object from the stack, so printed incorrect position details for interpreted frames. BUG=v8:7916 Change-Id: I2f87584117d88b7db3f3b9bdbfe793c4d3e33fe9 Reviewed-on: https://chromium-review.googlesource.com/1126313 Reviewed-by: Toon Verwaest Commit-Queue: Ross McIlroy Cr-Commit-Position: refs/heads/master@{#54253} Refs: https://github.com/v8/v8/commit/9a23bdd7ea2eba9a7a4439a7844e72fbf42bb3c4 Fixes: https://github.com/nodejs/node/issues/21988 PR-URL: https://github.com/nodejs/node/pull/22418 Refs: https://github.com/nodejs/node/pull/22338 Reviewed-By: Matheus Marchini Reviewed-By: Ujjwal Sharma Reviewed-By: Richard Lau --- deps/v8/include/v8-version.h | 2 +- deps/v8/src/isolate.cc | 14 +++++++++++--- 2 files changed, 12 insertions(+), 4 deletions(-) diff --git a/deps/v8/include/v8-version.h b/deps/v8/include/v8-version.h index 113e02667508ad..890a67c8446d40 100644 --- a/deps/v8/include/v8-version.h +++ b/deps/v8/include/v8-version.h @@ -11,7 +11,7 @@ #define V8_MAJOR_VERSION 6 #define V8_MINOR_VERSION 2 #define V8_BUILD_NUMBER 414 -#define V8_PATCH_LEVEL 66 +#define V8_PATCH_LEVEL 67 // Use 1 for candidates and 0 otherwise. // (Boolean macro values are not supported by all preprocessors.) diff --git a/deps/v8/src/isolate.cc b/deps/v8/src/isolate.cc index 5149a2650e2a10..02f7d1df64e288 100644 --- a/deps/v8/src/isolate.cc +++ b/deps/v8/src/isolate.cc @@ -1532,9 +1532,17 @@ void Isolate::PrintCurrentStackTrace(FILE* out) { Handle receiver(frame->receiver(), this); Handle function(frame->function(), this); - Handle code(AbstractCode::cast(frame->LookupCode()), this); - const int offset = - static_cast(frame->pc() - code->instruction_start()); + Handle code; + int offset; + if (frame->is_interpreted()) { + InterpretedFrame* interpreted_frame = reinterpret_cast(frame); + code = handle(AbstractCode::cast(interpreted_frame->GetBytecodeArray()), + this); + offset = interpreted_frame->GetBytecodeOffset(); + } else { + code = handle(AbstractCode::cast(frame->LookupCode()), this); + offset = static_cast(frame->pc() - code->instruction_start()); + } JSStackFrame site(this, receiver, function, code, offset); Handle line = site.ToString().ToHandleChecked(); From eaabbf4ff059573cbabf175a7eeaab145ccfc475 Mon Sep 17 00:00:00 2001 From: Rich Trott Date: Thu, 8 Mar 2018 22:27:47 -0800 Subject: [PATCH 013/129] doc: make caveat in stream.md more concise PR-URL: https://github.com/nodejs/node/pull/19251 Reviewed-By: Luigi Pinca Reviewed-By: Vse Mozhet Byt Reviewed-By: Colin Ihrig Reviewed-By: James M Snell --- doc/api/stream.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/doc/api/stream.md b/doc/api/stream.md index ec980230504656..5fa00932a5662b 100644 --- a/doc/api/stream.md +++ b/doc/api/stream.md @@ -21,10 +21,10 @@ The `stream` module can be accessed using: const stream = require('stream'); ``` -While it is important for all Node.js users to understand how streams work, -the `stream` module itself is most useful for developers that are creating new -types of stream instances. Developers who are primarily *consuming* stream -objects will rarely (if ever) have need to use the `stream` module directly. +While it is important to understand how streams work, the `stream` module itself +is most useful for developers that are creating new types of stream instances. +Developers who are primarily *consuming* stream objects will rarely need to use +the `stream` module directly. ## Organization of this Document From 534bc82578d1ea0dcae1d7ebff3f3bae1e02120e Mon Sep 17 00:00:00 2001 From: Ujjwal Sharma Date: Thu, 8 Mar 2018 02:24:08 +0530 Subject: [PATCH 014/129] test: name test files appropriately Rename the tests appropriately alongside mentioning the subsystem. Also, make a few basic changes to make sure the test conforms to the standard test structure. This renames: - test-regress-GH-1531 - test-regress-GH-2245 - test-regress-GH-3238 - test-regress-GH-3542 - test-regress-GH-3739 - test-regress-GH-4256 PR-URL: https://github.com/nodejs/node/pull/19212 Refs: https://github.com/nodejs/node/issues/19105 Refs: https://github.com/nodejs/node/blob/master/doc/guides/writing-tests.md#test-structure Reviewed-By: Rich Trott Reviewed-By: Ruben Bridgewater Reviewed-By: James M Snell --- ...ess-GH-3238.js => test-cluster-kill-disconnect.js} | 6 ++++++ ...-GH-2245.js => test-eval-strict-referenceerror.js} | 11 +++++------ ...regress-GH-3739.js => test-fs-existssync-false.js} | 9 ++++++--- ...ress-GH-3542.js => test-fs-readfilesync-enoent.js} | 5 +++++ ...-regress-GH-1531.js => test-http-request-agent.js} | 8 ++++---- ...ess-GH-4256.js => test-process-domain-segfault.js} | 5 +++++ 6 files changed, 31 insertions(+), 13 deletions(-) rename test/parallel/{test-regress-GH-3238.js => test-cluster-kill-disconnect.js} (70%) rename test/parallel/{test-regress-GH-2245.js => test-eval-strict-referenceerror.js} (62%) rename test/parallel/{test-regress-GH-3739.js => test-fs-existssync-false.js} (80%) rename test/parallel/{test-regress-GH-3542.js => test-fs-readfilesync-enoent.js} (79%) rename test/parallel/{test-regress-GH-1531.js => test-http-request-agent.js} (90%) rename test/parallel/{test-regress-GH-4256.js => test-process-domain-segfault.js} (88%) diff --git a/test/parallel/test-regress-GH-3238.js b/test/parallel/test-cluster-kill-disconnect.js similarity index 70% rename from test/parallel/test-regress-GH-3238.js rename to test/parallel/test-cluster-kill-disconnect.js index e6fe030bda9a10..53d5844e9c9c1a 100644 --- a/test/parallel/test-regress-GH-3238.js +++ b/test/parallel/test-cluster-kill-disconnect.js @@ -1,5 +1,11 @@ 'use strict'; const common = require('../common'); + +// Check that cluster works perfectly for both `kill` and `disconnect` cases. +// Also take into account that the `disconnect` event may be received after the +// `exit` event. +// https://github.com/nodejs/node/issues/3238 + const assert = require('assert'); const cluster = require('cluster'); diff --git a/test/parallel/test-regress-GH-2245.js b/test/parallel/test-eval-strict-referenceerror.js similarity index 62% rename from test/parallel/test-regress-GH-2245.js rename to test/parallel/test-eval-strict-referenceerror.js index 37260b59e34721..a96478a1bedaae 100644 --- a/test/parallel/test-regress-GH-2245.js +++ b/test/parallel/test-eval-strict-referenceerror.js @@ -1,12 +1,11 @@ /* eslint-disable strict */ require('../common'); -const assert = require('assert'); -/* -In Node.js 0.10, a bug existed that caused strict functions to not capture -their environment when evaluated. When run in 0.10 `test()` fails with a -`ReferenceError`. See https://github.com/nodejs/node/issues/2245 for details. -*/ +// In Node.js 0.10, a bug existed that caused strict functions to not capture +// their environment when evaluated. When run in 0.10 `test()` fails with a +// `ReferenceError`. See https://github.com/nodejs/node/issues/2245 for details. + +const assert = require('assert'); function test() { diff --git a/test/parallel/test-regress-GH-3739.js b/test/parallel/test-fs-existssync-false.js similarity index 80% rename from test/parallel/test-regress-GH-3739.js rename to test/parallel/test-fs-existssync-false.js index dbf77ad785cac9..de5ecfa66ad922 100644 --- a/test/parallel/test-regress-GH-3739.js +++ b/test/parallel/test-fs-existssync-false.js @@ -1,12 +1,15 @@ 'use strict'; - const common = require('../common'); +const tmpdir = require('../common/tmpdir'); + +// This test ensures that fs.existsSync doesn't incorrectly return false. +// (especially on Windows) +// https://github.com/nodejs/node-v0.x-archive/issues/3739 + const assert = require('assert'); const fs = require('fs'); const path = require('path'); -const tmpdir = require('../common/tmpdir'); - let dir = path.resolve(tmpdir.path); // Make sure that the tmp directory is clean diff --git a/test/parallel/test-regress-GH-3542.js b/test/parallel/test-fs-readfilesync-enoent.js similarity index 79% rename from test/parallel/test-regress-GH-3542.js rename to test/parallel/test-fs-readfilesync-enoent.js index b652c95c9ac881..3d421e52b120af 100644 --- a/test/parallel/test-regress-GH-3542.js +++ b/test/parallel/test-fs-readfilesync-enoent.js @@ -1,9 +1,14 @@ 'use strict'; const common = require('../common'); + // This test is only relevant on Windows. if (!common.isWindows) common.skip('Windows specific test.'); +// This test ensures fs.realpathSync works on properly on Windows without +// throwing ENOENT when the path involves a fileserver. +// https://github.com/nodejs/node-v0.x-archive/issues/3542 + const assert = require('assert'); const fs = require('fs'); const path = require('path'); diff --git a/test/parallel/test-regress-GH-1531.js b/test/parallel/test-http-request-agent.js similarity index 90% rename from test/parallel/test-regress-GH-1531.js rename to test/parallel/test-http-request-agent.js index a61cc64ab626af..a1f3077ed2820f 100644 --- a/test/parallel/test-regress-GH-1531.js +++ b/test/parallel/test-http-request-agent.js @@ -1,15 +1,15 @@ 'use strict'; const common = require('../common'); -// This test ensures that a http request callback is called -// when the agent option is set -// See https://github.com/nodejs/node-v0.x-archive/issues/1531 - if (!common.hasCrypto) common.skip('missing crypto'); const fixtures = require('../common/fixtures'); +// This test ensures that a http request callback is called when the agent +// option is set. +// See https://github.com/nodejs/node-v0.x-archive/issues/1531 + const https = require('https'); const options = { diff --git a/test/parallel/test-regress-GH-4256.js b/test/parallel/test-process-domain-segfault.js similarity index 88% rename from test/parallel/test-regress-GH-4256.js rename to test/parallel/test-process-domain-segfault.js index 6a4a4467b4fd81..78009f4687d8dc 100644 --- a/test/parallel/test-regress-GH-4256.js +++ b/test/parallel/test-process-domain-segfault.js @@ -21,6 +21,11 @@ 'use strict'; require('../common'); + +// This test ensures that setting `process.domain` to `null` does not result in +// node crashing with a segfault. +// https://github.com/nodejs/node-v0.x-archive/issues/4256 + process.domain = null; setTimeout(function() { console.log('this console.log statement should not make node crash'); From 48f31bdf206c18232beef5ae3e6777d071f44b3c Mon Sep 17 00:00:00 2001 From: Peter Marshall Date: Tue, 26 Jun 2018 18:37:43 +0200 Subject: [PATCH 015/129] deps: V8: backport 20 CPU profiler commits from upstream [cpu-profiler] Fix bugs and add tests for JITLineInfoTable https://chromium.googlesource.com/v8/v8/+/4feb5ce7fd5ef8c933f3f5dff2eca1173f85c1e9 [cpu-profiler] Fix incorrect line number calculation. https://chromium.googlesource.com/v8/v8/+/ddb2856f39632f9e9f623d3cdb4600e636172031 [cpu-profiler] Use std::unordered_map for hashmaps. https://chromium.googlesource.com/v8/v8/+/35985ce6abc80b85264fe3b87b246fed5f1806e6 [cpu-profiler] Do not store CodeEntries between profiling sessions. https://chromium.googlesource.com/v8/v8.git/+/8ec48b2117b8092c4956f1ee11a0c85bec3ba1f8 [cpu-profiler] Remove name_prefix field from CodeEntry https://chromium.googlesource.com/v8/v8.git/+/6f72af25fe43218b60c68129073ddcddb631566e [cpu-profiler] Extract rare used fields of CodeEntry to an optional object. https://chromium.googlesource.com/v8/v8.git/+/fcc1ebb55aab38013855834f556f6e874e0eb8b3 [profiler] Refactoring: decouple StringsStorage from Heap object. https://chromium.googlesource.com/v8/v8/+/a31320f59c911a277566d6c2fa0b0f2ac83e0748 [cpu-profiler] Add a HandleScope to limit memory consumption. https://chromium.googlesource.com/v8/v8.git/+/3e9f8a4f635e2d946651d6a4df81378266f32dc9 [cpu-profiler] Lazily create CPU profiler. https://chromium.googlesource.com/v8/v8/+/1426ea1d6d45be0b4d9476bdb5bf3f27cfe578a0 [cpu-profiler] turn several std::map's into unordered_map's. https://chromium.googlesource.com/v8/v8/+/3ed5dfb8a3cbc7aa0017bd01c2fdd6227485b8ad [cpu-profiler] Eagerly delete not used CodeEntry'es https://chromium.googlesource.com/v8/v8.git/+/c6c28f7a412a88df12055e953630a9e93cc64d49 [cpu-profiler] Move bailout reason into rare_info struct https://chromium.googlesource.com/v8/v8.git/+/29ea4d1ef5360e71c61ecf8db6a5a0a0c3391fd1 [cpu-profiler] Save space in the SourcePositionTable by using a vector. https://chromium.googlesource.com/v8/v8.git/+/1cb19f0e0a93adbac8c11bc906f951bd8098722d [cpu-profiler] Only store deopt inline frames for functions that need it https://chromium.googlesource.com/v8/v8.git/+/0bfcbdd4726920755e51dab28c18ab93e050819b [cpu-profiler] Add a new profiling mode with a more detailed call tree. https://chromium.googlesource.com/v8/v8.git/+/ecae80cdb350dde1e654c531b56f5b6c44dc8c77 [cpu-profiler] Reuse free slots in code_entries_ https://chromium.googlesource.com/v8/v8.git/+/3e1126bf15e62c433c4e9cb21316d182f691c63a [cpu-profiler] Use instruction start as the key for the CodeMap https://chromium.googlesource.com/v8/v8.git/+/ba752ea4c50713dff1e94f45a79db3ba968a8d66 [cpu-profiler] Add flag to always generate accurate line info. https://chromium.googlesource.com/v8/v8/+/56baf56790de439b3f69e887e94beb3b301ed77c [cpu-profiler] Turn on detailed line info for optimized code https://chromium.googlesource.com/v8/v8/+/84894ce6d2af7feb9e1f5574409355120887326c [cpu-profiler] Separate the flags for generating extra line information https://chromium.googlesource.com/v8/v8/+/30ff6719db441cc7ef220d449970cc169067e256 Backport-PR-URL: https://github.com/nodejs/node/pull/21558 PR-URL: https://github.com/nodejs/node/pull/21558 Reviewed-By: Matteo Collina Reviewed-By: Myles Borins Reviewed-By: Yang Guo Reviewed-By: Rod Vagg --- deps/v8/include/v8-profiler.h | 17 + deps/v8/include/v8-version.h | 2 +- deps/v8/src/api.cc | 21 +- deps/v8/src/code-events.h | 4 +- deps/v8/src/compilation-info.cc | 2 +- deps/v8/src/flag-definitions.h | 3 + deps/v8/src/heap/mark-compact.cc | 2 +- deps/v8/src/isolate.cc | 12 +- deps/v8/src/isolate.h | 4 +- deps/v8/src/log.cc | 48 +-- deps/v8/src/log.h | 14 +- deps/v8/src/perf-jit.cc | 2 +- deps/v8/src/perf-jit.h | 4 +- deps/v8/src/profiler/cpu-profiler-inl.h | 18 +- deps/v8/src/profiler/cpu-profiler.cc | 50 +-- deps/v8/src/profiler/cpu-profiler.h | 30 +- deps/v8/src/profiler/heap-profiler.cc | 4 +- deps/v8/src/profiler/profile-generator-inl.h | 26 +- deps/v8/src/profiler/profile-generator.cc | 367 +++++++++++------- deps/v8/src/profiler/profile-generator.h | 218 +++++++---- deps/v8/src/profiler/profiler-listener.cc | 176 ++++----- deps/v8/src/profiler/profiler-listener.h | 32 +- deps/v8/src/profiler/strings-storage.cc | 32 +- deps/v8/src/profiler/strings-storage.h | 6 +- deps/v8/src/snapshot/serializer.h | 4 +- deps/v8/test/cctest/cctest.status | 3 + deps/v8/test/cctest/test-cpu-profiler.cc | 196 +++++++--- deps/v8/test/cctest/test-log.cc | 2 +- deps/v8/test/cctest/test-profile-generator.cc | 115 ++++-- 29 files changed, 852 insertions(+), 562 deletions(-) diff --git a/deps/v8/include/v8-profiler.h b/deps/v8/include/v8-profiler.h index 2363f0514778f0..ba81a8b7d165a1 100644 --- a/deps/v8/include/v8-profiler.h +++ b/deps/v8/include/v8-profiler.h @@ -273,6 +273,16 @@ class V8_EXPORT CpuProfile { void Delete(); }; +enum CpuProfilingMode { + // In the resulting CpuProfile tree, intermediate nodes in a stack trace + // (from the root to a leaf) will have line numbers that point to the start + // line of the function, rather than the line of the callsite of the child. + kLeafNodeLineNumbers, + // In the resulting CpuProfile tree, nodes are separated based on the line + // number of their callsite in their parent. + kCallerLineNumbers, +}; + /** * Interface for controlling CPU profiling. Instance of the * profiler can be created using v8::CpuProfiler::New method. @@ -309,6 +319,13 @@ class V8_EXPORT CpuProfiler { * |record_samples| parameter controls whether individual samples should * be recorded in addition to the aggregated tree. */ + void StartProfiling(Local title, CpuProfilingMode mode, + bool record_samples = false); + /** + * The same as StartProfiling above, but the CpuProfilingMode defaults to + * kLeafNodeLineNumbers mode, which was the previous default behavior of the + * profiler. + */ void StartProfiling(Local title, bool record_samples = false); /** diff --git a/deps/v8/include/v8-version.h b/deps/v8/include/v8-version.h index 890a67c8446d40..f2531abb21f0aa 100644 --- a/deps/v8/include/v8-version.h +++ b/deps/v8/include/v8-version.h @@ -11,7 +11,7 @@ #define V8_MAJOR_VERSION 6 #define V8_MINOR_VERSION 2 #define V8_BUILD_NUMBER 414 -#define V8_PATCH_LEVEL 67 +#define V8_PATCH_LEVEL 68 // Use 1 for candidates and 0 otherwise. // (Boolean macro values are not supported by all preprocessors.) diff --git a/deps/v8/src/api.cc b/deps/v8/src/api.cc index b22c75704e0f7f..bc5848e939420a 100644 --- a/deps/v8/src/api.cc +++ b/deps/v8/src/api.cc @@ -8418,7 +8418,7 @@ HeapProfiler* Isolate::GetHeapProfiler() { CpuProfiler* Isolate::GetCpuProfiler() { i::CpuProfiler* cpu_profiler = - reinterpret_cast(this)->cpu_profiler(); + reinterpret_cast(this)->EnsureCpuProfiler(); return reinterpret_cast(cpu_profiler); } @@ -10138,15 +10138,7 @@ Local CpuProfileNode::GetFunctionName() const { const i::CodeEntry* entry = node->entry(); i::Handle name = isolate->factory()->InternalizeUtf8String(entry->name()); - if (!entry->has_name_prefix()) { - return ToApiHandle(name); - } else { - // We do not expect this to fail. Change this if it does. - i::Handle cons = isolate->factory()->NewConsString( - isolate->factory()->InternalizeUtf8String(entry->name_prefix()), - name).ToHandleChecked(); - return ToApiHandle(cons); - } + return ToApiHandle(name); } int debug::Coverage::BlockData::StartOffset() const { return block_->start; } @@ -10237,7 +10229,7 @@ const char* CpuProfileNode::GetScriptResourceNameStr() const { } int CpuProfileNode::GetLineNumber() const { - return reinterpret_cast(this)->entry()->line_number(); + return reinterpret_cast(this)->line_number(); } @@ -10370,9 +10362,14 @@ void CpuProfiler::CollectSample() { void CpuProfiler::StartProfiling(Local title, bool record_samples) { reinterpret_cast(this)->StartProfiling( - *Utils::OpenHandle(*title), record_samples); + *Utils::OpenHandle(*title), record_samples, kLeafNodeLineNumbers); } +void CpuProfiler::StartProfiling(Local title, CpuProfilingMode mode, + bool record_samples) { + reinterpret_cast(this)->StartProfiling( + *Utils::OpenHandle(*title), record_samples, mode); +} CpuProfile* CpuProfiler::StopProfiling(Local title) { return reinterpret_cast( diff --git a/deps/v8/src/code-events.h b/deps/v8/src/code-events.h index ca92e2b5e4e624..f26d6327b5cd0d 100644 --- a/deps/v8/src/code-events.h +++ b/deps/v8/src/code-events.h @@ -101,7 +101,7 @@ class CodeEventListener { virtual void GetterCallbackEvent(Name* name, Address entry_point) = 0; virtual void SetterCallbackEvent(Name* name, Address entry_point) = 0; virtual void RegExpCodeCreateEvent(AbstractCode* code, String* source) = 0; - virtual void CodeMoveEvent(AbstractCode* from, Address to) = 0; + virtual void CodeMoveEvent(AbstractCode* from, AbstractCode* to) = 0; virtual void SharedFunctionInfoMoveEvent(Address from, Address to) = 0; virtual void CodeMovingGCEvent() = 0; virtual void CodeDisableOptEvent(AbstractCode* code, @@ -163,7 +163,7 @@ class CodeEventDispatcher { void RegExpCodeCreateEvent(AbstractCode* code, String* source) { CODE_EVENT_DISPATCH(RegExpCodeCreateEvent(code, source)); } - void CodeMoveEvent(AbstractCode* from, Address to) { + void CodeMoveEvent(AbstractCode* from, AbstractCode* to) { CODE_EVENT_DISPATCH(CodeMoveEvent(from, to)); } void SharedFunctionInfoMoveEvent(Address from, Address to) { diff --git a/deps/v8/src/compilation-info.cc b/deps/v8/src/compilation-info.cc index f4566d29bdb328..b639def60f2766 100644 --- a/deps/v8/src/compilation-info.cc +++ b/deps/v8/src/compilation-info.cc @@ -51,7 +51,7 @@ CompilationInfo::CompilationInfo(Zone* zone, Isolate* isolate, // Collect source positions for optimized code when profiling or if debugger // is active, to be able to get more precise source positions at the price of // more memory consumption. - if (isolate_->NeedsSourcePositionsForProfiling()) { + if (isolate_->NeedsDetailedOptimizedCodeLineInfo()) { MarkAsSourcePositionsEnabled(); } } diff --git a/deps/v8/src/flag-definitions.h b/deps/v8/src/flag-definitions.h index bcb5a2c982b5a7..7fc25bcc91b66e 100644 --- a/deps/v8/src/flag-definitions.h +++ b/deps/v8/src/flag-definitions.h @@ -1082,6 +1082,9 @@ DEFINE_BOOL(log_source_code, false, "Log source code.") DEFINE_BOOL(prof, false, "Log statistical profiling information (implies --log-code).") +DEFINE_BOOL(detailed_line_info, true, + "Always generate detailed line information for CPU profiling.") + #if defined(ANDROID) // Phones and tablets have processors that are much slower than desktop // and laptop computers for which current heuristics are tuned. diff --git a/deps/v8/src/heap/mark-compact.cc b/deps/v8/src/heap/mark-compact.cc index 194415e94979de..945c95e13622f5 100644 --- a/deps/v8/src/heap/mark-compact.cc +++ b/deps/v8/src/heap/mark-compact.cc @@ -1484,7 +1484,7 @@ class ProfilingMigrationObserver final : public MigrationObserver { int size) final { if (dest == CODE_SPACE || (dest == OLD_SPACE && dst->IsBytecodeArray())) { PROFILE(heap_->isolate(), - CodeMoveEvent(AbstractCode::cast(src), dst->address())); + CodeMoveEvent(AbstractCode::cast(src), AbstractCode::cast(dst))); } heap_->OnMoveEvent(dst, src, size); } diff --git a/deps/v8/src/isolate.cc b/deps/v8/src/isolate.cc index 02f7d1df64e288..bccd0cb0b880e3 100644 --- a/deps/v8/src/isolate.cc +++ b/deps/v8/src/isolate.cc @@ -2697,7 +2697,6 @@ bool Isolate::Init(StartupDeserializer* des) { call_descriptor_data_ = new CallInterfaceDescriptorData[CallDescriptors::NUMBER_OF_DESCRIPTORS]; access_compiler_data_ = new AccessCompilerData(); - cpu_profiler_ = new CpuProfiler(this); heap_profiler_ = new HeapProfiler(heap()); interpreter_ = new interpreter::Interpreter(this); compiler_dispatcher_ = @@ -2970,6 +2969,10 @@ bool Isolate::use_optimizer() { !is_precise_count_code_coverage() && !is_block_count_code_coverage(); } +bool Isolate::NeedsDetailedOptimizedCodeLineInfo() const { + return NeedsSourcePositionsForProfiling() || FLAG_detailed_line_info; +} + bool Isolate::NeedsSourcePositionsForProfiling() const { return FLAG_trace_deopt || FLAG_trace_turbo || FLAG_trace_turbo_graph || FLAG_turbo_profiling || FLAG_perf_prof || is_profiling() || @@ -3694,6 +3697,13 @@ void Isolate::PrintWithTimestamp(const char* format, ...) { va_end(arguments); } +CpuProfiler* Isolate::EnsureCpuProfiler() { + if (!cpu_profiler_) { + cpu_profiler_ = new CpuProfiler(this); + } + return cpu_profiler_; +} + bool StackLimitCheck::JsHasOverflowed(uintptr_t gap) const { StackGuard* stack_guard = isolate_->stack_guard(); #ifdef USE_SIMULATOR diff --git a/deps/v8/src/isolate.h b/deps/v8/src/isolate.h index c3a0727e84a353..a98b6bc0eca4e5 100644 --- a/deps/v8/src/isolate.h +++ b/deps/v8/src/isolate.h @@ -1019,6 +1019,8 @@ class Isolate { bool NeedsSourcePositionsForProfiling() const; + bool NeedsDetailedOptimizedCodeLineInfo() const; + bool is_best_effort_code_coverage() const { return code_coverage_mode() == debug::Coverage::kBestEffort; } @@ -1449,7 +1451,7 @@ class Isolate { // TODO(alph): Remove along with the deprecated GetCpuProfiler(). friend v8::CpuProfiler* v8::Isolate::GetCpuProfiler(); - CpuProfiler* cpu_profiler() const { return cpu_profiler_; } + CpuProfiler* EnsureCpuProfiler(); base::Atomic32 id_; EntryStackItem* entry_stack_; diff --git a/deps/v8/src/log.cc b/deps/v8/src/log.cc index 0ba024b987189e..591338da9aacb2 100644 --- a/deps/v8/src/log.cc +++ b/deps/v8/src/log.cc @@ -22,7 +22,6 @@ #include "src/log-utils.h" #include "src/macro-assembler.h" #include "src/perf-jit.h" -#include "src/profiler/profiler-listener.h" #include "src/profiler/tick-sample.h" #include "src/runtime-profiler.h" #include "src/source-position-table.h" @@ -218,7 +217,7 @@ class PerfBasicLogger : public CodeEventLogger { PerfBasicLogger(); ~PerfBasicLogger() override; - void CodeMoveEvent(AbstractCode* from, Address to) override {} + void CodeMoveEvent(AbstractCode* from, AbstractCode* to) override {} void CodeDisableOptEvent(AbstractCode* code, SharedFunctionInfo* shared) override {} @@ -287,7 +286,7 @@ class LowLevelLogger : public CodeEventLogger { explicit LowLevelLogger(const char* file_name); ~LowLevelLogger() override; - void CodeMoveEvent(AbstractCode* from, Address to) override; + void CodeMoveEvent(AbstractCode* from, AbstractCode* to) override; void CodeDisableOptEvent(AbstractCode* code, SharedFunctionInfo* shared) override {} void SnapshotPositionEvent(HeapObject* obj, int pos); @@ -393,11 +392,10 @@ void LowLevelLogger::LogRecordedBuffer(AbstractCode* code, SharedFunctionInfo*, code->instruction_size()); } -void LowLevelLogger::CodeMoveEvent(AbstractCode* from, Address to) { +void LowLevelLogger::CodeMoveEvent(AbstractCode* from, AbstractCode* to) { CodeMoveStruct event; event.from_address = from->instruction_start(); - size_t header_size = from->instruction_start() - from->address(); - event.to_address = to + header_size; + event.to_address = to->instruction_start(); LogWriteStruct(event); } @@ -419,7 +417,7 @@ class JitLogger : public CodeEventLogger { public: explicit JitLogger(JitCodeEventHandler code_event_handler); - void CodeMoveEvent(AbstractCode* from, Address to) override; + void CodeMoveEvent(AbstractCode* from, AbstractCode* to) override; void CodeDisableOptEvent(AbstractCode* code, SharedFunctionInfo* shared) override {} void AddCodeLinePosInfoEvent(void* jit_handler_data, int pc_offset, @@ -460,19 +458,14 @@ void JitLogger::LogRecordedBuffer(AbstractCode* code, code_event_handler_(&event); } -void JitLogger::CodeMoveEvent(AbstractCode* from, Address to) { +void JitLogger::CodeMoveEvent(AbstractCode* from, AbstractCode* to) { base::LockGuard guard(&logger_mutex_); JitCodeEvent event; event.type = JitCodeEvent::CODE_MOVED; - event.code_start = from->instruction_start(); + event.code_start = reinterpret_cast(from->instruction_start()); event.code_len = from->instruction_size(); - - // Calculate the header size. - const size_t header_size = from->instruction_start() - from->address(); - - // Calculate the new start address of the instructions. - event.new_code_start = to + header_size; + event.new_code_start = reinterpret_cast(to->instruction_start()); code_event_handler_(&event); } @@ -739,7 +732,6 @@ Logger::Logger(Isolate* isolate) perf_jit_logger_(NULL), ll_logger_(NULL), jit_logger_(NULL), - listeners_(5), is_initialized_(false) {} Logger::~Logger() { @@ -1297,9 +1289,10 @@ void Logger::RegExpCodeCreateEvent(AbstractCode* code, String* source) { msg.WriteToLogFile(); } -void Logger::CodeMoveEvent(AbstractCode* from, Address to) { +void Logger::CodeMoveEvent(AbstractCode* from, AbstractCode* to) { if (!is_logging_code_events()) return; - MoveEventInternal(CodeEventListener::CODE_MOVE_EVENT, from->address(), to); + MoveEventInternal(CodeEventListener::CODE_MOVE_EVENT, from->address(), + to->address()); } void Logger::CodeLinePosInfoRecordEvent(AbstractCode* code, @@ -1876,8 +1869,6 @@ bool Logger::SetUp(Isolate* isolate) { profiler_->Engage(); } - profiler_listener_.reset(); - if (is_logging_) { addCodeEventListener(this); } @@ -1905,19 +1896,6 @@ void Logger::SetCodeEventHandler(uint32_t options, } } -void Logger::SetUpProfilerListener() { - if (!is_initialized_) return; - if (profiler_listener_.get() == nullptr) { - profiler_listener_.reset(new ProfilerListener(isolate_)); - } - addCodeEventListener(profiler_listener_.get()); -} - -void Logger::TearDownProfilerListener() { - if (profiler_listener_->HasObservers()) return; - removeCodeEventListener(profiler_listener_.get()); -} - sampler::Sampler* Logger::sampler() { return ticker_; } @@ -1961,10 +1939,6 @@ FILE* Logger::TearDown() { jit_logger_ = NULL; } - if (profiler_listener_.get() != nullptr) { - removeCodeEventListener(profiler_listener_.get()); - } - return log_->Close(); } diff --git a/deps/v8/src/log.h b/deps/v8/src/log.h index 3e4d385527b6d0..931227a1453c2e 100644 --- a/deps/v8/src/log.h +++ b/deps/v8/src/log.h @@ -74,7 +74,6 @@ class LowLevelLogger; class PerfBasicLogger; class PerfJitLogger; class Profiler; -class ProfilerListener; class RuntimeCallTimer; class Ticker; @@ -102,16 +101,8 @@ class Logger : public CodeEventListener { void SetCodeEventHandler(uint32_t options, JitCodeEventHandler event_handler); - // Sets up ProfilerListener. - void SetUpProfilerListener(); - - // Tear down ProfilerListener if it has no observers. - void TearDownProfilerListener(); - sampler::Sampler* sampler(); - ProfilerListener* profiler_listener() { return profiler_listener_.get(); } - // Frees resources acquired in SetUp. // When a temporary file is used for the log, returns its stream descriptor, // leaving the file open. @@ -177,7 +168,7 @@ class Logger : public CodeEventListener { // Emits a code create event for a RegExp. void RegExpCodeCreateEvent(AbstractCode* code, String* source); // Emits a code move event. - void CodeMoveEvent(AbstractCode* from, Address to); + void CodeMoveEvent(AbstractCode* from, AbstractCode* to); // Emits a code line info record event. void CodeLinePosInfoRecordEvent(AbstractCode* code, ByteArray* source_position_table); @@ -316,8 +307,6 @@ class Logger : public CodeEventListener { PerfJitLogger* perf_jit_logger_; LowLevelLogger* ll_logger_; JitLogger* jit_logger_; - std::unique_ptr profiler_listener_; - List listeners_; std::set logged_source_code_; uint32_t next_source_info_id_ = 0; @@ -400,7 +389,6 @@ class CodeEventLogger : public CodeEventListener { NameBuffer* name_buffer_; }; - } // namespace internal } // namespace v8 diff --git a/deps/v8/src/perf-jit.cc b/deps/v8/src/perf-jit.cc index 46597a968588c4..5ca1c2ba917a49 100644 --- a/deps/v8/src/perf-jit.cc +++ b/deps/v8/src/perf-jit.cc @@ -376,7 +376,7 @@ void PerfJitLogger::LogWriteUnwindingInfo(Code* code) { LogWriteBytes(padding_bytes, static_cast(padding_size)); } -void PerfJitLogger::CodeMoveEvent(AbstractCode* from, Address to) { +void PerfJitLogger::CodeMoveEvent(AbstractCode* from, AbstractCode* to) { // Code relocation not supported. UNREACHABLE(); } diff --git a/deps/v8/src/perf-jit.h b/deps/v8/src/perf-jit.h index 2b0b4831e0c43d..f68595f29ea6ff 100644 --- a/deps/v8/src/perf-jit.h +++ b/deps/v8/src/perf-jit.h @@ -41,7 +41,7 @@ class PerfJitLogger : public CodeEventLogger { PerfJitLogger(); virtual ~PerfJitLogger(); - void CodeMoveEvent(AbstractCode* from, Address to) override; + void CodeMoveEvent(AbstractCode* from, AbstractCode* to) override; void CodeDisableOptEvent(AbstractCode* code, SharedFunctionInfo* shared) override {} @@ -113,7 +113,7 @@ class PerfJitLogger : public CodeEventLogger { // PerfJitLogger is only implemented on Linux class PerfJitLogger : public CodeEventLogger { public: - void CodeMoveEvent(AbstractCode* from, Address to) override { + void CodeMoveEvent(AbstractCode* from, AbstractCode* to) override { UNIMPLEMENTED(); } diff --git a/deps/v8/src/profiler/cpu-profiler-inl.h b/deps/v8/src/profiler/cpu-profiler-inl.h index 440c6a1cce189b..2beb6f5aab2a94 100644 --- a/deps/v8/src/profiler/cpu-profiler-inl.h +++ b/deps/v8/src/profiler/cpu-profiler-inl.h @@ -16,31 +16,35 @@ namespace v8 { namespace internal { void CodeCreateEventRecord::UpdateCodeMap(CodeMap* code_map) { - code_map->AddCode(start, entry, size); + code_map->AddCode(instruction_start, entry, instruction_size); } void CodeMoveEventRecord::UpdateCodeMap(CodeMap* code_map) { - code_map->MoveCode(from, to); + code_map->MoveCode(from_instruction_start, to_instruction_start); } void CodeDisableOptEventRecord::UpdateCodeMap(CodeMap* code_map) { - CodeEntry* entry = code_map->FindEntry(start); - if (entry != NULL) { + CodeEntry* entry = code_map->FindEntry(instruction_start); + if (entry != nullptr) { entry->set_bailout_reason(bailout_reason); } } void CodeDeoptEventRecord::UpdateCodeMap(CodeMap* code_map) { - CodeEntry* entry = code_map->FindEntry(start); - if (entry != NULL) entry->set_deopt_info(deopt_reason, deopt_id); + CodeEntry* entry = code_map->FindEntry(instruction_start); + if (entry == nullptr) return; + std::vector frames_vector( + deopt_frames, deopt_frames + deopt_frame_count); + entry->set_deopt_info(deopt_reason, deopt_id, std::move(frames_vector)); + delete[] deopt_frames; } void ReportBuiltinEventRecord::UpdateCodeMap(CodeMap* code_map) { - CodeEntry* entry = code_map->FindEntry(start); + CodeEntry* entry = code_map->FindEntry(instruction_start); if (!entry) { // Code objects for builtins should already have been added to the map but // some of them have been filtered out by CpuProfiler. diff --git a/deps/v8/src/profiler/cpu-profiler.cc b/deps/v8/src/profiler/cpu-profiler.cc index 80d488f12c527f..69021cee344eb2 100644 --- a/deps/v8/src/profiler/cpu-profiler.cc +++ b/deps/v8/src/profiler/cpu-profiler.cc @@ -4,6 +4,12 @@ #include "src/profiler/cpu-profiler.h" +#include +#include + +#include "src/base/lazy-instance.h" +#include "src/base/platform/mutex.h" +#include "src/base/template-utils.h" #include "src/debug/debug.h" #include "src/deoptimizer.h" #include "src/frames-inl.h" @@ -275,20 +281,19 @@ void CpuProfiler::set_sampling_interval(base::TimeDelta value) { void CpuProfiler::ResetProfiles() { profiles_.reset(new CpuProfilesCollection(isolate_)); profiles_->set_cpu_profiler(this); + profiler_listener_.reset(); + generator_.reset(); } void CpuProfiler::CreateEntriesForRuntimeCallStats() { - static_entries_.clear(); RuntimeCallStats* rcs = isolate_->counters()->runtime_call_stats(); CodeMap* code_map = generator_->code_map(); for (int i = 0; i < RuntimeCallStats::counters_count; ++i) { RuntimeCallCounter* counter = &(rcs->*(RuntimeCallStats::counters[i])); DCHECK(counter->name()); - std::unique_ptr entry( - new CodeEntry(CodeEventListener::FUNCTION_TAG, counter->name(), - CodeEntry::kEmptyNamePrefix, "native V8Runtime")); - code_map->AddCode(reinterpret_cast
(counter), entry.get(), 1); - static_entries_.push_back(std::move(entry)); + auto entry = new CodeEntry(CodeEventListener::FUNCTION_TAG, counter->name(), + "native V8Runtime"); + code_map->AddCode(reinterpret_cast
(counter), entry, 1); } } @@ -298,20 +303,20 @@ void CpuProfiler::CollectSample() { } } -void CpuProfiler::StartProfiling(const char* title, bool record_samples) { - if (profiles_->StartProfiling(title, record_samples)) { +void CpuProfiler::StartProfiling(const char* title, bool record_samples, + ProfilingMode mode) { + if (profiles_->StartProfiling(title, record_samples, mode)) { TRACE_EVENT0("v8", "CpuProfiler::StartProfiling"); StartProcessorIfNotStarted(); } } - -void CpuProfiler::StartProfiling(String* title, bool record_samples) { - StartProfiling(profiles_->GetName(title), record_samples); +void CpuProfiler::StartProfiling(String* title, bool record_samples, + ProfilingMode mode) { + StartProfiling(profiles_->GetName(title), record_samples, mode); isolate_->debug()->feature_tracker()->Track(DebugFeatureTracker::kProfiler); } - void CpuProfiler::StartProcessorIfNotStarted() { if (processor_) { processor_->AddCurrentStack(isolate_); @@ -321,13 +326,17 @@ void CpuProfiler::StartProcessorIfNotStarted() { // Disable logging when using the new implementation. saved_is_logging_ = logger->is_logging_; logger->is_logging_ = false; - generator_.reset(new ProfileGenerator(profiles_.get())); + if (!generator_) { + generator_.reset(new ProfileGenerator(profiles_.get())); + CreateEntriesForRuntimeCallStats(); + } processor_.reset(new ProfilerEventsProcessor(isolate_, generator_.get(), sampling_interval_)); - CreateEntriesForRuntimeCallStats(); - logger->SetUpProfilerListener(); - ProfilerListener* profiler_listener = logger->profiler_listener(); - profiler_listener->AddObserver(this); + if (!profiler_listener_) { + profiler_listener_.reset(new ProfilerListener(isolate_, this)); + } + logger->addCodeEventListener(profiler_listener_.get()); + is_profiling_ = true; isolate_->set_is_profiling(true); // Enumerate stuff we already have in the heap. @@ -362,12 +371,9 @@ void CpuProfiler::StopProcessor() { Logger* logger = isolate_->logger(); is_profiling_ = false; isolate_->set_is_profiling(false); - ProfilerListener* profiler_listener = logger->profiler_listener(); - profiler_listener->RemoveObserver(this); + logger->removeCodeEventListener(profiler_listener_.get()); processor_->StopSynchronously(); - logger->TearDownProfilerListener(); processor_.reset(); - generator_.reset(); logger->is_logging_ = saved_is_logging_; } @@ -379,7 +385,7 @@ void CpuProfiler::LogBuiltins() { CodeEventsContainer evt_rec(CodeEventRecord::REPORT_BUILTIN); ReportBuiltinEventRecord* rec = &evt_rec.ReportBuiltinEventRecord_; Builtins::Name id = static_cast(i); - rec->start = builtins->builtin(id)->address(); + rec->instruction_start = builtins->builtin(id)->instruction_start(); rec->builtin_id = id; processor_->Enqueue(evt_rec); } diff --git a/deps/v8/src/profiler/cpu-profiler.h b/deps/v8/src/profiler/cpu-profiler.h index 5fd7fa14da1cab..2672c1b8961cf2 100644 --- a/deps/v8/src/profiler/cpu-profiler.h +++ b/deps/v8/src/profiler/cpu-profiler.h @@ -53,9 +53,9 @@ class CodeEventRecord { class CodeCreateEventRecord : public CodeEventRecord { public: - Address start; + Address instruction_start; CodeEntry* entry; - unsigned size; + unsigned instruction_size; INLINE(void UpdateCodeMap(CodeMap* code_map)); }; @@ -63,8 +63,8 @@ class CodeCreateEventRecord : public CodeEventRecord { class CodeMoveEventRecord : public CodeEventRecord { public: - Address from; - Address to; + Address from_instruction_start; + Address to_instruction_start; INLINE(void UpdateCodeMap(CodeMap* code_map)); }; @@ -72,7 +72,7 @@ class CodeMoveEventRecord : public CodeEventRecord { class CodeDisableOptEventRecord : public CodeEventRecord { public: - Address start; + Address instruction_start; const char* bailout_reason; INLINE(void UpdateCodeMap(CodeMap* code_map)); @@ -81,11 +81,13 @@ class CodeDisableOptEventRecord : public CodeEventRecord { class CodeDeoptEventRecord : public CodeEventRecord { public: - Address start; + Address instruction_start; const char* deopt_reason; int deopt_id; void* pc; int fp_to_sp_delta; + CpuProfileDeoptFrame* deopt_frames; + int deopt_frame_count; INLINE(void UpdateCodeMap(CodeMap* code_map)); }; @@ -93,7 +95,7 @@ class CodeDeoptEventRecord : public CodeEventRecord { class ReportBuiltinEventRecord : public CodeEventRecord { public: - Address start; + Address instruction_start; Builtins::Name builtin_id; INLINE(void UpdateCodeMap(CodeMap* code_map)); @@ -195,10 +197,13 @@ class CpuProfiler : public CodeEventObserver { ~CpuProfiler() override; + typedef v8::CpuProfilingMode ProfilingMode; + void set_sampling_interval(base::TimeDelta value); void CollectSample(); - void StartProfiling(const char* title, bool record_samples = false); - void StartProfiling(String* title, bool record_samples); + void StartProfiling(const char* title, bool record_samples = false, + ProfilingMode mode = ProfilingMode::kLeafNodeLineNumbers); + void StartProfiling(String* title, bool record_samples, ProfilingMode mode); CpuProfile* StopProfiling(const char* title); CpuProfile* StopProfiling(String* title); int GetProfilesCount(); @@ -214,6 +219,10 @@ class CpuProfiler : public CodeEventObserver { ProfilerEventsProcessor* processor() const { return processor_.get(); } Isolate* isolate() const { return isolate_; } + ProfilerListener* profiler_listener_for_test() { + return profiler_listener_.get(); + } + private: void StartProcessorIfNotStarted(); void StopProcessorIfLastProfile(const char* title); @@ -227,7 +236,7 @@ class CpuProfiler : public CodeEventObserver { std::unique_ptr profiles_; std::unique_ptr generator_; std::unique_ptr processor_; - std::vector> static_entries_; + std::unique_ptr profiler_listener_; bool saved_is_logging_; bool is_profiling_; @@ -237,5 +246,4 @@ class CpuProfiler : public CodeEventObserver { } // namespace internal } // namespace v8 - #endif // V8_PROFILER_CPU_PROFILER_H_ diff --git a/deps/v8/src/profiler/heap-profiler.cc b/deps/v8/src/profiler/heap-profiler.cc index 4706b914e7ac40..f587adda497daf 100644 --- a/deps/v8/src/profiler/heap-profiler.cc +++ b/deps/v8/src/profiler/heap-profiler.cc @@ -16,7 +16,7 @@ namespace internal { HeapProfiler::HeapProfiler(Heap* heap) : ids_(new HeapObjectsMap(heap)), - names_(new StringsStorage(heap)), + names_(new StringsStorage(heap->HashSeed())), is_tracking_object_moves_(false), get_retainer_infos_callback_(nullptr) {} @@ -34,7 +34,7 @@ HeapProfiler::~HeapProfiler() { void HeapProfiler::DeleteAllSnapshots() { snapshots_.Iterate(DeleteHeapSnapshot); snapshots_.Clear(); - names_.reset(new StringsStorage(heap())); + names_.reset(new StringsStorage(heap()->HashSeed())); } diff --git a/deps/v8/src/profiler/profile-generator-inl.h b/deps/v8/src/profiler/profile-generator-inl.h index 5a7017ad490767..31652ba9f98e83 100644 --- a/deps/v8/src/profiler/profile-generator-inl.h +++ b/deps/v8/src/profiler/profile-generator-inl.h @@ -11,33 +11,35 @@ namespace v8 { namespace internal { CodeEntry::CodeEntry(CodeEventListener::LogEventsAndTags tag, const char* name, - const char* name_prefix, const char* resource_name, - int line_number, int column_number, - JITLineInfoTable* line_info, Address instruction_start) + const char* resource_name, int line_number, + int column_number, + std::unique_ptr line_info, + Address instruction_start) : bit_field_(TagField::encode(tag) | BuiltinIdField::encode(Builtins::builtin_count)), - name_prefix_(name_prefix), name_(name), resource_name_(resource_name), line_number_(line_number), column_number_(column_number), script_id_(v8::UnboundScript::kNoScriptId), position_(0), - bailout_reason_(kEmptyBailoutReason), - deopt_reason_(kNoDeoptReason), - deopt_id_(kNoDeoptimizationId), - line_info_(line_info), + line_info_(std::move(line_info)), instruction_start_(instruction_start) {} +inline CodeEntry* ProfileGenerator::FindEntry(Address address) { + CodeEntry* entry = code_map_.FindEntry(address); + if (entry) entry->mark_used(); + return entry; +} + ProfileNode::ProfileNode(ProfileTree* tree, CodeEntry* entry, - ProfileNode* parent) + ProfileNode* parent, int line_number) : tree_(tree), entry_(entry), self_ticks_(0), - children_(CodeEntriesMatch), + line_number_(line_number), parent_(parent), - id_(tree->next_node_id()), - line_ticks_(LineTickMatch) { + id_(tree->next_node_id()) { tree_->EnqueueNode(this); } diff --git a/deps/v8/src/profiler/profile-generator.cc b/deps/v8/src/profiler/profile-generator.cc index 029b6826ec25fa..c01904b77801b1 100644 --- a/deps/v8/src/profiler/profile-generator.cc +++ b/deps/v8/src/profiler/profile-generator.cc @@ -18,33 +18,30 @@ namespace v8 { namespace internal { - -JITLineInfoTable::JITLineInfoTable() {} - - -JITLineInfoTable::~JITLineInfoTable() {} - - -void JITLineInfoTable::SetPosition(int pc_offset, int line) { - DCHECK(pc_offset >= 0); - DCHECK(line > 0); // The 1-based number of the source line. - if (GetSourceLineNumber(pc_offset) != line) { - pc_offset_map_.insert(std::make_pair(pc_offset, line)); +void SourcePositionTable::SetPosition(int pc_offset, int line) { + DCHECK_GE(pc_offset, 0); + DCHECK_GT(line, 0); // The 1-based number of the source line. + // Check that we are inserting in ascending order, so that the vector remains + // sorted. + DCHECK(pc_offsets_to_lines_.empty() || + pc_offsets_to_lines_.back().pc_offset < pc_offset); + if (pc_offsets_to_lines_.empty() || + pc_offsets_to_lines_.back().line_number != line) { + pc_offsets_to_lines_.push_back({pc_offset, line}); } } - -int JITLineInfoTable::GetSourceLineNumber(int pc_offset) const { - PcOffsetMap::const_iterator it = pc_offset_map_.lower_bound(pc_offset); - if (it == pc_offset_map_.end()) { - if (pc_offset_map_.empty()) return v8::CpuProfileNode::kNoLineNumberInfo; - return (--pc_offset_map_.end())->second; +int SourcePositionTable::GetSourceLineNumber(int pc_offset) const { + if (pc_offsets_to_lines_.empty()) { + return v8::CpuProfileNode::kNoLineNumberInfo; } - return it->second; + auto it = + std::upper_bound(pc_offsets_to_lines_.begin(), pc_offsets_to_lines_.end(), + PCOffsetAndLineNumber{pc_offset, 0}); + if (it != pc_offsets_to_lines_.begin()) --it; + return it->line_number; } - -const char* const CodeEntry::kEmptyNamePrefix = ""; const char* const CodeEntry::kEmptyResourceName = ""; const char* const CodeEntry::kEmptyBailoutReason = ""; const char* const CodeEntry::kNoDeoptReason = ""; @@ -85,24 +82,12 @@ CodeEntry* CodeEntry::UnresolvedEntryCreateTrait::Create() { CodeEntry::kUnresolvedFunctionName); } -CodeEntry::~CodeEntry() { - delete line_info_; - for (auto location : inline_locations_) { - for (auto entry : location.second) { - delete entry; - } - } -} - - uint32_t CodeEntry::GetHash() const { uint32_t hash = ComputeIntegerHash(tag()); if (script_id_ != v8::UnboundScript::kNoScriptId) { hash ^= ComputeIntegerHash(static_cast(script_id_)); hash ^= ComputeIntegerHash(static_cast(position_)); } else { - hash ^= ComputeIntegerHash( - static_cast(reinterpret_cast(name_prefix_))); hash ^= ComputeIntegerHash( static_cast(reinterpret_cast(name_))); hash ^= ComputeIntegerHash( @@ -112,14 +97,12 @@ uint32_t CodeEntry::GetHash() const { return hash; } - -bool CodeEntry::IsSameFunctionAs(CodeEntry* entry) const { +bool CodeEntry::IsSameFunctionAs(const CodeEntry* entry) const { if (this == entry) return true; if (script_id_ != v8::UnboundScript::kNoScriptId) { return script_id_ == entry->script_id_ && position_ == entry->position_; } - return name_prefix_ == entry->name_prefix_ && name_ == entry->name_ && - resource_name_ == entry->resource_name_ && + return name_ == entry->name_ && resource_name_ == entry->resource_name_ && line_number_ == entry->line_number_; } @@ -131,30 +114,31 @@ void CodeEntry::SetBuiltinId(Builtins::Name id) { int CodeEntry::GetSourceLine(int pc_offset) const { - if (line_info_ && !line_info_->empty()) { - return line_info_->GetSourceLineNumber(pc_offset); - } + if (line_info_) return line_info_->GetSourceLineNumber(pc_offset); return v8::CpuProfileNode::kNoLineNumberInfo; } -void CodeEntry::AddInlineStack(int pc_offset, - std::vector inline_stack) { - inline_locations_.insert(std::make_pair(pc_offset, std::move(inline_stack))); +void CodeEntry::AddInlineStack( + int pc_offset, std::vector> inline_stack) { + EnsureRareData()->inline_locations_.insert( + std::make_pair(pc_offset, std::move(inline_stack))); } -const std::vector* CodeEntry::GetInlineStack(int pc_offset) const { - auto it = inline_locations_.find(pc_offset); - return it != inline_locations_.end() ? &it->second : NULL; +const std::vector>* CodeEntry::GetInlineStack( + int pc_offset) const { + if (!rare_data_) return nullptr; + auto it = rare_data_->inline_locations_.find(pc_offset); + return it != rare_data_->inline_locations_.end() ? &it->second : nullptr; } -void CodeEntry::AddDeoptInlinedFrames( - int deopt_id, std::vector inlined_frames) { - deopt_inlined_frames_.insert( - std::make_pair(deopt_id, std::move(inlined_frames))); -} - -bool CodeEntry::HasDeoptInlinedFramesFor(int deopt_id) const { - return deopt_inlined_frames_.find(deopt_id) != deopt_inlined_frames_.end(); +void CodeEntry::set_deopt_info( + const char* deopt_reason, int deopt_id, + std::vector inlined_frames) { + DCHECK(!has_deopt_info()); + RareData* rare_data = EnsureRareData(); + rare_data->deopt_reason_ = deopt_reason; + rare_data->deopt_id_ = deopt_id; + rare_data->deopt_inlined_frames_ = std::move(inlined_frames); } void CodeEntry::FillFunctionInfo(SharedFunctionInfo* shared) { @@ -162,49 +146,53 @@ void CodeEntry::FillFunctionInfo(SharedFunctionInfo* shared) { Script* script = Script::cast(shared->script()); set_script_id(script->id()); set_position(shared->start_position()); - set_bailout_reason(GetBailoutReason(shared->disable_optimization_reason())); + if (shared->optimization_disabled()) { + set_bailout_reason(GetBailoutReason(shared->disable_optimization_reason())); + } } CpuProfileDeoptInfo CodeEntry::GetDeoptInfo() { DCHECK(has_deopt_info()); CpuProfileDeoptInfo info; - info.deopt_reason = deopt_reason_; - DCHECK_NE(kNoDeoptimizationId, deopt_id_); - if (deopt_inlined_frames_.find(deopt_id_) == deopt_inlined_frames_.end()) { + info.deopt_reason = rare_data_->deopt_reason_; + DCHECK_NE(kNoDeoptimizationId, rare_data_->deopt_id_); + if (rare_data_->deopt_inlined_frames_.empty()) { info.stack.push_back(CpuProfileDeoptFrame( {script_id_, static_cast(std::max(0, position()))})); } else { - info.stack = deopt_inlined_frames_[deopt_id_]; + info.stack = rare_data_->deopt_inlined_frames_; } return info; } +CodeEntry::RareData* CodeEntry::EnsureRareData() { + if (!rare_data_) { + rare_data_.reset(new RareData()); + } + return rare_data_.get(); +} void ProfileNode::CollectDeoptInfo(CodeEntry* entry) { deopt_infos_.push_back(entry->GetDeoptInfo()); entry->clear_deopt_info(); } - -ProfileNode* ProfileNode::FindChild(CodeEntry* entry) { - base::HashMap::Entry* map_entry = - children_.Lookup(entry, CodeEntryHash(entry)); - return map_entry != NULL ? - reinterpret_cast(map_entry->value) : NULL; +ProfileNode* ProfileNode::FindChild(CodeEntry* entry, int line_number) { + auto map_entry = children_.find({entry, line_number}); + return map_entry != children_.end() ? map_entry->second : nullptr; } - -ProfileNode* ProfileNode::FindOrAddChild(CodeEntry* entry) { - base::HashMap::Entry* map_entry = - children_.LookupOrInsert(entry, CodeEntryHash(entry)); - ProfileNode* node = reinterpret_cast(map_entry->value); - if (!node) { - node = new ProfileNode(tree_, entry, this); - map_entry->value = node; +ProfileNode* ProfileNode::FindOrAddChild(CodeEntry* entry, int line_number) { + auto map_entry = children_.find({entry, line_number}); + if (map_entry == children_.end()) { + ProfileNode* node = new ProfileNode(tree_, entry, this, line_number); + children_[{entry, line_number}] = node; children_list_.push_back(node); + return node; + } else { + return map_entry->second; } - return node; } @@ -212,10 +200,12 @@ void ProfileNode::IncrementLineTicks(int src_line) { if (src_line == v8::CpuProfileNode::kNoLineNumberInfo) return; // Increment a hit counter of a certain source line. // Add a new source line if not found. - base::HashMap::Entry* e = - line_ticks_.LookupOrInsert(reinterpret_cast(src_line), src_line); - DCHECK(e); - e->value = reinterpret_cast(reinterpret_cast(e->value) + 1); + auto map_entry = line_ticks_.find(src_line); + if (map_entry == line_ticks_.end()) { + line_ticks_[src_line] = 1; + } else { + line_ticks_[src_line]++; + } } @@ -223,19 +213,16 @@ bool ProfileNode::GetLineTicks(v8::CpuProfileNode::LineTick* entries, unsigned int length) const { if (entries == NULL || length == 0) return false; - unsigned line_count = line_ticks_.occupancy(); + unsigned line_count = static_cast(line_ticks_.size()); if (line_count == 0) return true; if (length < line_count) return false; v8::CpuProfileNode::LineTick* entry = entries; - for (base::HashMap::Entry *p = line_ticks_.Start(); p != NULL; - p = line_ticks_.Next(p), entry++) { - entry->line = - static_cast(reinterpret_cast(p->key)); - entry->hit_count = - static_cast(reinterpret_cast(p->value)); + for (auto p = line_ticks_.begin(); p != line_ticks_.end(); p++, entry++) { + entry->line = p->first; + entry->hit_count = p->second; } return true; @@ -243,9 +230,9 @@ bool ProfileNode::GetLineTicks(v8::CpuProfileNode::LineTick* entries, void ProfileNode::Print(int indent) { - base::OS::Print("%5u %*s %s%s %d #%d", self_ticks_, indent, "", - entry_->name_prefix(), entry_->name(), entry_->script_id(), - id()); + int line_number = line_number_ != 0 ? line_number_ : entry_->line_number(); + base::OS::Print("%5u %*s %s:%d %d #%d", self_ticks_, indent, "", + entry_->name(), line_number, entry_->script_id(), id()); if (entry_->resource_name()[0] != '\0') base::OS::Print(" %s:%d", entry_->resource_name(), entry_->line_number()); base::OS::Print("\n"); @@ -268,9 +255,8 @@ void ProfileNode::Print(int indent) { base::OS::Print("%*s bailed out due to '%s'\n", indent + 10, "", bailout_reason); } - for (base::HashMap::Entry* p = children_.Start(); p != NULL; - p = children_.Next(p)) { - reinterpret_cast(p->value)->Print(indent + 2); + for (auto child : children_) { + child.second->Print(indent + 2); } } @@ -291,8 +277,7 @@ ProfileTree::ProfileTree(Isolate* isolate) next_node_id_(1), root_(new ProfileNode(this, &root_entry_, nullptr)), isolate_(isolate), - next_function_id_(1), - function_ids_(ProfileNode::CodeEntriesMatch) {} + next_function_id_(1) {} ProfileTree::~ProfileTree() { DeleteNodesCallback cb; @@ -302,12 +287,11 @@ ProfileTree::~ProfileTree() { unsigned ProfileTree::GetFunctionId(const ProfileNode* node) { CodeEntry* code_entry = node->entry(); - base::HashMap::Entry* entry = - function_ids_.LookupOrInsert(code_entry, code_entry->GetHash()); - if (!entry->value) { - entry->value = reinterpret_cast(next_function_id_++); + auto map_entry = function_ids_.find(code_entry); + if (map_entry == function_ids_.end()) { + return function_ids_[code_entry] = next_function_id_++; } - return static_cast(reinterpret_cast(entry->value)); + return function_ids_[code_entry]; } ProfileNode* ProfileTree::AddPathFromEnd(const std::vector& path, @@ -317,7 +301,33 @@ ProfileNode* ProfileTree::AddPathFromEnd(const std::vector& path, for (auto it = path.rbegin(); it != path.rend(); ++it) { if (*it == NULL) continue; last_entry = *it; - node = node->FindOrAddChild(*it); + node = node->FindOrAddChild(*it, v8::CpuProfileNode::kNoLineNumberInfo); + } + if (last_entry && last_entry->has_deopt_info()) { + node->CollectDeoptInfo(last_entry); + } + if (update_stats) { + node->IncrementSelfTicks(); + if (src_line != v8::CpuProfileNode::kNoLineNumberInfo) { + node->IncrementLineTicks(src_line); + } + } + return node; +} + +ProfileNode* ProfileTree::AddPathFromEnd(const ProfileStackTrace& path, + int src_line, bool update_stats, + ProfilingMode mode) { + ProfileNode* node = root_; + CodeEntry* last_entry = nullptr; + int parent_line_number = v8::CpuProfileNode::kNoLineNumberInfo; + for (auto it = path.rbegin(); it != path.rend(); ++it) { + if ((*it).code_entry == nullptr) continue; + last_entry = (*it).code_entry; + node = node->FindOrAddChild((*it).code_entry, parent_line_number); + parent_line_number = mode == ProfilingMode::kCallerLineNumbers + ? (*it).line_number + : v8::CpuProfileNode::kNoLineNumberInfo; } if (last_entry && last_entry->has_deopt_info()) { node->CollectDeoptInfo(last_entry); @@ -384,9 +394,10 @@ void ProfileTree::TraverseDepthFirst(Callback* callback) { using v8::tracing::TracedValue; CpuProfile::CpuProfile(CpuProfiler* profiler, const char* title, - bool record_samples) + bool record_samples, ProfilingMode mode) : title_(title), record_samples_(record_samples), + mode_(mode), start_time_(base::TimeTicks::HighResolutionNow()), top_down_(profiler->isolate()), profiler_(profiler), @@ -399,14 +410,16 @@ CpuProfile::CpuProfile(CpuProfiler* profiler, const char* title, } void CpuProfile::AddPath(base::TimeTicks timestamp, - const std::vector& path, int src_line, + const ProfileStackTrace& path, int src_line, bool update_stats) { ProfileNode* top_frame_node = - top_down_.AddPathFromEnd(path, src_line, update_stats); + top_down_.AddPathFromEnd(path, src_line, update_stats, mode_); + if (record_samples_ && !timestamp.IsNull()) { timestamps_.Add(timestamp); samples_.Add(top_frame_node); } + const int kSamplesFlushCount = 100; const int kNodesFlushCount = 10; if (samples_.length() - streaming_next_sample_ >= kSamplesFlushCount || @@ -502,19 +515,40 @@ void CpuProfile::Print() { top_down_.Print(); } +CodeMap::CodeMap() = default; + +CodeMap::~CodeMap() { + // First clean the free list as it's otherwise impossible to tell + // the slot type. + unsigned free_slot = free_list_head_; + while (free_slot != kNoFreeSlot) { + unsigned next_slot = code_entries_[free_slot].next_free_slot; + code_entries_[free_slot].entry = nullptr; + free_slot = next_slot; + } + for (auto slot : code_entries_) delete slot.entry; +} + void CodeMap::AddCode(Address addr, CodeEntry* entry, unsigned size) { - DeleteAllCoveredCode(addr, addr + size); - code_map_.insert({addr, CodeEntryInfo(entry, size)}); + ClearCodesInRange(addr, addr + size); + unsigned index = AddCodeEntry(addr, entry); + code_map_.emplace(addr, CodeEntryMapInfo{index, size}); + DCHECK(entry->instruction_start() == kNullAddress || + addr == entry->instruction_start()); } -void CodeMap::DeleteAllCoveredCode(Address start, Address end) { +void CodeMap::ClearCodesInRange(Address start, Address end) { auto left = code_map_.upper_bound(start); if (left != code_map_.begin()) { --left; if (left->first + left->second.size <= start) ++left; } auto right = left; - while (right != code_map_.end() && right->first < end) ++right; + for (; right != code_map_.end() && right->first < end; ++right) { + if (!entry(right->second.index)->used()) { + DeleteCodeEntry(right->second.index); + } + } code_map_.erase(left, right); } @@ -522,28 +556,56 @@ CodeEntry* CodeMap::FindEntry(Address addr) { auto it = code_map_.upper_bound(addr); if (it == code_map_.begin()) return nullptr; --it; - Address end_address = it->first + it->second.size; - return addr < end_address ? it->second.entry : nullptr; + Address start_address = it->first; + Address end_address = start_address + it->second.size; + CodeEntry* ret = addr < end_address ? entry(it->second.index) : nullptr; + if (ret && ret->instruction_start() != nullptr) { + DCHECK_EQ(start_address, ret->instruction_start()); + DCHECK(addr >= start_address && addr < end_address); + } + return ret; } void CodeMap::MoveCode(Address from, Address to) { if (from == to) return; auto it = code_map_.find(from); if (it == code_map_.end()) return; - CodeEntryInfo info = it->second; + CodeEntryMapInfo info = it->second; code_map_.erase(it); - AddCode(to, info.entry, info.size); + DCHECK(from + info.size <= to || to + info.size <= from); + ClearCodesInRange(to, to + info.size); + code_map_.emplace(to, info); + + CodeEntry* entry = code_entries_[info.index].entry; + entry->set_instruction_start(to); +} + +unsigned CodeMap::AddCodeEntry(Address start, CodeEntry* entry) { + if (free_list_head_ == kNoFreeSlot) { + code_entries_.push_back(CodeEntrySlotInfo{entry}); + return static_cast(code_entries_.size()) - 1; + } + unsigned index = free_list_head_; + free_list_head_ = code_entries_[index].next_free_slot; + code_entries_[index].entry = entry; + return index; +} + +void CodeMap::DeleteCodeEntry(unsigned index) { + delete code_entries_[index].entry; + code_entries_[index].next_free_slot = free_list_head_; + free_list_head_ = index; } void CodeMap::Print() { - for (auto it = code_map_.begin(); it != code_map_.end(); ++it) { - base::OS::Print("%p %5d %s\n", static_cast(it->first), - it->second.size, it->second.entry->name()); + for (const auto& pair : code_map_) { + base::OS::Print("%p %5d %s\n", reinterpret_cast(pair.first), + pair.second.size, entry(pair.second.index)->name()); } } CpuProfilesCollection::CpuProfilesCollection(Isolate* isolate) - : resource_names_(isolate->heap()), + : resource_names_(isolate->heap()->HashSeed()), profiler_(nullptr), current_profiles_semaphore_(1) {} @@ -559,7 +621,8 @@ CpuProfilesCollection::~CpuProfilesCollection() { bool CpuProfilesCollection::StartProfiling(const char* title, - bool record_samples) { + bool record_samples, + ProfilingMode mode) { current_profiles_semaphore_.Wait(); if (current_profiles_.length() >= kMaxSimultaneousProfiles) { current_profiles_semaphore_.Signal(); @@ -573,7 +636,7 @@ bool CpuProfilesCollection::StartProfiling(const char* title, return true; } } - current_profiles_.Add(new CpuProfile(profiler_, title, record_samples)); + current_profiles_.Add(new CpuProfile(profiler_, title, record_samples, mode)); current_profiles_semaphore_.Signal(); return true; } @@ -619,8 +682,8 @@ void CpuProfilesCollection::RemoveProfile(CpuProfile* profile) { } void CpuProfilesCollection::AddPathToCurrentProfiles( - base::TimeTicks timestamp, const std::vector& path, - int src_line, bool update_stats) { + base::TimeTicks timestamp, const ProfileStackTrace& path, int src_line, + bool update_stats) { // As starting / stopping profiles is rare relatively to this // method, we don't bother minimizing the duration of lock holding, // e.g. copying contents of the list to a local vector. @@ -635,46 +698,52 @@ ProfileGenerator::ProfileGenerator(CpuProfilesCollection* profiles) : profiles_(profiles) {} void ProfileGenerator::RecordTickSample(const TickSample& sample) { - std::vector entries; + ProfileStackTrace stack_trace; // Conservatively reserve space for stack frames + pc + function + vm-state. // There could in fact be more of them because of inlined entries. - entries.reserve(sample.frames_count + 3); + stack_trace.reserve(sample.frames_count + 3); // The ProfileNode knows nothing about all versions of generated code for // the same JS function. The line number information associated with // the latest version of generated code is used to find a source line number // for a JS function. Then, the detected source line is passed to // ProfileNode to increase the tick count for this source line. - int src_line = v8::CpuProfileNode::kNoLineNumberInfo; + const int no_line_info = v8::CpuProfileNode::kNoLineNumberInfo; + int src_line = no_line_info; bool src_line_not_found = true; if (sample.pc != nullptr) { if (sample.has_external_callback && sample.state == EXTERNAL) { // Don't use PC when in external callback code, as it can point - // inside callback's code, and we will erroneously report + // inside a callback's code, and we will erroneously report // that a callback calls itself. - entries.push_back(FindEntry(sample.external_callback_entry)); + stack_trace.push_back( + {FindEntry(reinterpret_cast
(sample.external_callback_entry)), + no_line_info}); } else { - CodeEntry* pc_entry = FindEntry(sample.pc); - // If there is no pc_entry we're likely in native code. - // Find out, if top of stack was pointing inside a JS function - // meaning that we have encountered a frameless invocation. + Address attributed_pc = reinterpret_cast
(sample.pc); + CodeEntry* pc_entry = FindEntry(attributed_pc); + // If there is no pc_entry, we're likely in native code. Find out if the + // top of the stack (the return address) was pointing inside a JS + // function, meaning that we have encountered a frameless invocation. if (!pc_entry && !sample.has_external_callback) { - pc_entry = FindEntry(sample.tos); + attributed_pc = reinterpret_cast
(sample.tos); + pc_entry = FindEntry(attributed_pc); } // If pc is in the function code before it set up stack frame or after the - // frame was destroyed SafeStackFrameIterator incorrectly thinks that - // ebp contains return address of the current function and skips caller's - // frame. Check for this case and just skip such samples. + // frame was destroyed, SafeStackFrameIterator incorrectly thinks that + // ebp contains the return address of the current function and skips the + // caller's frame. Check for this case and just skip such samples. if (pc_entry) { - int pc_offset = static_cast(reinterpret_cast
(sample.pc) - - pc_entry->instruction_start()); + int pc_offset = + static_cast(attributed_pc - pc_entry->instruction_start()); + DCHECK_GE(pc_offset, 0); src_line = pc_entry->GetSourceLine(pc_offset); if (src_line == v8::CpuProfileNode::kNoLineNumberInfo) { src_line = pc_entry->line_number(); } src_line_not_found = false; - entries.push_back(pc_entry); + stack_trace.push_back({pc_entry, src_line}); if (pc_entry->builtin_id() == Builtins::kFunctionPrototypeApply || pc_entry->builtin_id() == Builtins::kFunctionPrototypeCall) { @@ -685,7 +754,8 @@ void ProfileGenerator::RecordTickSample(const TickSample& sample) { // former case we don't so we simply replace the frame with // 'unresolved' entry. if (!sample.has_external_callback) { - entries.push_back(CodeEntry::unresolved_entry()); + stack_trace.push_back( + {CodeEntry::unresolved_entry(), no_line_info}); } } } @@ -694,15 +764,21 @@ void ProfileGenerator::RecordTickSample(const TickSample& sample) { for (unsigned i = 0; i < sample.frames_count; ++i) { Address stack_pos = reinterpret_cast
(sample.stack[i]); CodeEntry* entry = FindEntry(stack_pos); + int line_number = no_line_info; if (entry) { // Find out if the entry has an inlining stack associated. int pc_offset = static_cast(stack_pos - entry->instruction_start()); - const std::vector* inline_stack = + DCHECK_GE(pc_offset, 0); + const std::vector>* inline_stack = entry->GetInlineStack(pc_offset); if (inline_stack) { - entries.insert(entries.end(), inline_stack->rbegin(), - inline_stack->rend()); + std::transform( + inline_stack->rbegin(), inline_stack->rend(), + std::back_inserter(stack_trace), + [=](const std::unique_ptr& ptr) { + return CodeEntryAndLineNumber{ptr.get(), no_line_info}; + }); } // Skip unresolved frames (e.g. internal frame) and get source line of // the first JS caller. @@ -713,33 +789,30 @@ void ProfileGenerator::RecordTickSample(const TickSample& sample) { } src_line_not_found = false; } + line_number = entry->GetSourceLine(pc_offset); } - entries.push_back(entry); + stack_trace.push_back({entry, line_number}); } } if (FLAG_prof_browser_mode) { bool no_symbolized_entries = true; - for (auto e : entries) { - if (e != NULL) { + for (auto e : stack_trace) { + if (e.code_entry != nullptr) { no_symbolized_entries = false; break; } } // If no frames were symbolized, put the VM state entry in. if (no_symbolized_entries) { - entries.push_back(EntryForVMState(sample.state)); + stack_trace.push_back({EntryForVMState(sample.state), no_line_info}); } } - profiles_->AddPathToCurrentProfiles(sample.timestamp, entries, src_line, + profiles_->AddPathToCurrentProfiles(sample.timestamp, stack_trace, src_line, sample.update_stats); } -CodeEntry* ProfileGenerator::FindEntry(void* address) { - return code_map_.FindEntry(reinterpret_cast
(address)); -} - CodeEntry* ProfileGenerator::EntryForVMState(StateTag tag) { switch (tag) { case GC: diff --git a/deps/v8/src/profiler/profile-generator.h b/deps/v8/src/profiler/profile-generator.h index ddd34b00a47e4b..f5b8ef39f26e67 100644 --- a/deps/v8/src/profiler/profile-generator.h +++ b/deps/v8/src/profiler/profile-generator.h @@ -5,9 +5,16 @@ #ifndef V8_PROFILER_PROFILE_GENERATOR_H_ #define V8_PROFILER_PROFILE_GENERATOR_H_ +#include +#include #include +#include +#include +#include +#include + +#include "include/v8-profiler.h" #include "src/allocation.h" -#include "src/base/hashmap.h" #include "src/log.h" #include "src/profiler/strings-storage.h" #include "src/source-position.h" @@ -17,65 +24,71 @@ namespace internal { struct TickSample; -// Provides a mapping from the offsets within generated code to -// the source line. -class JITLineInfoTable : public Malloced { +// Provides a mapping from the offsets within generated code or a bytecode array +// to the source line. +class SourcePositionTable : public Malloced { public: - JITLineInfoTable(); - ~JITLineInfoTable(); + SourcePositionTable() = default; void SetPosition(int pc_offset, int line); int GetSourceLineNumber(int pc_offset) const; - bool empty() const { return pc_offset_map_.empty(); } - private: - // pc_offset -> source line - typedef std::map PcOffsetMap; - PcOffsetMap pc_offset_map_; - DISALLOW_COPY_AND_ASSIGN(JITLineInfoTable); + struct PCOffsetAndLineNumber { + bool operator<(const PCOffsetAndLineNumber& other) const { + return pc_offset < other.pc_offset; + } + int pc_offset; + int line_number; + }; + // This is logically a map, but we store it as a vector of pairs, sorted by + // the pc offset, so that we can save space and look up items using binary + // search. + std::vector pc_offsets_to_lines_; + DISALLOW_COPY_AND_ASSIGN(SourcePositionTable); }; - class CodeEntry { public: // CodeEntry doesn't own name strings, just references them. inline CodeEntry(CodeEventListener::LogEventsAndTags tag, const char* name, - const char* name_prefix = CodeEntry::kEmptyNamePrefix, const char* resource_name = CodeEntry::kEmptyResourceName, int line_number = v8::CpuProfileNode::kNoLineNumberInfo, int column_number = v8::CpuProfileNode::kNoColumnNumberInfo, - JITLineInfoTable* line_info = NULL, + std::unique_ptr line_info = nullptr, Address instruction_start = NULL); - ~CodeEntry(); - const char* name_prefix() const { return name_prefix_; } - bool has_name_prefix() const { return name_prefix_[0] != '\0'; } const char* name() const { return name_; } const char* resource_name() const { return resource_name_; } int line_number() const { return line_number_; } int column_number() const { return column_number_; } - const JITLineInfoTable* line_info() const { return line_info_; } + const SourcePositionTable* line_info() const { return line_info_.get(); } int script_id() const { return script_id_; } void set_script_id(int script_id) { script_id_ = script_id; } int position() const { return position_; } void set_position(int position) { position_ = position; } void set_bailout_reason(const char* bailout_reason) { - bailout_reason_ = bailout_reason; + EnsureRareData()->bailout_reason_ = bailout_reason; } - const char* bailout_reason() const { return bailout_reason_; } - - void set_deopt_info(const char* deopt_reason, int deopt_id) { - DCHECK(!has_deopt_info()); - deopt_reason_ = deopt_reason; - deopt_id_ = deopt_id; + const char* bailout_reason() const { + return rare_data_ ? rare_data_->bailout_reason_ : kEmptyBailoutReason; } + + void set_deopt_info(const char* deopt_reason, int deopt_id, + std::vector inlined_frames); + CpuProfileDeoptInfo GetDeoptInfo(); - bool has_deopt_info() const { return deopt_id_ != kNoDeoptimizationId; } + bool has_deopt_info() const { + return rare_data_ && rare_data_->deopt_id_ != kNoDeoptimizationId; + } void clear_deopt_info() { - deopt_reason_ = kNoDeoptReason; - deopt_id_ = kNoDeoptimizationId; + if (!rare_data_) return; + // TODO(alph): Clear rare_data_ if that was the only field in use. + rare_data_->deopt_reason_ = kNoDeoptReason; + rare_data_->deopt_id_ = kNoDeoptimizationId; } + void mark_used() { bit_field_ = UsedField::update(bit_field_, true); } + bool used() const { return UsedField::decode(bit_field_); } void FillFunctionInfo(SharedFunctionInfo* shared); @@ -85,22 +98,22 @@ class CodeEntry { } uint32_t GetHash() const; - bool IsSameFunctionAs(CodeEntry* entry) const; + bool IsSameFunctionAs(const CodeEntry* entry) const; int GetSourceLine(int pc_offset) const; - void AddInlineStack(int pc_offset, std::vector inline_stack); - const std::vector* GetInlineStack(int pc_offset) const; - - void AddDeoptInlinedFrames(int deopt_id, std::vector); - bool HasDeoptInlinedFramesFor(int deopt_id) const; + void AddInlineStack(int pc_offset, + std::vector> inline_stack); + const std::vector>* GetInlineStack( + int pc_offset) const; + void set_instruction_start(Address start) { instruction_start_ = start; } Address instruction_start() const { return instruction_start_; } + CodeEventListener::LogEventsAndTags tag() const { return TagField::decode(bit_field_); } - static const char* const kEmptyNamePrefix; static const char* const kEmptyResourceName; static const char* const kEmptyBailoutReason; static const char* const kNoDeoptReason; @@ -122,6 +135,17 @@ class CodeEntry { } private: + struct RareData { + const char* deopt_reason_ = kNoDeoptReason; + const char* bailout_reason_ = kEmptyBailoutReason; + int deopt_id_ = kNoDeoptimizationId; + std::unordered_map>> + inline_locations_; + std::vector deopt_inlined_frames_; + }; + + RareData* EnsureRareData(); + struct ProgramEntryCreateTrait { static CodeEntry* Create(); }; @@ -144,38 +168,42 @@ class CodeEntry { static base::LazyDynamicInstance::type kUnresolvedEntry; - class TagField : public BitField {}; - class BuiltinIdField : public BitField {}; + using TagField = BitField; + using BuiltinIdField = BitField; + using UsedField = BitField; uint32_t bit_field_; - const char* name_prefix_; const char* name_; const char* resource_name_; int line_number_; int column_number_; int script_id_; int position_; - const char* bailout_reason_; - const char* deopt_reason_; - int deopt_id_; - JITLineInfoTable* line_info_; + std::unique_ptr line_info_; Address instruction_start_; - // Should be an unordered_map, but it doesn't currently work on Win & MacOS. - std::map> inline_locations_; - std::map> deopt_inlined_frames_; + std::unique_ptr rare_data_; DISALLOW_COPY_AND_ASSIGN(CodeEntry); }; +struct CodeEntryAndLineNumber { + CodeEntry* code_entry; + int line_number; +}; + +typedef std::vector ProfileStackTrace; class ProfileTree; class ProfileNode { public: - inline ProfileNode(ProfileTree* tree, CodeEntry* entry, ProfileNode* parent); + inline ProfileNode(ProfileTree* tree, CodeEntry* entry, ProfileNode* parent, + int line_number = 0); - ProfileNode* FindChild(CodeEntry* entry); - ProfileNode* FindOrAddChild(CodeEntry* entry); + ProfileNode* FindChild( + CodeEntry* entry, + int line_number = v8::CpuProfileNode::kNoLineNumberInfo); + ProfileNode* FindOrAddChild(CodeEntry* entry, int line_number = 0); void IncrementSelfTicks() { ++self_ticks_; } void IncreaseSelfTicks(unsigned amount) { self_ticks_ += amount; } void IncrementLineTicks(int src_line); @@ -186,7 +214,13 @@ class ProfileNode { unsigned id() const { return id_; } unsigned function_id() const; ProfileNode* parent() const { return parent_; } - unsigned int GetHitLineCount() const { return line_ticks_.occupancy(); } + int line_number() const { + return line_number_ != 0 ? line_number_ : entry_->line_number(); + } + + unsigned int GetHitLineCount() const { + return static_cast(line_ticks_.size()); + } bool GetLineTicks(v8::CpuProfileNode::LineTick* entries, unsigned int length) const; void CollectDeoptInfo(CodeEntry* entry); @@ -197,25 +231,31 @@ class ProfileNode { void Print(int indent); - static bool CodeEntriesMatch(void* entry1, void* entry2) { - return reinterpret_cast(entry1) - ->IsSameFunctionAs(reinterpret_cast(entry2)); - } - private: - static uint32_t CodeEntryHash(CodeEntry* entry) { return entry->GetHash(); } - - static bool LineTickMatch(void* a, void* b) { return a == b; } + struct Equals { + bool operator()(CodeEntryAndLineNumber lhs, + CodeEntryAndLineNumber rhs) const { + return lhs.code_entry->IsSameFunctionAs(rhs.code_entry) && + lhs.line_number == rhs.line_number; + } + }; + struct Hasher { + std::size_t operator()(CodeEntryAndLineNumber pair) const { + return pair.code_entry->GetHash() ^ ComputeIntegerHash(pair.line_number); + } + }; ProfileTree* tree_; CodeEntry* entry_; unsigned self_ticks_; - // Mapping from CodeEntry* to ProfileNode* - base::CustomMatcherHashMap children_; + std::unordered_map + children_; + int line_number_; std::vector children_list_; ProfileNode* parent_; unsigned id_; - base::CustomMatcherHashMap line_ticks_; + // maps line number --> number of ticks + std::unordered_map line_ticks_; std::vector deopt_infos_; @@ -228,10 +268,17 @@ class ProfileTree { explicit ProfileTree(Isolate* isolate); ~ProfileTree(); + typedef v8::CpuProfilingMode ProfilingMode; + ProfileNode* AddPathFromEnd( const std::vector& path, int src_line = v8::CpuProfileNode::kNoLineNumberInfo, bool update_stats = true); + ProfileNode* AddPathFromEnd( + const ProfileStackTrace& path, + int src_line = v8::CpuProfileNode::kNoLineNumberInfo, + bool update_stats = true, + ProfilingMode mode = ProfilingMode::kLeafNodeLineNumbers); ProfileNode* root() const { return root_; } unsigned next_node_id() { return next_node_id_++; } unsigned GetFunctionId(const ProfileNode* node); @@ -260,7 +307,7 @@ class ProfileTree { Isolate* isolate_; unsigned next_function_id_; - base::CustomMatcherHashMap function_ids_; + std::unordered_map function_ids_; DISALLOW_COPY_AND_ASSIGN(ProfileTree); }; @@ -268,10 +315,13 @@ class ProfileTree { class CpuProfile { public: - CpuProfile(CpuProfiler* profiler, const char* title, bool record_samples); + typedef v8::CpuProfilingMode ProfilingMode; + + CpuProfile(CpuProfiler* profiler, const char* title, bool record_samples, + ProfilingMode mode); // Add pc -> ... -> main() call path to the profile. - void AddPath(base::TimeTicks timestamp, const std::vector& path, + void AddPath(base::TimeTicks timestamp, const ProfileStackTrace& path, int src_line, bool update_stats); void FinishProfile(); @@ -297,6 +347,7 @@ class CpuProfile { const char* title_; bool record_samples_; + ProfilingMode mode_; base::TimeTicks start_time_; base::TimeTicks end_time_; List samples_; @@ -310,7 +361,8 @@ class CpuProfile { class CodeMap { public: - CodeMap() {} + CodeMap(); + ~CodeMap(); void AddCode(Address addr, CodeEntry* entry, unsigned size); void MoveCode(Address from, Address to); @@ -318,16 +370,27 @@ class CodeMap { void Print(); private: - struct CodeEntryInfo { - CodeEntryInfo(CodeEntry* an_entry, unsigned a_size) - : entry(an_entry), size(a_size) { } - CodeEntry* entry; + struct CodeEntryMapInfo { + unsigned index; unsigned size; }; - void DeleteAllCoveredCode(Address start, Address end); + union CodeEntrySlotInfo { + CodeEntry* entry; + unsigned next_free_slot; + }; + + static constexpr unsigned kNoFreeSlot = std::numeric_limits::max(); + + void ClearCodesInRange(Address start, Address end); + unsigned AddCodeEntry(Address start, CodeEntry*); + void DeleteCodeEntry(unsigned index); - std::map code_map_; + CodeEntry* entry(unsigned index) { return code_entries_[index].entry; } + + std::deque code_entries_; + std::map code_map_; + unsigned free_list_head_ = kNoFreeSlot; DISALLOW_COPY_AND_ASSIGN(CodeMap); }; @@ -337,8 +400,11 @@ class CpuProfilesCollection { explicit CpuProfilesCollection(Isolate* isolate); ~CpuProfilesCollection(); + typedef v8::CpuProfilingMode ProfilingMode; + void set_cpu_profiler(CpuProfiler* profiler) { profiler_ = profiler; } - bool StartProfiling(const char* title, bool record_samples); + bool StartProfiling(const char* title, bool record_samples, + ProfilingMode mode = ProfilingMode::kLeafNodeLineNumbers); CpuProfile* StopProfiling(const char* title); List* profiles() { return &finished_profiles_; } const char* GetName(Name* name) { return resource_names_.GetName(name); } @@ -347,8 +413,8 @@ class CpuProfilesCollection { // Called from profile generator thread. void AddPathToCurrentProfiles(base::TimeTicks timestamp, - const std::vector& path, - int src_line, bool update_stats); + const ProfileStackTrace& path, int src_line, + bool update_stats); // Limits the number of profiles that can be simultaneously collected. static const int kMaxSimultaneousProfiles = 100; @@ -365,7 +431,6 @@ class CpuProfilesCollection { DISALLOW_COPY_AND_ASSIGN(CpuProfilesCollection); }; - class ProfileGenerator { public: explicit ProfileGenerator(CpuProfilesCollection* profiles); @@ -375,7 +440,7 @@ class ProfileGenerator { CodeMap* code_map() { return &code_map_; } private: - CodeEntry* FindEntry(void* address); + CodeEntry* FindEntry(Address address); CodeEntry* EntryForVMState(StateTag tag); CpuProfilesCollection* profiles_; @@ -384,7 +449,6 @@ class ProfileGenerator { DISALLOW_COPY_AND_ASSIGN(ProfileGenerator); }; - } // namespace internal } // namespace v8 diff --git a/deps/v8/src/profiler/profiler-listener.cc b/deps/v8/src/profiler/profiler-listener.cc index b90f5a4894fc76..0feb16dc875346 100644 --- a/deps/v8/src/profiler/profiler-listener.cc +++ b/deps/v8/src/profiler/profiler-listener.cc @@ -13,17 +13,20 @@ namespace v8 { namespace internal { -ProfilerListener::ProfilerListener(Isolate* isolate) - : function_and_resource_names_(isolate->heap()) {} +ProfilerListener::ProfilerListener(Isolate* isolate, + CodeEventObserver* observer) + : isolate_(isolate), + observer_(observer), + function_and_resource_names_(isolate->heap()->HashSeed()) {} ProfilerListener::~ProfilerListener() = default; void ProfilerListener::CallbackEvent(Name* name, Address entry_point) { CodeEventsContainer evt_rec(CodeEventRecord::CODE_CREATION); CodeCreateEventRecord* rec = &evt_rec.CodeCreateEventRecord_; - rec->start = entry_point; + rec->instruction_start = entry_point; rec->entry = NewCodeEntry(CodeEventListener::CALLBACK_TAG, GetName(name)); - rec->size = 1; + rec->instruction_size = 1; DispatchCodeEvent(evt_rec); } @@ -31,13 +34,13 @@ void ProfilerListener::CodeCreateEvent(CodeEventListener::LogEventsAndTags tag, AbstractCode* code, const char* name) { CodeEventsContainer evt_rec(CodeEventRecord::CODE_CREATION); CodeCreateEventRecord* rec = &evt_rec.CodeCreateEventRecord_; - rec->start = code->address(); + rec->instruction_start = code->instruction_start(); rec->entry = NewCodeEntry( - tag, GetFunctionName(name), CodeEntry::kEmptyNamePrefix, + tag, GetFunctionName(name), CodeEntry::kEmptyResourceName, CpuProfileNode::kNoLineNumberInfo, - CpuProfileNode::kNoColumnNumberInfo, NULL, code->instruction_start()); + CpuProfileNode::kNoColumnNumberInfo, nullptr, code->instruction_start()); RecordInliningInfo(rec->entry, code); - rec->size = code->ExecutableSize(); + rec->instruction_size = code->instruction_size(); DispatchCodeEvent(evt_rec); } @@ -45,13 +48,13 @@ void ProfilerListener::CodeCreateEvent(CodeEventListener::LogEventsAndTags tag, AbstractCode* code, Name* name) { CodeEventsContainer evt_rec(CodeEventRecord::CODE_CREATION); CodeCreateEventRecord* rec = &evt_rec.CodeCreateEventRecord_; - rec->start = code->address(); + rec->instruction_start = code->instruction_start(); rec->entry = NewCodeEntry( - tag, GetFunctionName(name), CodeEntry::kEmptyNamePrefix, + tag, GetFunctionName(name), CodeEntry::kEmptyResourceName, CpuProfileNode::kNoLineNumberInfo, - CpuProfileNode::kNoColumnNumberInfo, NULL, code->instruction_start()); + CpuProfileNode::kNoColumnNumberInfo, nullptr, code->instruction_start()); RecordInliningInfo(rec->entry, code); - rec->size = code->ExecutableSize(); + rec->instruction_size = code->instruction_size(); DispatchCodeEvent(evt_rec); } @@ -61,15 +64,15 @@ void ProfilerListener::CodeCreateEvent(CodeEventListener::LogEventsAndTags tag, Name* script_name) { CodeEventsContainer evt_rec(CodeEventRecord::CODE_CREATION); CodeCreateEventRecord* rec = &evt_rec.CodeCreateEventRecord_; - rec->start = code->address(); + rec->instruction_start = code->instruction_start(); rec->entry = NewCodeEntry( - tag, GetFunctionName(shared->DebugName()), CodeEntry::kEmptyNamePrefix, + tag, GetFunctionName(shared->DebugName()), GetName(InferScriptName(script_name, shared)), CpuProfileNode::kNoLineNumberInfo, CpuProfileNode::kNoColumnNumberInfo, - NULL, code->instruction_start()); + nullptr, code->instruction_start()); RecordInliningInfo(rec->entry, code); rec->entry->FillFunctionInfo(shared); - rec->size = code->ExecutableSize(); + rec->instruction_size = code->instruction_size(); DispatchCodeEvent(evt_rec); } @@ -80,13 +83,11 @@ void ProfilerListener::CodeCreateEvent(CodeEventListener::LogEventsAndTags tag, int column) { CodeEventsContainer evt_rec(CodeEventRecord::CODE_CREATION); CodeCreateEventRecord* rec = &evt_rec.CodeCreateEventRecord_; - rec->start = abstract_code->address(); - JITLineInfoTable* line_table = NULL; + rec->instruction_start = abstract_code->instruction_start(); + std::unique_ptr line_table; if (shared->script()->IsScript()) { Script* script = Script::cast(shared->script()); - line_table = new JITLineInfoTable(); - int offset = abstract_code->IsCode() ? Code::kHeaderSize - : BytecodeArray::kHeaderSize; + line_table.reset(new SourcePositionTable()); for (SourcePositionTableIterator it(abstract_code->source_position_table()); !it.done(); it.Advance()) { // TODO(alph,tebbi) Skipping inlined positions for now, because they might @@ -95,18 +96,16 @@ void ProfilerListener::CodeCreateEvent(CodeEventListener::LogEventsAndTags tag, continue; int position = it.source_position().ScriptOffset(); int line_number = script->GetLineNumber(position) + 1; - int pc_offset = it.code_offset() + offset; - line_table->SetPosition(pc_offset, line_number); + line_table->SetPosition(it.code_offset(), line_number); } } - rec->entry = NewCodeEntry( - tag, GetFunctionName(shared->DebugName()), CodeEntry::kEmptyNamePrefix, - GetName(InferScriptName(script_name, shared)), line, column, line_table, - abstract_code->instruction_start()); + rec->entry = + NewCodeEntry(tag, GetFunctionName(shared->DebugName()), + GetName(InferScriptName(script_name, shared)), line, column, + std::move(line_table), abstract_code->instruction_start()); RecordInliningInfo(rec->entry, abstract_code); - RecordDeoptInlinedFrames(rec->entry, abstract_code); rec->entry->FillFunctionInfo(shared); - rec->size = abstract_code->ExecutableSize(); + rec->instruction_size = abstract_code->instruction_size(); DispatchCodeEvent(evt_rec); } @@ -114,21 +113,21 @@ void ProfilerListener::CodeCreateEvent(CodeEventListener::LogEventsAndTags tag, AbstractCode* code, int args_count) { CodeEventsContainer evt_rec(CodeEventRecord::CODE_CREATION); CodeCreateEventRecord* rec = &evt_rec.CodeCreateEventRecord_; - rec->start = code->address(); + rec->instruction_start = code->instruction_start(); rec->entry = NewCodeEntry( - tag, GetName(args_count), "args_count: ", CodeEntry::kEmptyResourceName, + tag, GetName(args_count), CodeEntry::kEmptyResourceName, CpuProfileNode::kNoLineNumberInfo, CpuProfileNode::kNoColumnNumberInfo, - NULL, code->instruction_start()); + nullptr, code->instruction_start()); RecordInliningInfo(rec->entry, code); - rec->size = code->ExecutableSize(); + rec->instruction_size = code->instruction_size(); DispatchCodeEvent(evt_rec); } -void ProfilerListener::CodeMoveEvent(AbstractCode* from, Address to) { +void ProfilerListener::CodeMoveEvent(AbstractCode* from, AbstractCode* to) { CodeEventsContainer evt_rec(CodeEventRecord::CODE_MOVE); CodeMoveEventRecord* rec = &evt_rec.CodeMoveEventRecord_; - rec->from = from->address(); - rec->to = to; + rec->from_instruction_start = from->instruction_start(); + rec->to_instruction_start = to->instruction_start(); DispatchCodeEvent(evt_rec); } @@ -136,7 +135,7 @@ void ProfilerListener::CodeDisableOptEvent(AbstractCode* code, SharedFunctionInfo* shared) { CodeEventsContainer evt_rec(CodeEventRecord::CODE_DISABLE_OPT); CodeDisableOptEventRecord* rec = &evt_rec.CodeDisableOptEventRecord_; - rec->start = code->address(); + rec->instruction_start = code->instruction_start(); rec->bailout_reason = GetBailoutReason(shared->disable_optimization_reason()); DispatchCodeEvent(evt_rec); } @@ -146,21 +145,25 @@ void ProfilerListener::CodeDeoptEvent(Code* code, DeoptKind kind, Address pc, CodeEventsContainer evt_rec(CodeEventRecord::CODE_DEOPT); CodeDeoptEventRecord* rec = &evt_rec.CodeDeoptEventRecord_; Deoptimizer::DeoptInfo info = Deoptimizer::GetDeoptInfo(code, pc); - rec->start = code->address(); + rec->instruction_start = code->instruction_start(); rec->deopt_reason = DeoptimizeReasonToString(info.deopt_reason); rec->deopt_id = info.deopt_id; rec->pc = reinterpret_cast(pc); rec->fp_to_sp_delta = fp_to_sp_delta; + + // When a function is deoptimized, we store the deoptimized frame information + // for the use of GetDeoptInfos(). + AttachDeoptInlinedFrames(code, rec); DispatchCodeEvent(evt_rec); } void ProfilerListener::GetterCallbackEvent(Name* name, Address entry_point) { CodeEventsContainer evt_rec(CodeEventRecord::CODE_CREATION); CodeCreateEventRecord* rec = &evt_rec.CodeCreateEventRecord_; - rec->start = entry_point; + rec->instruction_start = entry_point; rec->entry = - NewCodeEntry(CodeEventListener::CALLBACK_TAG, GetName(name), "get "); - rec->size = 1; + NewCodeEntry(CodeEventListener::CALLBACK_TAG, GetConsName("get ", name)); + rec->instruction_size = 1; DispatchCodeEvent(evt_rec); } @@ -168,22 +171,22 @@ void ProfilerListener::RegExpCodeCreateEvent(AbstractCode* code, String* source) { CodeEventsContainer evt_rec(CodeEventRecord::CODE_CREATION); CodeCreateEventRecord* rec = &evt_rec.CodeCreateEventRecord_; - rec->start = code->address(); + rec->instruction_start = code->instruction_start(); rec->entry = NewCodeEntry( - CodeEventListener::REG_EXP_TAG, GetName(source), "RegExp: ", + CodeEventListener::REG_EXP_TAG, GetConsName("RegExp: ", source), CodeEntry::kEmptyResourceName, CpuProfileNode::kNoLineNumberInfo, - CpuProfileNode::kNoColumnNumberInfo, NULL, code->instruction_start()); - rec->size = code->ExecutableSize(); + CpuProfileNode::kNoColumnNumberInfo, nullptr, code->instruction_start()); + rec->instruction_size = code->instruction_size(); DispatchCodeEvent(evt_rec); } void ProfilerListener::SetterCallbackEvent(Name* name, Address entry_point) { CodeEventsContainer evt_rec(CodeEventRecord::CODE_CREATION); CodeCreateEventRecord* rec = &evt_rec.CodeCreateEventRecord_; - rec->start = entry_point; + rec->instruction_start = entry_point; rec->entry = - NewCodeEntry(CodeEventListener::CALLBACK_TAG, GetName(name), "set "); - rec->size = 1; + NewCodeEntry(CodeEventListener::CALLBACK_TAG, GetConsName("set ", name)); + rec->instruction_size = 1; DispatchCodeEvent(evt_rec); } @@ -212,7 +215,7 @@ void ProfilerListener::RecordInliningInfo(CodeEntry* entry, DCHECK_EQ(Translation::BEGIN, opcode); it.Skip(Translation::NumberOfOperandsFor(opcode)); int depth = 0; - std::vector inline_stack; + std::vector> inline_stack; while (it.HasNext() && Translation::BEGIN != (opcode = static_cast(it.Next()))) { @@ -234,12 +237,12 @@ void ProfilerListener::RecordInliningInfo(CodeEntry* entry, CodeEntry* inline_entry = new CodeEntry(entry->tag(), GetFunctionName(shared_info->DebugName()), - CodeEntry::kEmptyNamePrefix, resource_name, + resource_name, CpuProfileNode::kNoLineNumberInfo, CpuProfileNode::kNoColumnNumberInfo, nullptr, code->instruction_start()); inline_entry->FillFunctionInfo(shared_info); - inline_stack.push_back(inline_entry); + inline_stack.emplace_back(inline_entry); } if (!inline_stack.empty()) { entry->AddInlineStack(pc_offset, std::move(inline_stack)); @@ -247,16 +250,18 @@ void ProfilerListener::RecordInliningInfo(CodeEntry* entry, } } -void ProfilerListener::RecordDeoptInlinedFrames(CodeEntry* entry, - AbstractCode* abstract_code) { - if (abstract_code->kind() != AbstractCode::OPTIMIZED_FUNCTION) return; - Handle code(abstract_code->GetCode()); - +void ProfilerListener::AttachDeoptInlinedFrames(Code* code, + CodeDeoptEventRecord* rec) { + int deopt_id = rec->deopt_id; SourcePosition last_position = SourcePosition::Unknown(); int mask = RelocInfo::ModeMask(RelocInfo::DEOPT_ID) | RelocInfo::ModeMask(RelocInfo::DEOPT_SCRIPT_OFFSET) | RelocInfo::ModeMask(RelocInfo::DEOPT_INLINING_ID); - for (RelocIterator it(*code, mask); !it.done(); it.next()) { + + rec->deopt_frames = nullptr; + rec->deopt_frame_count = 0; + + for (RelocIterator it(code, mask); !it.done(); it.next()) { RelocInfo* info = it.rinfo(); if (info->rmode() == RelocInfo::DEOPT_SCRIPT_OFFSET) { int script_offset = static_cast(info->data()); @@ -267,52 +272,39 @@ void ProfilerListener::RecordDeoptInlinedFrames(CodeEntry* entry, continue; } if (info->rmode() == RelocInfo::DEOPT_ID) { - int deopt_id = static_cast(info->data()); + if (deopt_id != static_cast(info->data())) continue; DCHECK(last_position.IsKnown()); - std::vector inlined_frames; - for (SourcePositionInfo& pos_info : last_position.InliningStack(code)) { - DCHECK(pos_info.position.ScriptOffset() != kNoSourcePosition); + + // SourcePosition::InliningStack allocates a handle for the SFI of each + // frame. These don't escape this function, but quickly add up. This + // scope limits their lifetime. + HandleScope scope(isolate_); + std::vector stack = + last_position.InliningStack(handle(code)); + CpuProfileDeoptFrame* deopt_frames = + new CpuProfileDeoptFrame[stack.size()]; + + int deopt_frame_count = 0; + for (SourcePositionInfo& pos_info : stack) { + if (pos_info.position.ScriptOffset() == kNoSourcePosition) continue; if (!pos_info.function->script()->IsScript()) continue; int script_id = Script::cast(pos_info.function->script())->id(); size_t offset = static_cast(pos_info.position.ScriptOffset()); - inlined_frames.push_back(CpuProfileDeoptFrame({script_id, offset})); - } - if (!inlined_frames.empty() && - !entry->HasDeoptInlinedFramesFor(deopt_id)) { - entry->AddDeoptInlinedFrames(deopt_id, std::move(inlined_frames)); + deopt_frames[deopt_frame_count++] = {script_id, offset}; } + rec->deopt_frames = deopt_frames; + rec->deopt_frame_count = deopt_frame_count; + break; } } } CodeEntry* ProfilerListener::NewCodeEntry( CodeEventListener::LogEventsAndTags tag, const char* name, - const char* name_prefix, const char* resource_name, int line_number, - int column_number, JITLineInfoTable* line_info, Address instruction_start) { - std::unique_ptr code_entry = base::make_unique( - tag, name, name_prefix, resource_name, line_number, column_number, - line_info, instruction_start); - CodeEntry* raw_code_entry = code_entry.get(); - code_entries_.push_back(std::move(code_entry)); - return raw_code_entry; -} - -void ProfilerListener::AddObserver(CodeEventObserver* observer) { - base::LockGuard guard(&mutex_); - if (observers_.empty()) { - code_entries_.clear(); - } - if (std::find(observers_.begin(), observers_.end(), observer) == - observers_.end()) { - observers_.push_back(observer); - } -} - -void ProfilerListener::RemoveObserver(CodeEventObserver* observer) { - base::LockGuard guard(&mutex_); - auto it = std::find(observers_.begin(), observers_.end(), observer); - if (it == observers_.end()) return; - observers_.erase(it); + const char* resource_name, int line_number, int column_number, + std::unique_ptr line_info, Address instruction_start) { + return new CodeEntry(tag, name, resource_name, line_number, column_number, + std::move(line_info), instruction_start); } } // namespace internal diff --git a/deps/v8/src/profiler/profiler-listener.h b/deps/v8/src/profiler/profiler-listener.h index 440afd87a2e97f..e21bcda28450cc 100644 --- a/deps/v8/src/profiler/profiler-listener.h +++ b/deps/v8/src/profiler/profiler-listener.h @@ -5,6 +5,7 @@ #ifndef V8_PROFILER_PROFILER_LISTENER_H_ #define V8_PROFILER_PROFILER_LISTENER_H_ +#include #include #include "src/code-events.h" @@ -14,16 +15,17 @@ namespace v8 { namespace internal { class CodeEventsContainer; +class CodeDeoptEventRecord; class CodeEventObserver { public: virtual void CodeEventHandler(const CodeEventsContainer& evt_rec) = 0; - virtual ~CodeEventObserver() {} + virtual ~CodeEventObserver() = default; }; class ProfilerListener : public CodeEventListener { public: - explicit ProfilerListener(Isolate* isolate); + ProfilerListener(Isolate*, CodeEventObserver*); ~ProfilerListener() override; void CallbackEvent(Name* name, Address entry_point) override; @@ -40,7 +42,7 @@ class ProfilerListener : public CodeEventListener { void CodeCreateEvent(CodeEventListener::LogEventsAndTags tag, AbstractCode* code, int args_count) override; void CodeMovingGCEvent() override {} - void CodeMoveEvent(AbstractCode* from, Address to) override; + void CodeMoveEvent(AbstractCode* from, AbstractCode* to) override; void CodeDisableOptEvent(AbstractCode* code, SharedFunctionInfo* shared) override; void CodeDeoptEvent(Code* code, DeoptKind kind, Address pc, @@ -52,15 +54,11 @@ class ProfilerListener : public CodeEventListener { CodeEntry* NewCodeEntry( CodeEventListener::LogEventsAndTags tag, const char* name, - const char* name_prefix = CodeEntry::kEmptyNamePrefix, const char* resource_name = CodeEntry::kEmptyResourceName, int line_number = v8::CpuProfileNode::kNoLineNumberInfo, int column_number = v8::CpuProfileNode::kNoColumnNumberInfo, - JITLineInfoTable* line_info = NULL, Address instruction_start = NULL); - - void AddObserver(CodeEventObserver* observer); - void RemoveObserver(CodeEventObserver* observer); - V8_INLINE bool HasObservers() { return !observers_.empty(); } + std::unique_ptr line_info = nullptr, + Address instruction_start = NULL); const char* GetName(Name* name) { return function_and_resource_names_.GetName(name); @@ -68,29 +66,27 @@ class ProfilerListener : public CodeEventListener { const char* GetName(int args_count) { return function_and_resource_names_.GetName(args_count); } + const char* GetConsName(const char* prefix, Name* name) { + return function_and_resource_names_.GetConsName(prefix, name); + } const char* GetFunctionName(Name* name) { return function_and_resource_names_.GetFunctionName(name); } const char* GetFunctionName(const char* name) { return function_and_resource_names_.GetFunctionName(name); } - size_t entries_count_for_test() const { return code_entries_.size(); } private: void RecordInliningInfo(CodeEntry* entry, AbstractCode* abstract_code); - void RecordDeoptInlinedFrames(CodeEntry* entry, AbstractCode* abstract_code); + void AttachDeoptInlinedFrames(Code* code, CodeDeoptEventRecord* rec); Name* InferScriptName(Name* name, SharedFunctionInfo* info); V8_INLINE void DispatchCodeEvent(const CodeEventsContainer& evt_rec) { - base::LockGuard guard(&mutex_); - for (auto observer : observers_) { - observer->CodeEventHandler(evt_rec); - } + observer_->CodeEventHandler(evt_rec); } + Isolate* isolate_; + CodeEventObserver* observer_; StringsStorage function_and_resource_names_; - std::vector> code_entries_; - std::vector observers_; - base::Mutex mutex_; DISALLOW_COPY_AND_ASSIGN(ProfilerListener); }; diff --git a/deps/v8/src/profiler/strings-storage.cc b/deps/v8/src/profiler/strings-storage.cc index 05f47788309b7c..cce5ad176560e5 100644 --- a/deps/v8/src/profiler/strings-storage.cc +++ b/deps/v8/src/profiler/strings-storage.cc @@ -6,21 +6,19 @@ #include +#include "src/allocation.h" #include "src/objects-inl.h" namespace v8 { namespace internal { - bool StringsStorage::StringsMatch(void* key1, void* key2) { return strcmp(reinterpret_cast(key1), reinterpret_cast(key2)) == 0; } - -StringsStorage::StringsStorage(Heap* heap) - : hash_seed_(heap->HashSeed()), names_(StringsMatch) {} - +StringsStorage::StringsStorage(uint32_t hash_seed) + : hash_seed_(hash_seed), names_(StringsMatch) {} StringsStorage::~StringsStorage() { for (base::HashMap::Entry* p = names_.Start(); p != NULL; @@ -29,7 +27,6 @@ StringsStorage::~StringsStorage() { } } - const char* StringsStorage::GetCopy(const char* src) { int len = static_cast(strlen(src)); base::HashMap::Entry* entry = GetEntry(src, len); @@ -43,7 +40,6 @@ const char* StringsStorage::GetCopy(const char* src) { return reinterpret_cast(entry->value); } - const char* StringsStorage::GetFormatted(const char* format, ...) { va_list args; va_start(args, format); @@ -52,7 +48,6 @@ const char* StringsStorage::GetFormatted(const char* format, ...) { return result; } - const char* StringsStorage::AddOrDisposeString(char* str, int len) { base::HashMap::Entry* entry = GetEntry(str, len); if (entry->value == NULL) { @@ -65,7 +60,6 @@ const char* StringsStorage::AddOrDisposeString(char* str, int len) { return reinterpret_cast(entry->value); } - const char* StringsStorage::GetVFormatted(const char* format, va_list args) { Vector str = Vector::New(1024); int len = VSNPrintF(str, format, args); @@ -76,7 +70,6 @@ const char* StringsStorage::GetVFormatted(const char* format, va_list args) { return AddOrDisposeString(str.start(), len); } - const char* StringsStorage::GetName(Name* name) { if (name->IsString()) { String* str = String::cast(name); @@ -95,6 +88,25 @@ const char* StringsStorage::GetName(int index) { return GetFormatted("%d", index); } +const char* StringsStorage::GetConsName(const char* prefix, Name* name) { + if (name->IsString()) { + String* str = String::cast(name); + int length = Min(kMaxNameSize, str->length()); + int actual_length = 0; + std::unique_ptr data = str->ToCString( + DISALLOW_NULLS, ROBUST_STRING_TRAVERSAL, 0, length, &actual_length); + + int cons_length = actual_length + static_cast(strlen(prefix)) + 1; + char* cons_result = NewArray(cons_length); + snprintf(cons_result, cons_length, "%s%s", prefix, data.get()); + + return AddOrDisposeString(cons_result, cons_length); + } else if (name->IsSymbol()) { + return ""; + } + return ""; +} + const char* StringsStorage::GetFunctionName(Name* name) { return GetName(name); } diff --git a/deps/v8/src/profiler/strings-storage.h b/deps/v8/src/profiler/strings-storage.h index d73a9dd208a9fe..246a4033d5ee57 100644 --- a/deps/v8/src/profiler/strings-storage.h +++ b/deps/v8/src/profiler/strings-storage.h @@ -7,18 +7,19 @@ #include -#include "src/allocation.h" #include "src/base/compiler-specific.h" #include "src/base/hashmap.h" namespace v8 { namespace internal { +class Name; + // Provides a storage of strings allocated in C++ heap, to hold them // forever, even if they disappear from JS heap or external storage. class StringsStorage { public: - explicit StringsStorage(Heap* heap); + explicit StringsStorage(uint32_t hash_seed); ~StringsStorage(); const char* GetCopy(const char* src); @@ -27,6 +28,7 @@ class StringsStorage { const char* GetVFormatted(const char* format, va_list args); const char* GetName(Name* name); const char* GetName(int index); + const char* GetConsName(const char* prefix, Name* name); const char* GetFunctionName(Name* name); const char* GetFunctionName(const char* name); diff --git a/deps/v8/src/snapshot/serializer.h b/deps/v8/src/snapshot/serializer.h index a1924b4f7ab469..c964b6cdd810f0 100644 --- a/deps/v8/src/snapshot/serializer.h +++ b/deps/v8/src/snapshot/serializer.h @@ -26,8 +26,8 @@ class CodeAddressMap : public CodeEventLogger { isolate_->logger()->removeCodeEventListener(this); } - void CodeMoveEvent(AbstractCode* from, Address to) override { - address_to_name_map_.Move(from->address(), to); + void CodeMoveEvent(AbstractCode* from, AbstractCode* to) override { + address_to_name_map_.Move(from->address(), to->address()); } void CodeDisableOptEvent(AbstractCode* code, diff --git a/deps/v8/test/cctest/cctest.status b/deps/v8/test/cctest/cctest.status index 2c9117ae802d52..f38894bc9c7515 100644 --- a/deps/v8/test/cctest/cctest.status +++ b/deps/v8/test/cctest/cctest.status @@ -71,8 +71,11 @@ # BUG(5193). The cpu profiler tests are notoriously flaky. 'test-cpu-profiler/CollectCpuProfile': [SKIP], + 'test-cpu-profiler/CollectCpuProfileCallerLineNumbers': [FAIL, PASS], + 'test-cpu-profiler/CollectCpuProfileSamples': [SKIP], 'test-cpu-profiler/CollectDeoptEvents': [SKIP], 'test-cpu-profiler/CpuProfileDeepStack': [SKIP], + 'test-cpu-profiler/DeoptUntrackedFunction': [SKIP], 'test-cpu-profiler/FunctionApplySample': [SKIP], 'test-cpu-profiler/HotDeoptNoFrameEntry': [SKIP], 'test-cpu-profiler/JsNative1JsNative2JsSample': [SKIP], diff --git a/deps/v8/test/cctest/test-cpu-profiler.cc b/deps/v8/test/cctest/test-cpu-profiler.cc index b441d04fdd31db..30cf2d8382b782 100644 --- a/deps/v8/test/cctest/test-cpu-profiler.cc +++ b/deps/v8/test/cctest/test-cpu-profiler.cc @@ -159,7 +159,6 @@ TEST(CodeEvents) { i::AbstractCode* aaa_code = CreateCode(&env); i::AbstractCode* comment_code = CreateCode(&env); - i::AbstractCode* args5_code = CreateCode(&env); i::AbstractCode* comment2_code = CreateCode(&env); i::AbstractCode* moved_code = CreateCode(&env); i::AbstractCode* args3_code = CreateCode(&env); @@ -172,9 +171,8 @@ TEST(CodeEvents) { CpuProfiler profiler(isolate, profiles, generator, processor); profiles->StartProfiling("", false); processor->Start(); - ProfilerListener profiler_listener(isolate); - isolate->code_event_dispatcher()->AddListener(&profiler_listener); - profiler_listener.AddObserver(&profiler); + ProfilerListener profiler_listener(isolate, &profiler); + isolate->logger()->addCodeEventListener(&profiler_listener); // Enqueue code creation events. const char* aaa_str = "aaa"; @@ -183,37 +181,33 @@ TEST(CodeEvents) { *aaa_name); profiler_listener.CodeCreateEvent(i::Logger::BUILTIN_TAG, comment_code, "comment"); - profiler_listener.CodeCreateEvent(i::Logger::STUB_TAG, args5_code, 5); profiler_listener.CodeCreateEvent(i::Logger::BUILTIN_TAG, comment2_code, "comment2"); - profiler_listener.CodeMoveEvent(comment2_code, moved_code->address()); + profiler_listener.CodeMoveEvent(comment2_code, moved_code); profiler_listener.CodeCreateEvent(i::Logger::STUB_TAG, args3_code, 3); profiler_listener.CodeCreateEvent(i::Logger::STUB_TAG, args4_code, 4); // Enqueue a tick event to enable code events processing. - EnqueueTickSampleEvent(processor, aaa_code->address()); + EnqueueTickSampleEvent(processor, aaa_code->instruction_start()); - profiler_listener.RemoveObserver(&profiler); - isolate->code_event_dispatcher()->RemoveListener(&profiler_listener); + isolate->logger()->removeCodeEventListener(&profiler_listener); processor->StopSynchronously(); // Check the state of profile generator. - CodeEntry* aaa = generator->code_map()->FindEntry(aaa_code->address()); + CodeEntry* aaa = + generator->code_map()->FindEntry(aaa_code->instruction_start()); CHECK(aaa); CHECK_EQ(0, strcmp(aaa_str, aaa->name())); CodeEntry* comment = - generator->code_map()->FindEntry(comment_code->address()); + generator->code_map()->FindEntry(comment_code->instruction_start()); CHECK(comment); CHECK_EQ(0, strcmp("comment", comment->name())); - CodeEntry* args5 = generator->code_map()->FindEntry(args5_code->address()); - CHECK(args5); - CHECK_EQ(0, strcmp("5", args5->name())); - - CHECK(!generator->code_map()->FindEntry(comment2_code->address())); + CHECK(!generator->code_map()->FindEntry(comment2_code->instruction_start())); - CodeEntry* comment2 = generator->code_map()->FindEntry(moved_code->address()); + CodeEntry* comment2 = + generator->code_map()->FindEntry(moved_code->instruction_start()); CHECK(comment2); CHECK_EQ(0, strcmp("comment2", comment2->name())); } @@ -241,9 +235,8 @@ TEST(TickEvents) { CpuProfiler profiler(isolate, profiles, generator, processor); profiles->StartProfiling("", false); processor->Start(); - ProfilerListener profiler_listener(isolate); - isolate->code_event_dispatcher()->AddListener(&profiler_listener); - profiler_listener.AddObserver(&profiler); + ProfilerListener profiler_listener(isolate, &profiler); + isolate->logger()->addCodeEventListener(&profiler_listener); profiler_listener.CodeCreateEvent(i::Logger::BUILTIN_TAG, frame1_code, "bbb"); profiler_listener.CodeCreateEvent(i::Logger::STUB_TAG, frame2_code, 5); @@ -258,8 +251,7 @@ TEST(TickEvents) { frame2_code->instruction_end() - 1, frame1_code->instruction_end() - 1); - profiler_listener.RemoveObserver(&profiler); - isolate->code_event_dispatcher()->RemoveListener(&profiler_listener); + isolate->logger()->removeCodeEventListener(&profiler_listener); processor->StopSynchronously(); CpuProfile* profile = profiles->StopProfiling(""); CHECK(profile); @@ -314,23 +306,19 @@ TEST(Issue1398) { CpuProfiler profiler(isolate, profiles, generator, processor); profiles->StartProfiling("", false); processor->Start(); - ProfilerListener profiler_listener(isolate); - isolate->code_event_dispatcher()->AddListener(&profiler_listener); - profiler_listener.AddObserver(&profiler); + ProfilerListener profiler_listener(isolate, &profiler); profiler_listener.CodeCreateEvent(i::Logger::BUILTIN_TAG, code, "bbb"); v8::TickSample* sample = processor->StartTickSample(); - sample->pc = code->address(); - sample->tos = 0; + sample->pc = reinterpret_cast(code->instruction_start()); + sample->tos = nullptr; sample->frames_count = v8::TickSample::kMaxFramesCount; for (unsigned i = 0; i < sample->frames_count; ++i) { - sample->stack[i] = code->address(); + sample->stack[i] = reinterpret_cast(code->instruction_start()); } processor->FinishTickSample(); - profiler_listener.RemoveObserver(&profiler); - isolate->code_event_dispatcher()->RemoveListener(&profiler_listener); processor->StopSynchronously(); CpuProfile* profile = profiles->StopProfiling(""); CHECK(profile); @@ -453,11 +441,14 @@ class ProfilerHelper { profiler_->Dispose(); } + typedef v8::CpuProfilingMode ProfilingMode; + v8::CpuProfile* Run(v8::Local function, v8::Local argv[], int argc, unsigned min_js_samples = 0, unsigned min_external_samples = 0, - bool collect_samples = false); + bool collect_samples = false, + ProfilingMode mode = ProfilingMode::kLeafNodeLineNumbers); v8::CpuProfiler* profiler() { return profiler_; } @@ -470,11 +461,11 @@ v8::CpuProfile* ProfilerHelper::Run(v8::Local function, v8::Local argv[], int argc, unsigned min_js_samples, unsigned min_external_samples, - bool collect_samples) { + bool collect_samples, ProfilingMode mode) { v8::Local profile_name = v8_str("my_profile"); profiler_->SetSamplingInterval(100); - profiler_->StartProfiling(profile_name, collect_samples); + profiler_->StartProfiling(profile_name, mode, collect_samples); v8::internal::CpuProfiler* iprofiler = reinterpret_cast(profiler_); @@ -521,7 +512,6 @@ static const v8::CpuProfileNode* GetChild(v8::Local context, return result; } - static void CheckSimpleBranch(v8::Local context, const v8::CpuProfileNode* node, const char* names[], int length) { @@ -531,7 +521,6 @@ static void CheckSimpleBranch(v8::Local context, } } - static const ProfileNode* GetSimpleBranch(v8::Local context, v8::CpuProfile* profile, const char* names[], int length) { @@ -542,6 +531,43 @@ static const ProfileNode* GetSimpleBranch(v8::Local context, return reinterpret_cast(node); } +struct NameLinePair { + const char* name; + int line_number; +}; + +static const v8::CpuProfileNode* FindChild(const v8::CpuProfileNode* node, + NameLinePair pair) { + for (int i = 0, count = node->GetChildrenCount(); i < count; ++i) { + const v8::CpuProfileNode* child = node->GetChild(i); + // The name and line number must match, or if the requested line number was + // -1, then match any function of the same name. + if (strcmp(child->GetFunctionNameStr(), pair.name) == 0 && + (child->GetLineNumber() == pair.line_number || + pair.line_number == -1)) { + return child; + } + } + return nullptr; +} + +static const v8::CpuProfileNode* GetChild(const v8::CpuProfileNode* node, + NameLinePair pair) { + const v8::CpuProfileNode* result = FindChild(node, pair); + char buffer[100]; + i::SNPrintF(i::ArrayVector(buffer), "Failed to GetChild: %s:%d", pair.name, pair.line_number); + if (!result) FATAL(buffer); + return result; +} + +static void CheckBranch(const v8::CpuProfileNode* node, NameLinePair path[], + int length) { + for (int i = 0; i < length; i++) { + NameLinePair pair = path[i]; + node = GetChild(node, pair); + } +} + static const char* cpu_profiler_test_source = "%NeverOptimizeFunction(loop);\n" "%NeverOptimizeFunction(delay);\n" @@ -622,6 +648,40 @@ TEST(CollectCpuProfile) { profile->Delete(); } +TEST(CollectCpuProfileCallerLineNumbers) { + i::FLAG_allow_natives_syntax = true; + LocalContext env; + v8::HandleScope scope(env->GetIsolate()); + + CompileRun(cpu_profiler_test_source); + v8::Local function = GetFunction(env.local(), "start"); + + int32_t profiling_interval_ms = 200; + v8::Local args[] = { + v8::Integer::New(env->GetIsolate(), profiling_interval_ms)}; + ProfilerHelper helper(env.local()); + helper.Run(function, args, arraysize(args), 1000, 0, false, + v8::CpuProfilingMode::kCallerLineNumbers); + v8::CpuProfile* profile = + helper.Run(function, args, arraysize(args), 1000, 0, false, + v8::CpuProfilingMode::kCallerLineNumbers); + + const v8::CpuProfileNode* root = profile->GetTopDownRoot(); + const v8::CpuProfileNode* start_node = GetChild(root, {"start", 27}); + const v8::CpuProfileNode* foo_node = GetChild(start_node, {"foo", 30}); + + NameLinePair bar_branch[] = {{"bar", 23}, {"delay", 19}, {"loop", 18}}; + CheckBranch(foo_node, bar_branch, arraysize(bar_branch)); + NameLinePair baz_branch[] = {{"baz", 25}, {"delay", 20}, {"loop", 18}}; + CheckBranch(foo_node, baz_branch, arraysize(baz_branch)); + NameLinePair delay_at22_branch[] = {{"delay", 22}, {"loop", 18}}; + CheckBranch(foo_node, delay_at22_branch, arraysize(delay_at22_branch)); + NameLinePair delay_at24_branch[] = {{"delay", 24}, {"loop", 18}}; + CheckBranch(foo_node, delay_at24_branch, arraysize(delay_at24_branch)); + + profile->Delete(); +} + static const char* hot_deopt_no_frame_entry_test_source = "%NeverOptimizeFunction(foo);\n" "%NeverOptimizeFunction(start);\n" @@ -1087,9 +1147,7 @@ static void TickLines(bool optimize) { CpuProfiler profiler(isolate, profiles, generator, processor); profiles->StartProfiling("", false); processor->Start(); - ProfilerListener profiler_listener(isolate); - isolate->code_event_dispatcher()->AddListener(&profiler_listener); - profiler_listener.AddObserver(&profiler); + ProfilerListener profiler_listener(isolate, &profiler); // Enqueue code creation events. i::Handle str = factory->NewStringFromAsciiChecked(func_name); @@ -1101,8 +1159,6 @@ static void TickLines(bool optimize) { // Enqueue a tick event to enable code events processing. EnqueueTickSampleEvent(processor, code_address); - profiler_listener.RemoveObserver(&profiler); - isolate->code_event_dispatcher()->RemoveListener(&profiler_listener); processor->StopSynchronously(); CpuProfile* profile = profiles->StopProfiling(""); @@ -1112,9 +1168,10 @@ static void TickLines(bool optimize) { CodeEntry* func_entry = generator->code_map()->FindEntry(code_address); CHECK(func_entry); CHECK_EQ(0, strcmp(func_name, func_entry->name())); - const i::JITLineInfoTable* line_info = func_entry->line_info(); + const i::SourcePositionTable* line_info = func_entry->line_info(); CHECK(line_info); - CHECK(!line_info->empty()); + CHECK_NE(v8::CpuProfileNode::kNoLineNumberInfo, + line_info->GetSourceLineNumber(100)); // Check the hit source lines using V8 Public APIs. const i::ProfileTree* tree = profile->top_down(); @@ -2293,8 +2350,57 @@ TEST(CodeEntriesMemoryLeak) { v8::CpuProfile* profile = helper.Run(function, nullptr, 0); profile->Delete(); } - ProfilerListener* profiler_listener = - CcTest::i_isolate()->logger()->profiler_listener(); - CHECK_GE(10000ul, profiler_listener->entries_count_for_test()); + i::CpuProfiler* profiler = + reinterpret_cast(helper.profiler()); + CHECK(!profiler->profiler_listener_for_test()); +} + +TEST(SourcePositionTable) { + i::SourcePositionTable info; + + // Newly created tables should return NoLineNumberInfo for any lookup. + int no_info = v8::CpuProfileNode::kNoLineNumberInfo; + CHECK_EQ(no_info, info.GetSourceLineNumber(std::numeric_limits::min())); + CHECK_EQ(no_info, info.GetSourceLineNumber(0)); + CHECK_EQ(no_info, info.GetSourceLineNumber(1)); + CHECK_EQ(no_info, info.GetSourceLineNumber(9)); + CHECK_EQ(no_info, info.GetSourceLineNumber(10)); + CHECK_EQ(no_info, info.GetSourceLineNumber(11)); + CHECK_EQ(no_info, info.GetSourceLineNumber(19)); + CHECK_EQ(no_info, info.GetSourceLineNumber(20)); + CHECK_EQ(no_info, info.GetSourceLineNumber(21)); + CHECK_EQ(no_info, info.GetSourceLineNumber(100)); + CHECK_EQ(no_info, info.GetSourceLineNumber(std::numeric_limits::max())); + + info.SetPosition(10, 1); + info.SetPosition(20, 2); + + // The only valid return values are 1 or 2 - every pc maps to a line number. + CHECK_EQ(1, info.GetSourceLineNumber(std::numeric_limits::min())); + CHECK_EQ(1, info.GetSourceLineNumber(0)); + CHECK_EQ(1, info.GetSourceLineNumber(1)); + CHECK_EQ(1, info.GetSourceLineNumber(9)); + CHECK_EQ(1, info.GetSourceLineNumber(10)); + CHECK_EQ(1, info.GetSourceLineNumber(11)); + CHECK_EQ(1, info.GetSourceLineNumber(19)); + CHECK_EQ(2, info.GetSourceLineNumber(20)); + CHECK_EQ(2, info.GetSourceLineNumber(21)); + CHECK_EQ(2, info.GetSourceLineNumber(100)); + CHECK_EQ(2, info.GetSourceLineNumber(std::numeric_limits::max())); + + // Test SetPosition behavior. + info.SetPosition(25, 3); + CHECK_EQ(2, info.GetSourceLineNumber(21)); + CHECK_EQ(3, info.GetSourceLineNumber(100)); + CHECK_EQ(3, info.GetSourceLineNumber(std::numeric_limits::max())); +} + +TEST(MultipleProfilers) { + std::unique_ptr profiler1(new CpuProfiler(CcTest::i_isolate())); + std::unique_ptr profiler2(new CpuProfiler(CcTest::i_isolate())); + profiler1->StartProfiling("1"); + profiler2->StartProfiling("2"); + profiler1->StopProfiling("1"); + profiler2->StopProfiling("2"); } diff --git a/deps/v8/test/cctest/test-log.cc b/deps/v8/test/cctest/test-log.cc index 03d90b5012a453..46c065722b2e68 100644 --- a/deps/v8/test/cctest/test-log.cc +++ b/deps/v8/test/cctest/test-log.cc @@ -570,7 +570,7 @@ TEST(LogVersion) { TEST(Issue539892) { class : public i::CodeEventLogger { public: - void CodeMoveEvent(i::AbstractCode* from, Address to) override {} + void CodeMoveEvent(i::AbstractCode* from, i::AbstractCode* to) override {} void CodeDisableOptEvent(i::AbstractCode* code, i::SharedFunctionInfo* shared) override {} diff --git a/deps/v8/test/cctest/test-profile-generator.cc b/deps/v8/test/cctest/test-profile-generator.cc index 67d289302495e7..8447ef84c71a6b 100644 --- a/deps/v8/test/cctest/test-profile-generator.cc +++ b/deps/v8/test/cctest/test-profile-generator.cc @@ -72,6 +72,25 @@ TEST(ProfileNodeFindOrAddChild) { CHECK_EQ(childNode3, node->FindOrAddChild(&entry3)); } +TEST(ProfileNodeFindOrAddChildWithLineNumber) { + CcTest::InitializeVM(); + ProfileTree tree(CcTest::i_isolate()); + ProfileNode* root = tree.root(); + CodeEntry a(i::CodeEventListener::FUNCTION_TAG, "a"); + ProfileNode* a_node = root->FindOrAddChild(&a, -1); + + // a --(22)--> child1 + // --(23)--> child1 + + CodeEntry child1(i::CodeEventListener::FUNCTION_TAG, "child1"); + ProfileNode* child1_node = a_node->FindOrAddChild(&child1, 22); + CHECK(child1_node); + CHECK_EQ(child1_node, a_node->FindOrAddChild(&child1, 22)); + + ProfileNode* child2_node = a_node->FindOrAddChild(&child1, 23); + CHECK(child2_node); + CHECK_NE(child1_node, child2_node); +} TEST(ProfileNodeFindOrAddChildForSameFunction) { CcTest::InitializeVM(); @@ -180,6 +199,29 @@ TEST(ProfileTreeAddPathFromEnd) { CHECK_EQ(1u, node4->self_ticks()); } +TEST(ProfileTreeAddPathFromEndWithLineNumbers) { + CcTest::InitializeVM(); + CodeEntry a(i::CodeEventListener::FUNCTION_TAG, "a"); + CodeEntry b(i::CodeEventListener::FUNCTION_TAG, "b"); + CodeEntry c(i::CodeEventListener::FUNCTION_TAG, "c"); + ProfileTree tree(CcTest::i_isolate()); + ProfileTreeTestHelper helper(&tree); + + v8::internal::ProfileStackTrace path = {{&c, 5}, {&b, 3}, {&a, 1}}; + tree.AddPathFromEnd(path, v8::CpuProfileNode::kNoLineNumberInfo, true, + v8::CpuProfilingMode::kCallerLineNumbers); + + ProfileNode* a_node = + tree.root()->FindChild(&a, v8::CpuProfileNode::kNoLineNumberInfo); + tree.Print(); + CHECK(a_node); + + ProfileNode* b_node = a_node->FindChild(&b, 1); + CHECK(b_node); + + ProfileNode* c_node = b_node->FindChild(&c, 3); + CHECK(c_node); +} TEST(ProfileTreeCalculateTotalTicks) { CcTest::InitializeVM(); @@ -278,52 +320,50 @@ static inline i::Address ToAddress(int n) { TEST(CodeMapAddCode) { CodeMap code_map; - CodeEntry entry1(i::CodeEventListener::FUNCTION_TAG, "aaa"); - CodeEntry entry2(i::CodeEventListener::FUNCTION_TAG, "bbb"); - CodeEntry entry3(i::CodeEventListener::FUNCTION_TAG, "ccc"); - CodeEntry entry4(i::CodeEventListener::FUNCTION_TAG, "ddd"); - code_map.AddCode(ToAddress(0x1500), &entry1, 0x200); - code_map.AddCode(ToAddress(0x1700), &entry2, 0x100); - code_map.AddCode(ToAddress(0x1900), &entry3, 0x50); - code_map.AddCode(ToAddress(0x1950), &entry4, 0x10); + CodeEntry* entry1 = new CodeEntry(i::CodeEventListener::FUNCTION_TAG, "aaa"); + CodeEntry* entry2 = new CodeEntry(i::CodeEventListener::FUNCTION_TAG, "bbb"); + CodeEntry* entry3 = new CodeEntry(i::CodeEventListener::FUNCTION_TAG, "ccc"); + CodeEntry* entry4 = new CodeEntry(i::CodeEventListener::FUNCTION_TAG, "ddd"); + code_map.AddCode(ToAddress(0x1500), entry1, 0x200); + code_map.AddCode(ToAddress(0x1700), entry2, 0x100); + code_map.AddCode(ToAddress(0x1900), entry3, 0x50); + code_map.AddCode(ToAddress(0x1950), entry4, 0x10); CHECK(!code_map.FindEntry(0)); CHECK(!code_map.FindEntry(ToAddress(0x1500 - 1))); - CHECK_EQ(&entry1, code_map.FindEntry(ToAddress(0x1500))); - CHECK_EQ(&entry1, code_map.FindEntry(ToAddress(0x1500 + 0x100))); - CHECK_EQ(&entry1, code_map.FindEntry(ToAddress(0x1500 + 0x200 - 1))); - CHECK_EQ(&entry2, code_map.FindEntry(ToAddress(0x1700))); - CHECK_EQ(&entry2, code_map.FindEntry(ToAddress(0x1700 + 0x50))); - CHECK_EQ(&entry2, code_map.FindEntry(ToAddress(0x1700 + 0x100 - 1))); + CHECK_EQ(entry1, code_map.FindEntry(ToAddress(0x1500))); + CHECK_EQ(entry1, code_map.FindEntry(ToAddress(0x1500 + 0x100))); + CHECK_EQ(entry1, code_map.FindEntry(ToAddress(0x1500 + 0x200 - 1))); + CHECK_EQ(entry2, code_map.FindEntry(ToAddress(0x1700))); + CHECK_EQ(entry2, code_map.FindEntry(ToAddress(0x1700 + 0x50))); + CHECK_EQ(entry2, code_map.FindEntry(ToAddress(0x1700 + 0x100 - 1))); CHECK(!code_map.FindEntry(ToAddress(0x1700 + 0x100))); CHECK(!code_map.FindEntry(ToAddress(0x1900 - 1))); - CHECK_EQ(&entry3, code_map.FindEntry(ToAddress(0x1900))); - CHECK_EQ(&entry3, code_map.FindEntry(ToAddress(0x1900 + 0x28))); - CHECK_EQ(&entry4, code_map.FindEntry(ToAddress(0x1950))); - CHECK_EQ(&entry4, code_map.FindEntry(ToAddress(0x1950 + 0x7))); - CHECK_EQ(&entry4, code_map.FindEntry(ToAddress(0x1950 + 0x10 - 1))); + CHECK_EQ(entry3, code_map.FindEntry(ToAddress(0x1900))); + CHECK_EQ(entry3, code_map.FindEntry(ToAddress(0x1900 + 0x28))); + CHECK_EQ(entry4, code_map.FindEntry(ToAddress(0x1950))); + CHECK_EQ(entry4, code_map.FindEntry(ToAddress(0x1950 + 0x7))); + CHECK_EQ(entry4, code_map.FindEntry(ToAddress(0x1950 + 0x10 - 1))); CHECK(!code_map.FindEntry(ToAddress(0x1950 + 0x10))); CHECK(!code_map.FindEntry(ToAddress(0xFFFFFFFF))); } - TEST(CodeMapMoveAndDeleteCode) { CodeMap code_map; - CodeEntry entry1(i::CodeEventListener::FUNCTION_TAG, "aaa"); - CodeEntry entry2(i::CodeEventListener::FUNCTION_TAG, "bbb"); - code_map.AddCode(ToAddress(0x1500), &entry1, 0x200); - code_map.AddCode(ToAddress(0x1700), &entry2, 0x100); - CHECK_EQ(&entry1, code_map.FindEntry(ToAddress(0x1500))); - CHECK_EQ(&entry2, code_map.FindEntry(ToAddress(0x1700))); + CodeEntry* entry1 = new CodeEntry(i::CodeEventListener::FUNCTION_TAG, "aaa"); + CodeEntry* entry2 = new CodeEntry(i::CodeEventListener::FUNCTION_TAG, "bbb"); + code_map.AddCode(ToAddress(0x1500), entry1, 0x200); + code_map.AddCode(ToAddress(0x1700), entry2, 0x100); + CHECK_EQ(entry1, code_map.FindEntry(ToAddress(0x1500))); + CHECK_EQ(entry2, code_map.FindEntry(ToAddress(0x1700))); code_map.MoveCode(ToAddress(0x1500), ToAddress(0x1700)); // Deprecate bbb. CHECK(!code_map.FindEntry(ToAddress(0x1500))); - CHECK_EQ(&entry1, code_map.FindEntry(ToAddress(0x1700))); - CodeEntry entry3(i::CodeEventListener::FUNCTION_TAG, "ccc"); - code_map.AddCode(ToAddress(0x1750), &entry3, 0x100); + CHECK_EQ(entry1, code_map.FindEntry(ToAddress(0x1700))); + CodeEntry* entry3 = new CodeEntry(i::CodeEventListener::FUNCTION_TAG, "ccc"); + code_map.AddCode(ToAddress(0x1750), entry3, 0x100); CHECK(!code_map.FindEntry(ToAddress(0x1700))); - CHECK_EQ(&entry3, code_map.FindEntry(ToAddress(0x1750))); + CHECK_EQ(entry3, code_map.FindEntry(ToAddress(0x1750))); } - namespace { class TestSetup { @@ -401,10 +441,6 @@ TEST(RecordTickSample) { ProfileNode* node4 = top_down_test_helper.Walk(entry1, entry3, entry1); CHECK(node4); CHECK_EQ(entry1, node4->entry()); - - delete entry1; - delete entry2; - delete entry3; } static void CheckNodeIds(const ProfileNode* node, unsigned* expectedId) { @@ -466,10 +502,6 @@ TEST(SampleIds) { for (int i = 0; i < 3; i++) { CHECK_EQ(expected_id[i], profile->sample(i)->id()); } - - delete entry1; - delete entry2; - delete entry3; } @@ -498,8 +530,6 @@ TEST(NoSamples) { CHECK_EQ(3u, nodeId - 1); CHECK_EQ(0, profile->samples_count()); - - delete entry1; } @@ -656,7 +686,8 @@ int GetFunctionLineNumber(CpuProfiler& profiler, LocalContext& env, i::Handle func = i::Handle::cast( v8::Utils::OpenHandle(*v8::Local::Cast( env->Global()->Get(env.local(), v8_str(name)).ToLocalChecked()))); - CodeEntry* func_entry = code_map->FindEntry(func->abstract_code()->address()); + CodeEntry* func_entry = + code_map->FindEntry(func->abstract_code()->instruction_start()); if (!func_entry) FATAL(name); return func_entry->line_number(); From ca8d4e3450b5a83a05b25e0c5046417d3360184b Mon Sep 17 00:00:00 2001 From: Ben Noordhuis Date: Tue, 23 Jan 2018 01:17:21 +0100 Subject: [PATCH 016/129] build: define NOMINMAX on windows Build with `-DNOMINMAX` to stop `` from defining macros that conflict with `std::min()` and `std::max()`. Backport-PR-URL: https://github.com/nodejs/node/pull/22731 PR-URL: https://github.com/nodejs/node/pull/18216 Reviewed-By: Colin Ihrig Reviewed-By: James M Snell Reviewed-By: Joyee Cheung PR-URL: https://github.com/nodejs/node/pull/22731 --- node.gyp | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/node.gyp b/node.gyp index 1f9d34b200966c..cb0036629e21a1 100644 --- a/node.gyp +++ b/node.gyp @@ -446,6 +446,10 @@ 'FD_SETSIZE=1024', # we need to use node's preferred "win32" rather than gyp's preferred "win" 'NODE_PLATFORM="win32"', + # Stop from defining macros that conflict with + # std::min() and std::max(). We don't use (much) + # but we still inherit it from uv.h. + 'NOMINMAX', '_UNICODE=1', ], 'libraries': [ '-lpsapi.lib' ] From b67bf38f31576c22143f51a3b314883e97f39c29 Mon Sep 17 00:00:00 2001 From: Ben Noordhuis Date: Tue, 23 Jan 2018 01:17:21 +0100 Subject: [PATCH 017/129] src: fix fs.write() externalized string handling * Respect `encoding` argument when the string is externalized. * Copy the string when the write request can outlive the externalized string. This commit removes `StringBytes::GetExternalParts()` because it is fundamentally broken. Fixes: https://github.com/nodejs/node/issues/18146 Fixes: https://github.com/nodejs/node/issues/22728 Backport-PR-URL: https://github.com/nodejs/node/pull/22731 PR-URL: https://github.com/nodejs/node/pull/18216 Reviewed-By: James M Snell --- src/node_file.cc | 50 ++++++++++----- src/string_bytes.cc | 113 +++++++++------------------------ src/string_bytes.h | 6 -- test/parallel/test-fs-write.js | 44 +++++++++++++ 4 files changed, 106 insertions(+), 107 deletions(-) diff --git a/src/node_file.cc b/src/node_file.cc index 39cce2ea6bdd64..e67be4ef4b0b21 100644 --- a/src/node_file.cc +++ b/src/node_file.cc @@ -37,6 +37,7 @@ # include #endif +#include #include namespace node { @@ -1127,39 +1128,54 @@ static void WriteString(const FunctionCallbackInfo& args) { if (!args[0]->IsInt32()) return env->ThrowTypeError("First argument must be file descriptor"); + std::unique_ptr delete_on_return; Local req; - Local string = args[1]; + Local value = args[1]; int fd = args[0]->Int32Value(); char* buf = nullptr; - int64_t pos; size_t len; FSReqWrap::Ownership ownership = FSReqWrap::COPY; - // will assign buf and len if string was external - if (!StringBytes::GetExternalParts(string, - const_cast(&buf), - &len)) { - enum encoding enc = ParseEncoding(env->isolate(), args[3], UTF8); - len = StringBytes::StorageSize(env->isolate(), string, enc); + const int64_t pos = GET_OFFSET(args[2]); + const auto enc = ParseEncoding(env->isolate(), args[3], UTF8); + const auto is_async = args[4]->IsObject(); + + // Avoid copying the string when it is externalized but only when: + // 1. The target encoding is compatible with the string's encoding, and + // 2. The write is synchronous, otherwise the string might get neutered + // while the request is in flight, and + // 3. For UCS2, when the host system is little-endian. Big-endian systems + // need to call StringBytes::Write() to ensure proper byte swapping. + // The const_casts are conceptually sound: memory is read but not written. + if (!is_async && value->IsString()) { + auto string = value.As(); + if ((enc == ASCII || enc == LATIN1) && string->IsExternalOneByte()) { + auto ext = string->GetExternalOneByteStringResource(); + buf = const_cast(ext->data()); + len = ext->length(); + } else if (enc == UCS2 && IsLittleEndian() && string->IsExternal()) { + auto ext = string->GetExternalStringResource(); + buf = reinterpret_cast(const_cast(ext->data())); + len = ext->length() * sizeof(*ext->data()); + } + } + + if (buf == nullptr) { + len = StringBytes::StorageSize(env->isolate(), value, enc); buf = new char[len]; + // SYNC_CALL returns on error. Make sure to always free the memory. + if (!is_async) delete_on_return.reset(buf); // StorageSize may return too large a char, so correct the actual length // by the write size len = StringBytes::Write(env->isolate(), buf, len, args[1], enc); ownership = FSReqWrap::MOVE; } - pos = GET_OFFSET(args[2]); + req = args[4]; - uv_buf_t uvbuf = uv_buf_init(const_cast(buf), len); + uv_buf_t uvbuf = uv_buf_init(buf, len); if (!req->IsObject()) { - // SYNC_CALL returns on error. Make sure to always free the memory. - struct Delete { - inline explicit Delete(char* pointer) : pointer_(pointer) {} - inline ~Delete() { delete[] pointer_; } - char* const pointer_; - }; - Delete delete_on_return(ownership == FSReqWrap::MOVE ? buf : nullptr); SYNC_CALL(write, nullptr, fd, &uvbuf, 1, pos) return args.GetReturnValue().Set(SYNC_RESULT); } diff --git a/src/string_bytes.cc b/src/string_bytes.cc index 258bc72cf62a43..2e41c1478c92fc 100644 --- a/src/string_bytes.cc +++ b/src/string_bytes.cc @@ -27,6 +27,8 @@ #include #include // memcpy + +#include #include // When creating strings >= this length v8's gc spins up and consumes @@ -269,39 +271,6 @@ static size_t hex_decode(char* buf, } -bool StringBytes::GetExternalParts(Local val, - const char** data, - size_t* len) { - if (Buffer::HasInstance(val)) { - *data = Buffer::Data(val); - *len = Buffer::Length(val); - return true; - } - - if (!val->IsString()) - return false; - - Local str = val.As(); - - if (str->IsExternalOneByte()) { - const String::ExternalOneByteStringResource* ext; - ext = str->GetExternalOneByteStringResource(); - *data = ext->data(); - *len = ext->length(); - return true; - - } else if (str->IsExternal()) { - const String::ExternalStringResource* ext; - ext = str->GetExternalStringResource(); - *data = reinterpret_cast(ext->data()); - *len = ext->length() * sizeof(*ext->data()); - return true; - } - - return false; -} - - size_t StringBytes::WriteUCS2(char* buf, size_t buflen, Local str, @@ -351,17 +320,15 @@ size_t StringBytes::Write(Isolate* isolate, enum encoding encoding, int* chars_written) { HandleScope scope(isolate); - const char* data = nullptr; - size_t nbytes = 0; - const bool is_extern = GetExternalParts(val, &data, &nbytes); - const size_t external_nbytes = nbytes; + size_t nbytes; + int nchars; + + if (chars_written == nullptr) + chars_written = &nchars; CHECK(val->IsString() == true); Local str = val.As(); - if (nbytes > buflen) - nbytes = buflen; - int flags = String::HINT_MANY_WRITES_EXPECTED | String::NO_NULL_TERMINATION | String::REPLACE_INVALID_UTF8; @@ -369,14 +336,15 @@ size_t StringBytes::Write(Isolate* isolate, switch (encoding) { case ASCII: case LATIN1: - if (is_extern && str->IsOneByte()) { - memcpy(buf, data, nbytes); + if (str->IsExternalOneByte()) { + auto ext = str->GetExternalOneByteStringResource(); + nbytes = std::min(buflen, ext->length()); + memcpy(buf, ext->data(), nbytes); } else { uint8_t* const dst = reinterpret_cast(buf); nbytes = str->WriteOneByte(dst, 0, buflen, flags); } - if (chars_written != nullptr) - *chars_written = nbytes; + *chars_written = nbytes; break; case BUFFER: @@ -387,14 +355,8 @@ size_t StringBytes::Write(Isolate* isolate, case UCS2: { size_t nchars; - if (is_extern && !str->IsOneByte()) { - memcpy(buf, data, nbytes); - nchars = nbytes / sizeof(uint16_t); - } else { - nbytes = WriteUCS2(buf, buflen, str, flags, &nchars); - } - if (chars_written != nullptr) - *chars_written = nchars; + nbytes = WriteUCS2(buf, buflen, str, flags, &nchars); + *chars_written = static_cast(nchars); // Node's "ucs2" encoding wants LE character data stored in // the Buffer, so we need to reorder on BE platforms. See @@ -407,27 +369,25 @@ size_t StringBytes::Write(Isolate* isolate, } case BASE64: - if (is_extern) { - nbytes = base64_decode(buf, buflen, data, external_nbytes); + if (str->IsExternalOneByte()) { + auto ext = str->GetExternalOneByteStringResource(); + nbytes = base64_decode(buf, buflen, ext->data(), ext->length()); } else { String::Value value(str); nbytes = base64_decode(buf, buflen, *value, value.length()); } - if (chars_written != nullptr) { - *chars_written = nbytes; - } + *chars_written = nbytes; break; case HEX: - if (is_extern) { - nbytes = hex_decode(buf, buflen, data, external_nbytes); + if (str->IsExternalOneByte()) { + auto ext = str->GetExternalOneByteStringResource(); + nbytes = hex_decode(buf, buflen, ext->data(), ext->length()); } else { String::Value value(str); nbytes = hex_decode(buf, buflen, *value, value.length()); } - if (chars_written != nullptr) { - *chars_written = nbytes; - } + *chars_written = nbytes; break; default: @@ -504,49 +464,34 @@ size_t StringBytes::Size(Isolate* isolate, Local val, enum encoding encoding) { HandleScope scope(isolate); - size_t data_size = 0; - bool is_buffer = Buffer::HasInstance(val); - if (is_buffer && (encoding == BUFFER || encoding == LATIN1)) + if (Buffer::HasInstance(val) && (encoding == BUFFER || encoding == LATIN1)) return Buffer::Length(val); - const char* data; - if (GetExternalParts(val, &data, &data_size)) - return data_size; - Local str = val->ToString(isolate); switch (encoding) { case ASCII: case LATIN1: - data_size = str->Length(); - break; + return str->Length(); case BUFFER: case UTF8: - data_size = str->Utf8Length(); - break; + return str->Utf8Length(); case UCS2: - data_size = str->Length() * sizeof(uint16_t); - break; + return str->Length() * sizeof(uint16_t); case BASE64: { String::Value value(str); - data_size = base64_decoded_size(*value, value.length()); - break; + return base64_decoded_size(*value, value.length()); } case HEX: - data_size = str->Length() / 2; - break; - - default: - CHECK(0 && "unknown encoding"); - break; + return str->Length() / 2; } - return data_size; + UNREACHABLE(); } diff --git a/src/string_bytes.h b/src/string_bytes.h index 17bbd80c0ab1c2..7a70f06f63dec4 100644 --- a/src/string_bytes.h +++ b/src/string_bytes.h @@ -81,12 +81,6 @@ class StringBytes { v8::Local val, enum encoding enc); - // If the string is external then assign external properties to data and len, - // then return true. If not return false. - static bool GetExternalParts(v8::Local val, - const char** data, - size_t* len); - // Write the bytes from the string or buffer into the char* // returns the number of bytes written, which will always be // <= buflen. Use StorageSize/Size first to know how much diff --git a/test/parallel/test-fs-write.js b/test/parallel/test-fs-write.js index 5dd9ca0e902a1e..f50f1eb314fdc8 100644 --- a/test/parallel/test-fs-write.js +++ b/test/parallel/test-fs-write.js @@ -19,6 +19,7 @@ // OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE // USE OR OTHER DEALINGS IN THE SOFTWARE. +// Flags: --expose_externalize_string 'use strict'; const common = require('../common'); const assert = require('assert'); @@ -34,6 +35,49 @@ const fn3 = path.join(tmpdir.path, 'write3.txt'); const expected = 'ümlaut.'; const constants = fs.constants; +/* eslint-disable no-undef */ +common.allowGlobals(externalizeString, isOneByteString, x); + +{ + const expected = 'ümlaut eins'; // Must be a unique string. + externalizeString(expected); + assert.strictEqual(true, isOneByteString(expected)); + const fd = fs.openSync(fn, 'w'); + fs.writeSync(fd, expected, 0, 'latin1'); + fs.closeSync(fd); + assert.strictEqual(expected, fs.readFileSync(fn, 'latin1')); +} + +{ + const expected = 'ümlaut zwei'; // Must be a unique string. + externalizeString(expected); + assert.strictEqual(true, isOneByteString(expected)); + const fd = fs.openSync(fn, 'w'); + fs.writeSync(fd, expected, 0, 'utf8'); + fs.closeSync(fd); + assert.strictEqual(expected, fs.readFileSync(fn, 'utf8')); +} + +{ + const expected = '中文 1'; // Must be a unique string. + externalizeString(expected); + assert.strictEqual(false, isOneByteString(expected)); + const fd = fs.openSync(fn, 'w'); + fs.writeSync(fd, expected, 0, 'ucs2'); + fs.closeSync(fd); + assert.strictEqual(expected, fs.readFileSync(fn, 'ucs2')); +} + +{ + const expected = '中文 2'; // Must be a unique string. + externalizeString(expected); + assert.strictEqual(false, isOneByteString(expected)); + const fd = fs.openSync(fn, 'w'); + fs.writeSync(fd, expected, 0, 'utf8'); + fs.closeSync(fd); + assert.strictEqual(expected, fs.readFileSync(fn, 'utf8')); +} + fs.open(fn, 'w', 0o644, common.mustCall(function(err, fd) { assert.ifError(err); From 4bcdc1b83c9fabe930f6f990a5446b031a3f6779 Mon Sep 17 00:00:00 2001 From: Rich Trott Date: Fri, 9 Mar 2018 10:05:36 -0800 Subject: [PATCH 018/129] test: fix assertion argument order MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fix the assertion argument order so that it will report "actual" and "expected" correctly when the test fails. Ref: https://github.com/nodejs/node/issues/19263 PR-URL: https://github.com/nodejs/node/pull/19264 Refs: https://github.com/nodejs/node/issues/19263 Reviewed-By: Richard Lau Reviewed-By: Colin Ihrig Reviewed-By: Luigi Pinca Reviewed-By: James M Snell Reviewed-By: Tobias Nießen --- test/sequential/test-inspector-stop-profile-after-done.js | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/test/sequential/test-inspector-stop-profile-after-done.js b/test/sequential/test-inspector-stop-profile-after-done.js index b488281b0cf25f..8801656bdfc22c 100644 --- a/test/sequential/test-inspector-stop-profile-after-done.js +++ b/test/sequential/test-inspector-stop-profile-after-done.js @@ -24,7 +24,7 @@ async function runTests() { 'Waiting for the debugger to disconnect...'); await session.send({ method: 'Profiler.stop' }); session.disconnect(); - assert.strictEqual(0, (await child.expectShutdown()).exitCode); + assert.strictEqual((await child.expectShutdown()).exitCode, 0); } common.crashOnUnhandledRejection(); From 459690aca46cd3c76093103fe390257190080280 Mon Sep 17 00:00:00 2001 From: Rich Trott Date: Fri, 9 Mar 2018 14:49:59 -0800 Subject: [PATCH 019/129] doc: improve style guide text * Specify that personal pronouns are OK in colloquial documentation rather than just pronouns. Pronouns are OK in all documentation. (For example, "it" is a pronoun and is acceptable in all types of documentation.) Specify "personal pronouns" for clarity. * more colloquial -> colloquial * like -> such as * Remove "mass nouns" as no mass nouns are given as examples. Plural nouns seems to be what was meant, so use that instead. * Repeat "gender-neutral" to make it clear that it refers to both terms and not merely the first term it appears before. * Remove "non-comprehensive examples". Examples are, by definition, non-comprehensive. No need to announce that the examples are examples. It is obvious. PR-URL: https://github.com/nodejs/node/pull/19269 Reviewed-By: Luigi Pinca --- doc/STYLE_GUIDE.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/doc/STYLE_GUIDE.md b/doc/STYLE_GUIDE.md index b4ed5f932014cd..93d4b3a806aa01 100644 --- a/doc/STYLE_GUIDE.md +++ b/doc/STYLE_GUIDE.md @@ -16,9 +16,8 @@ "color" vs. "colour", etc. * Use [serial commas][]. * Avoid personal pronouns in reference documentation ("I", "you", "we"). - * Pronouns are acceptable in more colloquial documentation, like guides. - * Use gender-neutral pronouns and mass nouns. Non-comprehensive - examples: + * Personal pronouns are acceptable in colloquial documentation such as guides. + * Use gender-neutral pronouns and gender-neutral plural nouns. * OK: "they", "their", "them", "folks", "people", "developers" * NOT OK: "his", "hers", "him", "her", "guys", "dudes" * When combining wrapping elements (parentheses and quotes), terminal From fe3836a8711a94686c70b3abb4d03c3683650209 Mon Sep 17 00:00:00 2001 From: Ujjwal Sharma Date: Sun, 11 Mar 2018 15:30:24 +0530 Subject: [PATCH 020/129] test: delete test/parallel/test-regress-GH-4948 Refs: https://github.com/nodejs/node/pull/19275#issuecomment-372101538 --- test/parallel/test-regress-GH-4948.js | 43 --------------------------- 1 file changed, 43 deletions(-) delete mode 100644 test/parallel/test-regress-GH-4948.js diff --git a/test/parallel/test-regress-GH-4948.js b/test/parallel/test-regress-GH-4948.js deleted file mode 100644 index d4d63600868470..00000000000000 --- a/test/parallel/test-regress-GH-4948.js +++ /dev/null @@ -1,43 +0,0 @@ -'use strict'; -// https://github.com/joyent/node/issues/4948 - -require('../common'); -const http = require('http'); - -let reqCount = 0; -const server = http.createServer(function(serverReq, serverRes) { - if (reqCount) { - serverRes.end(); - server.close(); - return; - } - reqCount = 1; - - - // normally the use case would be to call an external site - // does not require connecting locally or to itself to fail - const r = http.request({ hostname: 'localhost', - port: this.address().port }, function(res) { - // required, just needs to be in the client response somewhere - serverRes.end(); - - // required for test to fail - res.on('data', () => {}); - - }); - r.on('error', () => {}); - r.end(); - - serverRes.write('some data'); -}).listen(0, function() { - // simulate a client request that closes early - const net = require('net'); - - const sock = new net.Socket(); - sock.connect(this.address().port, 'localhost'); - - sock.on('connect', function() { - sock.write('GET / HTTP/1.1\r\n\r\n'); - sock.end(); - }); -}); From 5862d0372c26c3fe3884bcdf16eafdb3fd3a9b39 Mon Sep 17 00:00:00 2001 From: James M Snell Date: Wed, 11 Apr 2018 13:59:26 -0700 Subject: [PATCH 021/129] http2: fix ping duration calculation Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/19956 Reviewed-By: Ruben Bridgewater Reviewed-By: Colin Ihrig Reviewed-By: Trivikram Kamat --- src/node_http2.cc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/node_http2.cc b/src/node_http2.cc index e9a06a88635882..036decb89581c9 100644 --- a/src/node_http2.cc +++ b/src/node_http2.cc @@ -2832,8 +2832,8 @@ void Http2Session::Http2Ping::Send(uint8_t* payload) { } void Http2Session::Http2Ping::Done(bool ack, const uint8_t* payload) { - session_->statistics_.ping_rtt = (uv_hrtime() - startTime_); - double duration = (session_->statistics_.ping_rtt - startTime_) / 1e6; + session_->statistics_.ping_rtt = uv_hrtime() - startTime_; + double duration = session_->statistics_.ping_rtt / 1e6; Local buf = Undefined(env()->isolate()); if (payload != nullptr) { From c0d1423bd318959d3bac2ba7f045198e6a304b37 Mon Sep 17 00:00:00 2001 From: Indranil Dasgupta Date: Sat, 14 Apr 2018 15:19:45 +0200 Subject: [PATCH 022/129] doc: close event does not take arguments Explicitly added in the docs that the close event does not expect any arguments when invoked. Fixes: https://github.com/nodejs/node/issues/20018 Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/20031 Reviewed-By: Vse Mozhet Byt Reviewed-By: Trivikram Kamat Reviewed-By: James M Snell --- doc/api/http2.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/doc/api/http2.md b/doc/api/http2.md index df70278a071732..ea5449733b051b 100644 --- a/doc/api/http2.md +++ b/doc/api/http2.md @@ -128,7 +128,8 @@ solely on the API of the `Http2Session`. added: v8.4.0 --> -The `'close'` event is emitted once the `Http2Session` has been destroyed. +The `'close'` event is emitted once the `Http2Session` has been destroyed. Its +listener does not expect any arguments. #### Event: 'connect' -The `'goaway'` event is emitted when a GOAWAY frame is received. When invoked, +The `'goaway'` event is emitted when a `GOAWAY` frame is received. When invoked, the handler function will receive three arguments: -* `errorCode` {number} The HTTP/2 error code specified in the GOAWAY frame. +* `errorCode` {number} The HTTP/2 error code specified in the `GOAWAY` frame. * `lastStreamID` {number} The ID of the last stream the remote peer successfully processed (or `0` if no ID is specified). -* `opaqueData` {Buffer} If additional opaque data was included in the GOAWAY +* `opaqueData` {Buffer} If additional opaque data was included in the `GOAWAY` frame, a `Buffer` instance will be passed containing that data. *Note*: The `Http2Session` instance will be shut down automatically when the @@ -193,7 +193,7 @@ the handler function will receive three arguments: added: v8.4.0 --> -The `'localSettings'` event is emitted when an acknowledgment SETTINGS frame +The `'localSettings'` event is emitted when an acknowledgment `SETTINGS` frame has been received. When invoked, the handler function will receive a copy of the local settings. @@ -214,7 +214,7 @@ session.on('localSettings', (settings) => { added: v8.4.0 --> -The `'remoteSettings'` event is emitted when a new SETTINGS frame is received +The `'remoteSettings'` event is emitted when a new `SETTINGS` frame is received from the connected peer. When invoked, the handler function will receive a copy of the remote settings. @@ -385,7 +385,7 @@ added: v8.11.2 * `code` {number} An HTTP/2 error code * `lastStreamID` {number} The numeric ID of the last processed `Http2Stream` * `opaqueData` {Buffer|TypedArray|DataView} A `TypedArray` or `DataView` - instance containing additional data to be carried within the GOAWAY frame. + instance containing additional data to be carried within the `GOAWAY` frame. Transmits a `GOAWAY` frame to the connected peer *without* shutting down the `Http2Session`. @@ -419,7 +419,7 @@ added: v8.4.0 * Value: {boolean} Indicates whether or not the `Http2Session` is currently waiting for an -acknowledgment for a sent SETTINGS frame. Will be `true` after calling the +acknowledgment for a sent `SETTINGS` frame. Will be `true` after calling the `http2session.settings()` method. Will be `false` once all sent SETTINGS frames have been acknowledged. @@ -554,9 +554,9 @@ Once called, the `http2session.pendingSettingsAck` property will be `true` while the session is waiting for the remote peer to acknowledge the new settings. -*Note*: The new settings will not become effective until the SETTINGS +*Note*: The new settings will not become effective until the `SETTINGS` acknowledgment is received and the `'localSettings'` event is emitted. It -is possible to send multiple SETTINGS frames while acknowledgment is still +is possible to send multiple `SETTINGS` frames while acknowledgment is still pending. #### http2session.type @@ -699,8 +699,8 @@ added: v8.4.0 * `weight` {number} Specifies the relative dependency of a stream in relation to other streams with the same `parent`. The value is a number between `1` and `256` (inclusive). - * `getTrailers` {Function} Callback function invoked to collect trailer - headers. + * `waitForTrailers` {boolean} When `true`, the `Http2Stream` will emit the + `'wantTrailers'` event after the final `DATA` frame has been sent. * Returns: {ClientHttp2Stream} @@ -727,15 +727,15 @@ req.on('response', (headers) => { }); ``` -When set, the `options.getTrailers()` function is called immediately after -queuing the last chunk of payload data to be sent. The callback is passed a -single object (with a `null` prototype) that the listener may use to specify -the trailing header fields to send to the peer. +When the `options.waitForTrailers` option is set, the `'wantTrailers'` event +is emitted immediately after queuing the last chunk of payload data to be sent. +The `http2stream.sendTrailers()` method can then be called to send trailing +headers to the peer. -*Note*: The HTTP/1 specification forbids trailers from containing HTTP/2 -pseudo-header fields (e.g. `':method'`, `':path'`, etc). An `'error'` event -will be emitted if the `getTrailers` callback attempts to set such header -fields. +It is important to note that when `options.waitForTrailers` is set, the +`Http2Stream` will *not* automatically close when the final `DATA` frame is +transmitted. User code *must* call either `http2stream.sendTrailers()` or +`http2stream.close()` to close the `Http2Stream`. The `:method` and `:path` pseudo-headers are not specified within `headers`, they respectively default to: @@ -881,6 +881,16 @@ stream.on('trailers', (headers, flags) => { }); ``` +#### Event: 'wantTrailers' + + +The `'wantTrailers'` event is emitted when the `Http2Stream` has queued the +final `DATA` frame to be sent on a frame and the `Http2Stream` is ready to send +trailing headers. When initiating a request or response, the `waitForTrailers` +option must be set for this event to be emitted. + #### http2stream.aborted + +* `headers` {HTTP/2 Headers Object} + +Sends a trailing `HEADERS` frame to the connected HTTP/2 peer. This method +will cause the `Http2Stream` to be immediately closed and must only be +called after the `'wantTrailers'` event has been emitted. When sending a +request or sending a response, the `options.waitForTrailers` option must be set +in order to keep the `Http2Stream` open after the final `DATA` frame so that +trailers can be sent. + +```js +const http2 = require('http2'); +const server = http2.createServer(); +server.on('stream', (stream) => { + stream.respond(undefined, { waitForTrailers: true }); + stream.on('wantTrailers', () => { + stream.sendTrailers({ xyz: 'abc' }); + }); + stream.end('Hello World'); +}); +``` + +The HTTP/1 specification forbids trailers from containing HTTP/2 pseudo-header +fields (e.g. `':method'`, `':path'`, etc). + ### Class: ClientHttp2Stream +* `session` {Http2Session} +* `socket` {net.Socket} + The `'connect'` event is emitted once the `Http2Session` has been successfully connected to the remote peer and communication may begin. From 5abe246a4473ffe7fa0fa73dc0f599078b75d573 Mon Sep 17 00:00:00 2001 From: "Christine E. Taylor" Date: Fri, 20 Apr 2018 20:21:44 -0700 Subject: [PATCH 026/129] test: add strictEqual method to assert Adds strictEqual method to assert on stream.session.alpnProtocol to verify expected value 'h2'. This changes 'h2' from an assertion error message to the value expected from stream.session.alpnProtocol. Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/20189 Reviewed-By: Rich Trott Reviewed-By: Richard Lau Reviewed-By: Trivikram Kamat Reviewed-By: Luigi Pinca Reviewed-By: Benjamin Gruenbaum Reviewed-By: Anna Henningsen Reviewed-By: Colin Ihrig Reviewed-By: Ruben Bridgewater --- test/parallel/test-http2-create-client-secure-session.js | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/test/parallel/test-http2-create-client-secure-session.js b/test/parallel/test-http2-create-client-secure-session.js index 1f20ec8e42a871..8b2aa1c168cb5e 100644 --- a/test/parallel/test-http2-create-client-secure-session.js +++ b/test/parallel/test-http2-create-client-secure-session.js @@ -21,7 +21,7 @@ function onStream(stream, headers) { const socket = stream.session[kSocket]; assert(stream.session.encrypted); - assert(stream.session.alpnProtocol, 'h2'); + assert.strictEqual(stream.session.alpnProtocol, 'h2'); const originSet = stream.session.originSet; assert(Array.isArray(originSet)); assert.strictEqual(originSet[0], From b32cf8fa40894e2f7bc8ef09bf378bbbcfedba26 Mon Sep 17 00:00:00 2001 From: Ujjwal Sharma Date: Sun, 22 Apr 2018 02:49:45 +0530 Subject: [PATCH 027/129] doc: add parameters for Http2Session:error event Add parameters for the callback for the Http2Session:error event inline with the pattern in the rest of the documentation. Refs: https://github.com/nodejs/help/issues/877#issuecomment-381253464 Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/20206 Reviewed-By: Matteo Collina Reviewed-By: Vse Mozhet Byt Reviewed-By: Luigi Pinca Reviewed-By: Trivikram Kamat --- doc/api/http2.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/api/http2.md b/doc/api/http2.md index a32fd36764b2a8..93c7b8dff442ac 100644 --- a/doc/api/http2.md +++ b/doc/api/http2.md @@ -149,6 +149,8 @@ connected to the remote peer and communication may begin. added: v8.4.0 --> +* `error` {Error} + The `'error'` event is emitted when an error occurs during the processing of an `Http2Session`. From 98ed30f3f546ded9ff9c014bc5ba69d6222f1ffa Mon Sep 17 00:00:00 2001 From: Ujjwal Sharma Date: Tue, 24 Apr 2018 02:47:21 +0530 Subject: [PATCH 028/129] doc: improve docs for Http2Session:frameError Improve documentation regarding the callback parameters for the frameError event for instances of Http2Session, making it inline with the currently accepted structure, like the rest of the documentation. Refs: https://github.com/nodejs/help/issues/877#issuecomment-381253464 Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/20236 Reviewed-By: Vse Mozhet Byt Reviewed-By: Trivikram Kamat Reviewed-By: Ruben Bridgewater --- doc/api/http2.md | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/doc/api/http2.md b/doc/api/http2.md index 93c7b8dff442ac..7d9cbadd1de917 100644 --- a/doc/api/http2.md +++ b/doc/api/http2.md @@ -159,18 +159,16 @@ an `Http2Session`. added: v8.4.0 --> +* `type` {integer} The frame type. +* `code` {integer} The error code. +* `id` {integer} The stream id (or `0` if the frame isn't associated with a + stream). + The `'frameError'` event is emitted when an error occurs while attempting to send a frame on the session. If the frame that could not be sent is associated with a specific `Http2Stream`, an attempt to emit `'frameError'` event on the `Http2Stream` is made. -When invoked, the handler function will receive three arguments: - -* An integer identifying the frame type. -* An integer identifying the error code. -* An integer identifying the stream (or 0 if the frame is not associated with - a stream). - If the `'frameError'` event is associated with a stream, the stream will be closed and destroyed immediately following the `'frameError'` event. If the event is not associated with a stream, the `Http2Session` will be shut down From 1edd7f6393f3ca16650e0aa5f4b3c1abc270b566 Mon Sep 17 00:00:00 2001 From: Robert Nagy Date: Tue, 17 Apr 2018 11:37:50 +0200 Subject: [PATCH 029/129] http: added aborted property to request Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/20094 Reviewed-By: James M Snell Reviewed-By: Luigi Pinca --- doc/api/http.md | 12 +++++++++- doc/api/http2.md | 10 ++++++++ lib/_http_client.js | 6 ++++- lib/_http_incoming.js | 2 ++ lib/_http_server.js | 1 + lib/internal/http2/compat.js | 7 ++++++ test/parallel/test-http-aborted.js | 26 +++++++++++++++++++++ test/parallel/test-http2-compat-aborted.js | 27 ++++++++++++++++++++++ 8 files changed, 89 insertions(+), 2 deletions(-) create mode 100644 test/parallel/test-http-aborted.js create mode 100644 test/parallel/test-http2-compat-aborted.js diff --git a/doc/api/http.md b/doc/api/http.md index 0640ec417d0b88..ecffb87be4bbaf 100644 --- a/doc/api/http.md +++ b/doc/api/http.md @@ -1401,7 +1401,7 @@ following additional events, methods, and properties. added: v0.3.8 --> -Emitted when the request has been aborted and the network socket has closed. +Emitted when the request has been aborted. ### Event: 'close' + +* {boolean} + +The `message.aborted` property will be `true` if the request has +been aborted. + ### message.destroy([error]) + +* {boolean} + +The `request.aborted` property will be `true` if the request has +been aborted. + #### request.destroy([error]) -The `'goaway'` event is emitted when a `GOAWAY` frame is received. When invoked, -the handler function will receive three arguments: - * `errorCode` {number} The HTTP/2 error code specified in the `GOAWAY` frame. * `lastStreamID` {number} The ID of the last stream the remote peer successfully processed (or `0` if no ID is specified). * `opaqueData` {Buffer} If additional opaque data was included in the `GOAWAY` frame, a `Buffer` instance will be passed containing that data. -*Note*: The `Http2Session` instance will be shut down automatically when the -`'goaway'` event is emitted. +The `'goaway'` event is emitted when a `GOAWAY` frame is received. + +The `Http2Session` instance will be shut down automatically when the `'goaway'` +event is emitted. #### Event: 'localSettings' +* `settings` {HTTP/2 Settings Object} A copy of the `SETTINGS` frame received. + The `'localSettings'` event is emitted when an acknowledgment `SETTINGS` frame -has been received. When invoked, the handler function will receive a copy of -the local settings. +has been received. *Note*: When using `http2session.settings()` to submit new settings, the modified settings do not take effect until the `'localSettings'` event is @@ -216,9 +217,10 @@ session.on('localSettings', (settings) => { added: v8.4.0 --> +* `settings` {HTTP/2 Settings Object} A copy of the `SETTINGS` frame received. + The `'remoteSettings'` event is emitted when a new `SETTINGS` frame is received -from the connected peer. When invoked, the handler function will receive a copy -of the remote settings. +from the connected peer. ```js session.on('remoteSettings', (settings) => { From 74192ddb6617b2656d9cdf77b3500d83f949e2f8 Mon Sep 17 00:00:00 2001 From: Daniel Bevenius Date: Mon, 30 Apr 2018 14:18:53 +0200 Subject: [PATCH 036/129] http2: reduce require calls in http2/core This commit removes unnecesary requires of http and internal/util in http2/core.js Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/20422 Reviewed-By: Anatoli Papirovski Reviewed-By: Tiancheng "Timothy" Gu Reviewed-By: Luigi Pinca Reviewed-By: Trivikram Kamat --- lib/internal/http2/core.js | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/lib/internal/http2/core.js b/lib/internal/http2/core.js index 5fa8e87ddbd04b..a42e416851af32 100644 --- a/lib/internal/http2/core.js +++ b/lib/internal/http2/core.js @@ -23,9 +23,12 @@ const { onServerStream, Http2ServerResponse, } = require('internal/http2/compat'); const { utcDate } = require('internal/http'); -const { promisify } = require('internal/util'); +const { + promisify, + customInspectSymbol: kInspect +} = require('internal/util'); const { isArrayBufferView } = require('internal/util/types'); -const { _connectionListener: httpConnectionListener } = require('http'); +const { _connectionListener: httpConnectionListener } = http; const { createPromise, promiseResolve } = process.binding('util'); const debug = util.debuglog('http2'); @@ -67,7 +70,6 @@ const { constants, nameForErrorCode } = binding; const NETServer = net.Server; const TLSServer = tls.Server; -const kInspect = require('internal/util').customInspectSymbol; const { kIncomingMessage } = require('_http_common'); const { kServerResponse } = require('_http_server'); From 395ce845da8dd5af2bf97baab67f5810adb051d7 Mon Sep 17 00:00:00 2001 From: Daniel Bevenius Date: Mon, 30 Apr 2018 15:12:57 +0200 Subject: [PATCH 037/129] http2: rename http2_state class to Http2State This commit renames the http2_state class to follow the guidelines in CPP_STYLE_GUIDE.md. Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/20423 Reviewed-By: Anna Henningsen --- src/env-inl.h | 4 ++-- src/env.h | 6 +++--- src/node_http2.cc | 2 +- src/node_http2_state.h | 4 ++-- 4 files changed, 8 insertions(+), 8 deletions(-) diff --git a/src/env-inl.h b/src/env-inl.h index bdf3e8ae453f9f..e30370633fd3da 100644 --- a/src/env-inl.h +++ b/src/env-inl.h @@ -417,12 +417,12 @@ inline void Environment::set_http_parser_buffer(char* buffer) { http_parser_buffer_ = buffer; } -inline http2::http2_state* Environment::http2_state() const { +inline http2::Http2State* Environment::http2_state() const { return http2_state_.get(); } inline void Environment::set_http2_state( - std::unique_ptr buffer) { + std::unique_ptr buffer) { CHECK(!http2_state_); // Should be set only once. http2_state_ = std::move(buffer); } diff --git a/src/env.h b/src/env.h index 7b4a6999755bcf..c098ca1d2b9a9d 100644 --- a/src/env.h +++ b/src/env.h @@ -617,8 +617,8 @@ class Environment { inline char* http_parser_buffer() const; inline void set_http_parser_buffer(char* buffer); - inline http2::http2_state* http2_state() const; - inline void set_http2_state(std::unique_ptr state); + inline http2::Http2State* http2_state() const; + inline void set_http2_state(std::unique_ptr state); inline double* fs_stats_field_array() const; inline void set_fs_stats_field_array(double* fields); @@ -759,7 +759,7 @@ class Environment { double* heap_space_statistics_buffer_ = nullptr; char* http_parser_buffer_; - std::unique_ptr http2_state_; + std::unique_ptr http2_state_; double* fs_stats_field_array_; diff --git a/src/node_http2.cc b/src/node_http2.cc index 2744c8bb43df51..237e5c01b91696 100644 --- a/src/node_http2.cc +++ b/src/node_http2.cc @@ -2868,7 +2868,7 @@ void Initialize(Local target, Isolate* isolate = env->isolate(); HandleScope scope(isolate); - std::unique_ptr state(new http2_state(isolate)); + std::unique_ptr state(new Http2State(isolate)); #define SET_STATE_TYPEDARRAY(name, field) \ target->Set(context, \ diff --git a/src/node_http2_state.h b/src/node_http2_state.h index ed88f068a04b16..64a0942f7ffa67 100644 --- a/src/node_http2_state.h +++ b/src/node_http2_state.h @@ -84,9 +84,9 @@ namespace http2 { IDX_SESSION_STATS_COUNT }; -class http2_state { +class Http2State { public: - explicit http2_state(v8::Isolate* isolate) : + explicit Http2State(v8::Isolate* isolate) : root_buffer( isolate, sizeof(http2_state_internal)), From 448922d0debaf71134665966bb6d6a427e1b4f08 Mon Sep 17 00:00:00 2001 From: Ujjwal Sharma Date: Sun, 6 May 2018 00:31:03 +0530 Subject: [PATCH 038/129] doc: add parameters for Http2Session:stream event Add parameters for the callback for the Http2Session:stream event inline with the pattern in the rest of the documentation. Refs: https://github.com/nodejs/help/issues/877#issuecomment-381253464 Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/20547 Reviewed-By: Anna Henningsen Reviewed-By: Vse Mozhet Byt Reviewed-By: Matteo Collina Reviewed-By: Trivikram Kamat --- doc/api/http2.md | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/doc/api/http2.md b/doc/api/http2.md index c2fa1de02288a7..0db1e044cac2e4 100644 --- a/doc/api/http2.md +++ b/doc/api/http2.md @@ -233,10 +233,13 @@ session.on('remoteSettings', (settings) => { added: v8.4.0 --> -The `'stream'` event is emitted when a new `Http2Stream` is created. When -invoked, the handler function will receive a reference to the `Http2Stream` -object, a [HTTP/2 Headers Object][], and numeric flags associated with the -creation of the stream. +* `stream` {Http2Stream} A reference to the stream +* `headers` {HTTP/2 Headers Object} An object describing the headers +* `flags` {number} The associated numeric flags +* `rawHeaders` {Array} An array containing the raw header names followed by + their respective values. + +The `'stream'` event is emitted when a new `Http2Stream` is created. ```js const http2 = require('http2'); From c30a8f468dab583c9ee5acfd998bd2511f78d6a6 Mon Sep 17 00:00:00 2001 From: Anatoli Papirovski Date: Sun, 6 May 2018 08:01:02 +0200 Subject: [PATCH 039/129] test: fix flaky http2-flow-control test Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/20556 Reviewed-By: James M Snell Reviewed-By: Trivikram Kamat Reviewed-By: Matteo Collina --- test/parallel/test-http2-misbehaving-flow-control-paused.js | 2 -- 1 file changed, 2 deletions(-) diff --git a/test/parallel/test-http2-misbehaving-flow-control-paused.js b/test/parallel/test-http2-misbehaving-flow-control-paused.js index 60a2cdabf847d9..26d2ed5dd244a2 100644 --- a/test/parallel/test-http2-misbehaving-flow-control-paused.js +++ b/test/parallel/test-http2-misbehaving-flow-control-paused.js @@ -70,8 +70,6 @@ server.on('stream', (stream) => { client.destroy(); })); stream.on('end', common.mustNotCall()); - stream.respond(); - stream.end('ok'); }); server.listen(0, () => { From 77acef4af27eea3064e49c1c2d8f8d9216c20b26 Mon Sep 17 00:00:00 2001 From: Ujjwal Sharma Date: Tue, 8 May 2018 17:59:14 +0530 Subject: [PATCH 040/129] doc: add params for ClientHttp2Session:altsvc Add parameters for the callback for the ClientHttp2Session:altsvc event inline with the pattern in the rest of the documentation. Refs: https://github.com/nodejs/help/issues/877#issuecomment-381253464 Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/20598 Reviewed-By: Vse Mozhet Byt Reviewed-By: James M Snell Reviewed-By: Luigi Pinca Reviewed-By: Tiancheng "Timothy" Gu Reviewed-By: Trivikram Kamat Reviewed-By: Colin Ihrig --- doc/api/http2.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/doc/api/http2.md b/doc/api/http2.md index 0db1e044cac2e4..bb8bc8dc461a10 100644 --- a/doc/api/http2.md +++ b/doc/api/http2.md @@ -667,9 +667,9 @@ added: v8.4.0 added: v8.11.2 --> -* `alt`: {string} -* `origin`: {string} -* `streamId`: {number} +* `alt` {string} +* `origin` {string} +* `streamId` {number} The `'altsvc'` event is emitted whenever an `ALTSVC` frame is received by the client. The event is emitted with the `ALTSVC` value, origin, and stream From 4f0035485f9f2a52526bbeaf9662b9071af9fbf4 Mon Sep 17 00:00:00 2001 From: Ujjwal Sharma Date: Wed, 9 May 2018 01:16:57 +0530 Subject: [PATCH 041/129] doc: add parameters for Http2Stream:error event Add parameters for the callback for the Http2Stream:error event inline with the pattern in the rest of the documentation. Refs: https://github.com/nodejs/help/issues/877#issuecomment-381253464 Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/20610 Reviewed-By: Vse Mozhet Byt Reviewed-By: Trivikram Kamat Reviewed-By: Colin Ihrig --- doc/api/http2.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/api/http2.md b/doc/api/http2.md index bb8bc8dc461a10..7c162d1062d2c8 100644 --- a/doc/api/http2.md +++ b/doc/api/http2.md @@ -850,6 +850,8 @@ code specified when closing the stream. If the code is any value other than added: v8.4.0 --> +* `error` {Error} + The `'error'` event is emitted when an error occurs during the processing of an `Http2Stream`. From d1b78252b18a212e49f49bb03bc5ebbd61e88ff5 Mon Sep 17 00:00:00 2001 From: Robert Nagy Date: Sat, 28 Apr 2018 00:34:52 +0200 Subject: [PATCH 042/129] http2: avoid bind and properly clean up in compat Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/20374 Reviewed-By: Anatoli Papirovski Reviewed-By: James M Snell Reviewed-By: Trivikram Kamat --- lib/internal/http2/compat.js | 78 +++++++++++++++++++++--------------- 1 file changed, 46 insertions(+), 32 deletions(-) diff --git a/lib/internal/http2/compat.js b/lib/internal/http2/compat.js index 02b595b6a023f8..009bc7a2ac7ab6 100644 --- a/lib/internal/http2/compat.js +++ b/lib/internal/http2/compat.js @@ -7,7 +7,6 @@ const constants = binding.constants; const errors = require('internal/errors'); const { kSocket } = require('internal/http2/util'); -const kFinish = Symbol('finish'); const kBeginSend = Symbol('begin-send'); const kState = Symbol('state'); const kStream = Symbol('stream'); @@ -211,6 +210,27 @@ const proxySocketHandler = { } }; +function onStreamCloseRequest() { + const req = this[kRequest]; + + if (req === undefined) + return; + + const state = req[kState]; + state.closed = true; + + req.push(null); + // if the user didn't interact with incoming data and didn't pipe it, + // dump it for compatibility with http1 + if (!state.didRead && !req._readableState.resumeScheduled) + req.resume(); + + this[kProxySocket] = null; + this[kRequest] = undefined; + + req.emit('close'); +} + class Http2ServerRequest extends Readable { constructor(stream, headers, options, rawHeaders) { super(options); @@ -234,7 +254,7 @@ class Http2ServerRequest extends Readable { stream.on('end', onStreamEnd); stream.on('error', onStreamError); stream.on('aborted', onStreamAbortedRequest); - stream.on('close', this[kFinish].bind(this)); + stream.on('close', onStreamCloseRequest); this.on('pause', onRequestPause); this.on('resume', onRequestResume); } @@ -335,24 +355,30 @@ class Http2ServerRequest extends Readable { return; this[kStream].setTimeout(msecs, callback); } - - [kFinish]() { - const state = this[kState]; - if (state.closed) - return; - state.closed = true; - this.push(null); - this[kStream][kRequest] = undefined; - // if the user didn't interact with incoming data and didn't pipe it, - // dump it for compatibility with http1 - if (!state.didRead && !this._readableState.resumeScheduled) - this.resume(); - this.emit('close'); - } } function onStreamTrailersReady() { - this[kStream].sendTrailers(this[kTrailers]); + this.sendTrailers(this[kResponse][kTrailers]); +} + +function onStreamCloseResponse() { + const res = this[kResponse]; + + if (res === undefined) + return; + + const state = res[kState]; + + if (this.headRequest !== state.headRequest) + return; + + state.closed = true; + + this[kProxySocket] = null; + this[kResponse] = undefined; + + res.emit('finish'); + res.emit('close'); } class Http2ServerResponse extends Stream { @@ -373,8 +399,8 @@ class Http2ServerResponse extends Stream { this.writable = true; stream.on('drain', onStreamDrain); stream.on('aborted', onStreamAbortedResponse); - stream.on('close', this[kFinish].bind(this)); - stream.on('wantTrailers', onStreamTrailersReady.bind(this)); + stream.on('close', onStreamCloseResponse); + stream.on('wantTrailers', onStreamTrailersReady); } // User land modules such as finalhandler just check truthiness of this @@ -605,7 +631,7 @@ class Http2ServerResponse extends Stream { this.writeHead(this[kState].statusCode); if (isFinished) - this[kFinish](); + onStreamCloseResponse.call(stream); else stream.end(); } @@ -649,18 +675,6 @@ class Http2ServerResponse extends Stream { this[kStream].respond(headers, options); } - [kFinish]() { - const stream = this[kStream]; - const state = this[kState]; - if (state.closed || stream.headRequest !== state.headRequest) - return; - state.closed = true; - this[kProxySocket] = null; - stream[kResponse] = undefined; - this.emit('finish'); - this.emit('close'); - } - // TODO doesn't support callbacks writeContinue() { const stream = this[kStream]; From e5f8b0830585c80108f1bcd995b986b364a684b7 Mon Sep 17 00:00:00 2001 From: Rich Trott Date: Thu, 10 May 2018 20:45:44 -0700 Subject: [PATCH 043/129] test: improve reliability of http2-session-timeout Check actual expired time rather than relying on a number of calls to setTimeout() in test-http2-session-timeout more robust. Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/20692 Fixes: https://github.com/nodejs/node/issues/20628 Reviewed-By: Anatoli Papirovski Reviewed-By: Trivikram Kamat Reviewed-By: Ruben Bridgewater Reviewed-By: James M Snell --- test/sequential/test-http2-session-timeout.js | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/test/sequential/test-http2-session-timeout.js b/test/sequential/test-http2-session-timeout.js index fce4570563c584..48e98998c700b6 100644 --- a/test/sequential/test-http2-session-timeout.js +++ b/test/sequential/test-http2-session-timeout.js @@ -7,7 +7,6 @@ const h2 = require('http2'); const serverTimeout = common.platformTimeout(200); const callTimeout = common.platformTimeout(20); -const minRuns = Math.ceil(serverTimeout / callTimeout) * 2; const mustNotCall = common.mustNotCall(); const server = h2.createServer(); @@ -21,9 +20,10 @@ server.listen(0, common.mustCall(() => { const url = `http://localhost:${port}`; const client = h2.connect(url); - makeReq(minRuns); + const startTime = process.hrtime(); + makeReq(); - function makeReq(attempts) { + function makeReq() { const request = client.request({ ':path': '/foobar', ':method': 'GET', @@ -34,12 +34,14 @@ server.listen(0, common.mustCall(() => { request.end(); request.on('end', () => { - if (attempts) { - setTimeout(() => makeReq(attempts - 1), callTimeout); + const diff = process.hrtime(startTime); + const milliseconds = (diff[0] * 1e3 + diff[1] / 1e6); + if (milliseconds < serverTimeout * 2) { + setTimeout(makeReq, callTimeout); } else { server.removeListener('timeout', mustNotCall); - client.close(); server.close(); + client.close(); } }); } From b0c92cadfa25b2fe07778f3829084e0693fff843 Mon Sep 17 00:00:00 2001 From: Anatoli Papirovski Date: Wed, 9 May 2018 12:12:43 +0200 Subject: [PATCH 044/129] http2: fix end without read Adjust http2 behaviour to allow ending a stream even after some data comes in (when the user has no intention of reading that data). Also correctly end a stream when trailers are present. Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/20621 Fixes: https://github.com/nodejs/node/issues/20060 Reviewed-By: James M Snell Reviewed-By: Matteo Collina Reviewed-By: Trivikram Kamat --- lib/internal/http2/compat.js | 10 +++-- lib/internal/http2/core.js | 12 ++--- .../test-http2-client-upload-reject.js | 15 ++++--- .../test-http2-compat-client-upload-reject.js | 44 +++++++++++++++++++ 4 files changed, 66 insertions(+), 15 deletions(-) create mode 100644 test/parallel/test-http2-compat-client-upload-reject.js diff --git a/lib/internal/http2/compat.js b/lib/internal/http2/compat.js index 009bc7a2ac7ab6..9b2367c4c97a57 100644 --- a/lib/internal/http2/compat.js +++ b/lib/internal/http2/compat.js @@ -248,8 +248,6 @@ class Http2ServerRequest extends Readable { stream[kRequest] = this; // Pause the stream.. - stream.pause(); - stream.on('data', onStreamData); stream.on('trailers', onStreamTrailers); stream.on('end', onStreamEnd); stream.on('error', onStreamError); @@ -316,8 +314,12 @@ class Http2ServerRequest extends Readable { _read(nread) { const state = this[kState]; if (!state.closed) { - state.didRead = true; - process.nextTick(resumeStream, this[kStream]); + if (!state.didRead) { + state.didRead = true; + this[kStream].on('data', onStreamData); + } else { + process.nextTick(resumeStream, this[kStream]); + } } else { this.emit('error', new errors.Error('ERR_HTTP2_INVALID_STREAM')); } diff --git a/lib/internal/http2/core.js b/lib/internal/http2/core.js index a42e416851af32..5d70f436bbc676 100644 --- a/lib/internal/http2/core.js +++ b/lib/internal/http2/core.js @@ -300,11 +300,11 @@ function onStreamClose(code) { // Push a null so the stream can end whenever the client consumes // it completely. stream.push(null); - - // Same as net. - if (stream._readableState.length === 0) { - stream.read(0); - } + // If the client hasn't tried to consume the stream and there is no + // resume scheduled (which would indicate they would consume in the future), + // then just dump the incoming data so that the stream can be destroyed. + if (!stream[kState].didRead && !stream._readableState.resumeScheduled) + stream.resume(); } } @@ -1789,6 +1789,8 @@ class Http2Stream extends Duplex { const ret = this[kHandle].trailers(headersList); if (ret < 0) this.destroy(new NghttpError(ret)); + else + this[kMaybeDestroy](); } get closed() { diff --git a/test/parallel/test-http2-client-upload-reject.js b/test/parallel/test-http2-client-upload-reject.js index ece7cbdf233f1f..678114130e3dba 100644 --- a/test/parallel/test-http2-client-upload-reject.js +++ b/test/parallel/test-http2-client-upload-reject.js @@ -20,12 +20,15 @@ fs.readFile(loc, common.mustCall((err, data) => { const server = http2.createServer(); server.on('stream', common.mustCall((stream) => { - stream.on('close', common.mustCall(() => { - assert.strictEqual(stream.rstCode, 0); - })); - - stream.respond({ ':status': 400 }); - stream.end(); + // Wait for some data to come through. + setImmediate(() => { + stream.on('close', common.mustCall(() => { + assert.strictEqual(stream.rstCode, 0); + })); + + stream.respond({ ':status': 400 }); + stream.end(); + }); })); server.listen(0, common.mustCall(() => { diff --git a/test/parallel/test-http2-compat-client-upload-reject.js b/test/parallel/test-http2-compat-client-upload-reject.js new file mode 100644 index 00000000000000..e6a187cb12b264 --- /dev/null +++ b/test/parallel/test-http2-compat-client-upload-reject.js @@ -0,0 +1,44 @@ +'use strict'; + +// Verifies that uploading data from a client works + +const common = require('../common'); +if (!common.hasCrypto) + common.skip('missing crypto'); +const assert = require('assert'); +const http2 = require('http2'); +const fs = require('fs'); +const fixtures = require('../common/fixtures'); + +const loc = fixtures.path('person-large.jpg'); + +assert(fs.existsSync(loc)); + +fs.readFile(loc, common.mustCall((err, data) => { + assert.ifError(err); + + const server = http2.createServer(common.mustCall((req, res) => { + setImmediate(() => { + res.writeHead(400); + res.end(); + }); + })); + + server.listen(0, common.mustCall(() => { + const client = http2.connect(`http://localhost:${server.address().port}`); + + const req = client.request({ ':method': 'POST' }); + req.on('response', common.mustCall((headers) => { + assert.strictEqual(headers[':status'], 400); + })); + + req.resume(); + req.on('end', common.mustCall(() => { + server.close(); + client.close(); + })); + + const str = fs.createReadStream(loc); + str.pipe(req); + })); +})); From b0e3ce9c4b7ec73267b93f0f631152ee92dd5f41 Mon Sep 17 00:00:00 2001 From: Ujjwal Sharma Date: Thu, 10 May 2018 02:47:56 +0530 Subject: [PATCH 045/129] net,http2: refactor _write and _writev Refactor writable part (the _write and _writev functions) in net.Socket and http2.Http2Stream classes. Also involves adding a generic "WriteGeneric" method to the Http2Stream class based on net.Socket._writeGeneric, but behind a symbol. Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/20643 Reviewed-By: Anatoli Papirovski Reviewed-By: Ruben Bridgewater --- lib/internal/http2/core.js | 67 ++++++++++++++------------------------ 1 file changed, 24 insertions(+), 43 deletions(-) diff --git a/lib/internal/http2/core.js b/lib/internal/http2/core.js index 5d70f436bbc676..106991dbd4224d 100644 --- a/lib/internal/http2/core.js +++ b/lib/internal/http2/core.js @@ -95,6 +95,7 @@ const kSession = Symbol('session'); const kState = Symbol('state'); const kType = Symbol('type'); const kUpdateTimer = Symbol('update-timer'); +const kWriteGeneric = Symbol('write-generic'); const kDefaultSocketTimeout = 2 * 60 * 1000; @@ -1635,13 +1636,16 @@ class Http2Stream extends Duplex { 'bug in Node.js'); } - _write(data, encoding, cb) { + [kWriteGeneric](writev, data, encoding, cb) { // When the Http2Stream is first created, it is corked until the // handle and the stream ID is assigned. However, if the user calls // uncork() before that happens, the Duplex will attempt to pass // writes through. Those need to be queued up here. if (this.pending) { - this.once('ready', this._write.bind(this, data, encoding, cb)); + this.once( + 'ready', + this[kWriteGeneric].bind(this, writev, data, encoding, cb) + ); return; } @@ -1665,53 +1669,30 @@ class Http2Stream extends Duplex { req.callback = cb; req.oncomplete = afterDoStreamWrite; req.async = false; - const err = createWriteReq(req, handle, data, encoding); + + let err; + if (writev) { + const chunks = new Array(data.length << 1); + for (var i = 0; i < data.length; i++) { + const entry = data[i]; + chunks[i * 2] = entry.chunk; + chunks[i * 2 + 1] = entry.encoding; + } + err = handle.writev(req, chunks); + } else { + err = createWriteReq(req, handle, data, encoding); + } if (err) return this.destroy(errors.errnoException(err, 'write', req.error), cb); trackWriteState(this, req.bytes); } - _writev(data, cb) { - // When the Http2Stream is first created, it is corked until the - // handle and the stream ID is assigned. However, if the user calls - // uncork() before that happens, the Duplex will attempt to pass - // writes through. Those need to be queued up here. - if (this.pending) { - this.once('ready', this._writev.bind(this, data, cb)); - return; - } - - // If the stream has been destroyed, there's nothing else we can do - // because the handle has been destroyed. This should only be an - // issue if a write occurs before the 'ready' event in the case where - // the duplex is uncorked before the stream is ready to go. In that - // case, drop the data on the floor. An error should have already been - // emitted. - if (this.destroyed) - return; - - this[kUpdateTimer](); - - if (!this.headersSent) - this[kProceed](); + _write(data, encoding, cb) { + this[kWriteGeneric](false, data, encoding, cb); + } - const handle = this[kHandle]; - const req = new WriteWrap(); - req.stream = this[kID]; - req.handle = handle; - req.callback = cb; - req.oncomplete = afterDoStreamWrite; - req.async = false; - const chunks = new Array(data.length << 1); - for (var i = 0; i < data.length; i++) { - const entry = data[i]; - chunks[i * 2] = entry.chunk; - chunks[i * 2 + 1] = entry.encoding; - } - const err = handle.writev(req, chunks); - if (err) - return this.destroy(errors.errnoException(err, 'write', req.error), cb); - trackWriteState(this, req.bytes); + _writev(data, cb) { + this[kWriteGeneric](true, data, '', cb); } _final(cb) { From 11a63ddf4885cae4bfd6995f945d8fdd963b97bc Mon Sep 17 00:00:00 2001 From: Keita Akutsu Date: Sun, 20 May 2018 14:01:44 +0900 Subject: [PATCH 046/129] doc: fix typo in http2.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/20843 Reviewed-By: Colin Ihrig Reviewed-By: Trivikram Kamat Reviewed-By: Luigi Pinca Reviewed-By: Vse Mozhet Byt Reviewed-By: Tobias Nießen --- doc/api/http2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/api/http2.md b/doc/api/http2.md index 7c162d1062d2c8..479b6bb48310ee 100644 --- a/doc/api/http2.md +++ b/doc/api/http2.md @@ -3123,7 +3123,7 @@ added: v8.4.0 --> Call [`http2stream.pushStream()`][] with the given headers, and wraps the -given newly created [`Http2Stream`] on `Http2ServerRespose`. +given newly created [`Http2Stream`] on `Http2ServerResponse`. The callback will be called with an error with code `ERR_HTTP2_STREAM_CLOSED` if the stream is closed. From 0a6672fbcf7bd41585d9b9b2436d3797674a4048 Mon Sep 17 00:00:00 2001 From: Anatoli Papirovski Date: Thu, 17 May 2018 23:03:15 +0400 Subject: [PATCH 047/129] http2: fix several serious bugs Currently http2 does not properly submit GOAWAY frames when a session is being destroyed. It also doesn't properly handle when the other party severs the connection after sending a GOAWAY frame, even though it should. Edge, IE & Safari are currently unable to handle empty TRAILERS frames despite them being correctly to spec. Instead send an empty DATA frame with END_STREAM flag in those situations. Fix and adjust several flaky and/or incorrect tests. Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/20772 Fixes: https://github.com/nodejs/node/issues/20705 Fixes: https://github.com/nodejs/node/issues/20750 Fixes: https://github.com/nodejs/node/issues/20850 Reviewed-By: Matteo Collina Reviewed-By: Rich Trott Reviewed-By: James M Snell --- doc/api/http2.md | 4 + lib/internal/http2/core.js | 109 +++++++++++------- src/node_http2.cc | 50 +++++--- src/node_http2.h | 8 +- test/parallel/test-http2-client-destroy.js | 8 +- .../test-http2-no-wanttrailers-listener.js | 5 +- .../test-http2-server-close-callback.js | 19 +-- .../test-http2-server-sessionerror.js | 13 ++- ...st-http2-server-shutdown-options-errors.js | 6 - .../test-http2-server-socket-destroy.js | 10 +- ...est-http2-server-stream-session-destroy.js | 8 -- test/parallel/test-http2-server-timeout.js | 10 +- test/parallel/test-http2-too-many-settings.js | 7 +- test/parallel/test-http2-trailers.js | 3 +- 14 files changed, 145 insertions(+), 115 deletions(-) diff --git a/doc/api/http2.md b/doc/api/http2.md index 479b6bb48310ee..3ffb41e1402ca2 100644 --- a/doc/api/http2.md +++ b/doc/api/http2.md @@ -884,6 +884,10 @@ The `'trailers'` event is emitted when a block of headers associated with trailing header fields is received. The listener callback is passed the [HTTP/2 Headers Object][] and flags associated with the headers. +Note that this event might not be emitted if `http2stream.end()` is called +before trailers are received and the incoming data is not being read or +listened for. + ```js stream.on('trailers', (headers, flags) => { console.log(headers); diff --git a/lib/internal/http2/core.js b/lib/internal/http2/core.js index 106991dbd4224d..5a284a60771f26 100644 --- a/lib/internal/http2/core.js +++ b/lib/internal/http2/core.js @@ -292,20 +292,25 @@ function onStreamClose(code) { tryClose(stream[kState].fd); // Defer destroy we actually emit end. - if (stream._readableState.endEmitted || code !== NGHTTP2_NO_ERROR) { + if (!stream.readable || code !== NGHTTP2_NO_ERROR) { // If errored or ended, we can destroy immediately. - stream[kMaybeDestroy](null, code); + stream[kMaybeDestroy](code); } else { // Wait for end to destroy. stream.on('end', stream[kMaybeDestroy]); // Push a null so the stream can end whenever the client consumes // it completely. stream.push(null); - // If the client hasn't tried to consume the stream and there is no - // resume scheduled (which would indicate they would consume in the future), - // then just dump the incoming data so that the stream can be destroyed. - if (!stream[kState].didRead && !stream._readableState.resumeScheduled) + + // If the user hasn't tried to consume the stream (and this is a server + // session) then just dump the incoming data so that the stream can + // be destroyed. + if (stream[kSession][kType] === NGHTTP2_SESSION_SERVER && + !stream[kState].didRead && + stream._readableState.flowing === null) stream.resume(); + else + stream.read(0); } } @@ -330,7 +335,7 @@ function onStreamRead(nread, buf) { `${sessionName(stream[kSession][kType])}]: ending readable.`); // defer this until we actually emit end - if (stream._readableState.endEmitted) { + if (!stream.readable) { stream[kMaybeDestroy](); } else { stream.on('end', stream[kMaybeDestroy]); @@ -421,7 +426,7 @@ function onGoawayData(code, lastStreamID, buf) { // condition on this side of the session that caused the // shutdown. session.destroy(new errors.Error('ERR_HTTP2_SESSION_ERROR', code), - { errorCode: NGHTTP2_NO_ERROR }); + NGHTTP2_NO_ERROR); } } @@ -772,6 +777,21 @@ function emitClose(self, error) { self.emit('close'); } +function finishSessionDestroy(session, error) { + const socket = session[kSocket]; + if (!socket.destroyed) + socket.destroy(error); + + session[kProxySocket] = undefined; + session[kSocket] = undefined; + session[kHandle] = undefined; + socket[kSession] = undefined; + socket[kServer] = undefined; + + // Finally, emit the close and error events (if necessary) on next tick. + process.nextTick(emitClose, session, error); +} + // Upon creation, the Http2Session takes ownership of the socket. The session // may not be ready to use immediately if the socket is not yet fully connected. // In that case, the Http2Session will wait for the socket to connect. Once @@ -828,6 +848,8 @@ class Http2Session extends EventEmitter { this[kState] = { flags: SESSION_FLAGS_PENDING, + goawayCode: null, + goawayLastStreamID: null, streams: new Map(), pendingStreams: new Set(), pendingAck: 0, @@ -1130,25 +1152,13 @@ class Http2Session extends EventEmitter { if (handle !== undefined) handle.destroy(code, socket.destroyed); - // If there is no error, use setImmediate to destroy the socket on the + // If the socket is alive, use setImmediate to destroy the session on the // next iteration of the event loop in order to give data time to transmit. // Otherwise, destroy immediately. - if (!socket.destroyed) { - if (!error) { - setImmediate(socket.destroy.bind(socket)); - } else { - socket.destroy(error); - } - } - - this[kProxySocket] = undefined; - this[kSocket] = undefined; - this[kHandle] = undefined; - socket[kSession] = undefined; - socket[kServer] = undefined; - - // Finally, emit the close and error events (if necessary) on next tick. - process.nextTick(emitClose, this, error); + if (!socket.destroyed) + setImmediate(finishSessionDestroy, this, error); + else + finishSessionDestroy(this, error); } // Closing the session will: @@ -1422,11 +1432,8 @@ function afterDoStreamWrite(status, handle, req) { } function streamOnResume() { - if (!this.destroyed && !this.pending) { - if (!this[kState].didRead) - this[kState].didRead = true; + if (!this.destroyed) this[kHandle].readStart(); - } } function streamOnPause() { @@ -1441,6 +1448,16 @@ function afterShutdown() { stream[kMaybeDestroy](); } +function finishSendTrailers(stream, headersList) { + stream[kState].flags &= ~STREAM_FLAGS_HAS_TRAILERS; + + const ret = stream[kHandle].trailers(headersList); + if (ret < 0) + stream.destroy(new NghttpError(ret)); + else + stream[kMaybeDestroy](); +} + function closeStream(stream, code, shouldSubmitRstStream = true) { const state = stream[kState]; state.flags |= STREAM_FLAGS_CLOSED; @@ -1502,6 +1519,10 @@ class Http2Stream extends Duplex { this[kSession] = session; session[kState].pendingStreams.add(this); + // Allow our logic for determining whether any reads have happened to + // work in all situations. This is similar to what we do in _http_incoming. + this._readableState.readingMore = true; + this[kState] = { didRead: false, flags: STREAM_FLAGS_PENDING, @@ -1510,7 +1531,6 @@ class Http2Stream extends Duplex { trailersReady: false }; - this.on('resume', streamOnResume); this.on('pause', streamOnPause); } @@ -1717,6 +1737,10 @@ class Http2Stream extends Duplex { this.push(null); return; } + if (!this[kState].didRead) { + this._readableState.readingMore = false; + this[kState].didRead = true; + } if (!this.pending) { streamOnResume.call(this); } else { @@ -1765,13 +1789,8 @@ class Http2Stream extends Duplex { throw headersList; this[kSentTrailers] = headers; - this[kState].flags &= ~STREAM_FLAGS_HAS_TRAILERS; - - const ret = this[kHandle].trailers(headersList); - if (ret < 0) - this.destroy(new NghttpError(ret)); - else - this[kMaybeDestroy](); + // Send the trailers in setImmediate so we don't do it on nghttp2 stack. + setImmediate(finishSendTrailers, this, headersList); } get closed() { @@ -1861,15 +1880,15 @@ class Http2Stream extends Duplex { } // The Http2Stream can be destroyed if it has closed and if the readable // side has received the final chunk. - [kMaybeDestroy](error, code = NGHTTP2_NO_ERROR) { - if (error || code !== NGHTTP2_NO_ERROR) { - this.destroy(error); + [kMaybeDestroy](code = NGHTTP2_NO_ERROR) { + if (code !== NGHTTP2_NO_ERROR) { + this.destroy(); return; } // TODO(mcollina): remove usage of _*State properties - if (this._writableState.ended && this._writableState.pendingcb === 0) { - if (this._readableState.ended && this.closed) { + if (!this.writable) { + if (!this.readable && this.closed) { this.destroy(); return; } @@ -1882,7 +1901,7 @@ class Http2Stream extends Duplex { this[kSession][kType] === NGHTTP2_SESSION_SERVER && !(state.flags & STREAM_FLAGS_HAS_TRAILERS) && !state.didRead && - !this._readableState.resumeScheduled) { + this._readableState.flowing === null) { this.close(); } } @@ -2445,6 +2464,10 @@ Object.defineProperty(Http2Session.prototype, 'setTimeout', setTimeout); function socketOnError(error) { const session = this[kSession]; if (session !== undefined) { + // We can ignore ECONNRESET after GOAWAY was received as there's nothing + // we can do and the other side is fully within its rights to do so. + if (error.code === 'ECONNRESET' && session[kState].goawayCode !== null) + return session.destroy(); debug(`Http2Session ${sessionName(session[kType])}: socket error [` + `${error.message}]`); session.destroy(error); diff --git a/src/node_http2.cc b/src/node_http2.cc index 237e5c01b91696..4630e1228307c6 100644 --- a/src/node_http2.cc +++ b/src/node_http2.cc @@ -631,9 +631,9 @@ inline void Http2Session::EmitStatistics() { void Http2Session::Close(uint32_t code, bool socket_closed) { DEBUG_HTTP2SESSION(this, "closing session"); - if (flags_ & SESSION_STATE_CLOSED) + if (flags_ & SESSION_STATE_CLOSING) return; - flags_ |= SESSION_STATE_CLOSED; + flags_ |= SESSION_STATE_CLOSING; // Stop reading on the i/o stream if (stream_ != nullptr) @@ -641,16 +641,18 @@ void Http2Session::Close(uint32_t code, bool socket_closed) { // If the socket is not closed, then attempt to send a closing GOAWAY // frame. There is no guarantee that this GOAWAY will be received by - // the peer but the HTTP/2 spec recommends sendinng it anyway. We'll + // the peer but the HTTP/2 spec recommends sending it anyway. We'll // make a best effort. if (!socket_closed) { - Http2Scope h2scope(this); DEBUG_HTTP2SESSION2(this, "terminating session with code %d", code); CHECK_EQ(nghttp2_session_terminate_session(session_, code), 0); + SendPendingData(); } else { Unconsume(); } + flags_ |= SESSION_STATE_CLOSED; + // If there are outstanding pings, those will need to be canceled, do // so on the next iteration of the event loop to avoid calling out into // javascript since this may be called during garbage collection. @@ -1387,25 +1389,32 @@ void Http2Session::MaybeScheduleWrite() { } } +void Http2Session::MaybeStopReading() { + int want_read = nghttp2_session_want_read(session_); + DEBUG_HTTP2SESSION2(this, "wants read? %d", want_read); + if (want_read == 0) + stream_->ReadStop(); +} + // Unset the sending state, finish up all current writes, and reset // storage for data and metadata that was associated with these writes. void Http2Session::ClearOutgoing(int status) { CHECK_NE(flags_ & SESSION_STATE_SENDING, 0); + flags_ &= ~SESSION_STATE_SENDING; + if (outgoing_buffers_.size() > 0) { outgoing_storage_.clear(); - for (const nghttp2_stream_write& wr : outgoing_buffers_) { + std::vector current_outgoing_buffers_; + current_outgoing_buffers_.swap(outgoing_buffers_); + for (const nghttp2_stream_write& wr : current_outgoing_buffers_) { WriteWrap* wrap = wr.req_wrap; if (wrap != nullptr) wrap->Done(status); } - - outgoing_buffers_.clear(); } - flags_ &= ~SESSION_STATE_SENDING; - // Now that we've finished sending queued data, if there are any pending // RstStreams we should try sending again and then flush them one by one. if (pending_rst_streams_.size() > 0) { @@ -1525,8 +1534,7 @@ uint8_t Http2Session::SendPendingData() { req->Dispose(); } - DEBUG_HTTP2SESSION2(this, "wants data in return? %d", - nghttp2_session_want_read(session_)); + MaybeStopReading(); return 0; } @@ -1691,8 +1699,7 @@ void Http2Session::OnStreamReadImpl(ssize_t nread, }; session->MakeCallback(env->error_string(), arraysize(argv), argv); } else { - DEBUG_HTTP2SESSION2(session, "processed %d bytes. wants more? %d", ret, - nghttp2_session_want_read(**session)); + session->MaybeStopReading(); } } @@ -1928,6 +1935,7 @@ void Http2Stream::OnTrailers() { HandleScope scope(isolate); Local context = env()->context(); Context::Scope context_scope(context); + flags_ &= ~NGHTTP2_STREAM_FLAG_TRAILERS; MakeCallback(env()->ontrailers_string(), 0, nullptr); } @@ -1936,7 +1944,16 @@ int Http2Stream::SubmitTrailers(nghttp2_nv* nva, size_t len) { CHECK(!this->IsDestroyed()); Http2Scope h2scope(this); DEBUG_HTTP2STREAM2(this, "sending %d trailers", len); - int ret = nghttp2_submit_trailer(**session_, id_, nva, len); + int ret; + // Sending an empty trailers frame poses problems in Safari, Edge & IE. + // Instead we can just send an empty data frame with NGHTTP2_FLAG_END_STREAM + // to indicate that the stream is ready to be closed. + if (len == 0) { + Http2Stream::Provider::Stream prov(this, 0); + ret = nghttp2_submit_data(**session_, NGHTTP2_FLAG_END_STREAM, id_, *prov); + } else { + ret = nghttp2_submit_trailer(**session_, id_, nva, len); + } CHECK_NE(ret, NGHTTP2_ERR_NOMEM); return ret; } @@ -2562,8 +2579,7 @@ void Http2Stream::Info(const FunctionCallbackInfo& args) { Headers list(isolate, context, headers); args.GetReturnValue().Set(stream->SubmitInfo(*list, list.length())); - DEBUG_HTTP2STREAM2(stream, "%d informational headers sent", - headers->Length()); + DEBUG_HTTP2STREAM2(stream, "%d informational headers sent", list.length()); } // Submits trailing headers on the Http2Stream @@ -2578,7 +2594,7 @@ void Http2Stream::Trailers(const FunctionCallbackInfo& args) { Headers list(isolate, context, headers); args.GetReturnValue().Set(stream->SubmitTrailers(*list, list.length())); - DEBUG_HTTP2STREAM2(stream, "%d trailing headers sent", headers->Length()); + DEBUG_HTTP2STREAM2(stream, "%d trailing headers sent", list.length()); } // Grab the numeric id of the Http2Stream diff --git a/src/node_http2.h b/src/node_http2.h index 972c08a2cd98ba..009c05b731eee8 100644 --- a/src/node_http2.h +++ b/src/node_http2.h @@ -433,7 +433,8 @@ enum session_state_flags { SESSION_STATE_HAS_SCOPE = 0x1, SESSION_STATE_WRITE_SCHEDULED = 0x2, SESSION_STATE_CLOSED = 0x4, - SESSION_STATE_SENDING = 0x8, + SESSION_STATE_CLOSING = 0x8, + SESSION_STATE_SENDING = 0x10, }; // This allows for 4 default-sized frames with their frame headers @@ -624,7 +625,7 @@ class Http2Stream : public AsyncWrap, inline bool IsClosed() const { return flags_ & NGHTTP2_STREAM_FLAG_CLOSED; - } + } inline bool HasTrailers() const { return flags_ & NGHTTP2_STREAM_FLAG_TRAILERS; @@ -846,6 +847,9 @@ class Http2Session : public AsyncWrap { // Schedule a write if nghttp2 indicates it wants to write to the socket. void MaybeScheduleWrite(); + // Stop reading if nghttp2 doesn't want to anymore. + void MaybeStopReading(); + // Returns pointer to the stream, or nullptr if stream does not exist inline Http2Stream* FindStream(int32_t id); diff --git a/test/parallel/test-http2-client-destroy.js b/test/parallel/test-http2-client-destroy.js index 6238363511a791..43fc6819e21f7a 100644 --- a/test/parallel/test-http2-client-destroy.js +++ b/test/parallel/test-http2-client-destroy.js @@ -109,9 +109,6 @@ const Countdown = require('../common/countdown'); server.listen(0, common.mustCall(() => { const client = h2.connect(`http://localhost:${server.address().port}`); - // On some platforms (e.g. windows), an ECONNRESET may occur at this - // point -- or it may not. Do not make this a mustCall - client.on('error', () => {}); client.on('close', () => { server.close(); @@ -119,10 +116,7 @@ const Countdown = require('../common/countdown'); client.destroy(); }); - const req = client.request(); - // On some platforms (e.g. windows), an ECONNRESET may occur at this - // point -- or it may not. Do not make this a mustCall - req.on('error', () => {}); + client.request(); })); } diff --git a/test/parallel/test-http2-no-wanttrailers-listener.js b/test/parallel/test-http2-no-wanttrailers-listener.js index 7ba25d0e491e15..87bc21df48aa2c 100644 --- a/test/parallel/test-http2-no-wanttrailers-listener.js +++ b/test/parallel/test-http2-no-wanttrailers-listener.js @@ -3,7 +3,6 @@ const common = require('../common'); if (!common.hasCrypto) common.skip('missing crypto'); -const assert = require('assert'); const h2 = require('http2'); const server = h2.createServer(); @@ -25,9 +24,7 @@ server.on('listening', common.mustCall(function() { const client = h2.connect(`http://localhost:${this.address().port}`); const req = client.request(); req.resume(); - req.on('trailers', common.mustCall((headers) => { - assert.strictEqual(Object.keys(headers).length, 0); - })); + req.on('trailers', common.mustNotCall()); req.on('close', common.mustCall(() => { server.close(); client.close(); diff --git a/test/parallel/test-http2-server-close-callback.js b/test/parallel/test-http2-server-close-callback.js index 66887aa62bebe5..f822d8a4a92d78 100644 --- a/test/parallel/test-http2-server-close-callback.js +++ b/test/parallel/test-http2-server-close-callback.js @@ -4,21 +4,24 @@ const common = require('../common'); if (!common.hasCrypto) common.skip('missing crypto'); +const Countdown = require('../common/countdown'); const http2 = require('http2'); const server = http2.createServer(); +let session; + +const countdown = new Countdown(2, () => { + server.close(common.mustCall()); + session.destroy(); +}); + server.listen(0, common.mustCall(() => { const client = http2.connect(`http://localhost:${server.address().port}`); - client.on('error', (err) => { - if (err.code !== 'ECONNRESET') - throw err; - }); + client.on('connect', common.mustCall(() => countdown.dec())); })); server.on('session', common.mustCall((s) => { - setImmediate(() => { - server.close(common.mustCall()); - s.destroy(); - }); + session = s; + countdown.dec(); })); diff --git a/test/parallel/test-http2-server-sessionerror.js b/test/parallel/test-http2-server-sessionerror.js index 525eb2e6efd11a..c50352fcc35c28 100644 --- a/test/parallel/test-http2-server-sessionerror.js +++ b/test/parallel/test-http2-server-sessionerror.js @@ -35,14 +35,17 @@ server.on('session', common.mustCall((session) => { server.listen(0, common.mustCall(() => { const url = `http://localhost:${server.address().port}`; http2.connect(url) - // An ECONNRESET error may occur depending on the platform (due largely - // to differences in the timing of socket closing). Do not wrap this in - // a common must call. - .on('error', () => {}) + .on('error', common.expectsError({ + code: 'ERR_HTTP2_SESSION_ERROR', + message: 'Session closed with error code 2', + })) .on('close', () => { server.removeAllListeners('error'); http2.connect(url) - .on('error', () => {}) + .on('error', common.expectsError({ + code: 'ERR_HTTP2_SESSION_ERROR', + message: 'Session closed with error code 2', + })) .on('close', () => server.close()); }); })); diff --git a/test/parallel/test-http2-server-shutdown-options-errors.js b/test/parallel/test-http2-server-shutdown-options-errors.js index 2aedec1140701a..94733b199366db 100644 --- a/test/parallel/test-http2-server-shutdown-options-errors.js +++ b/test/parallel/test-http2-server-shutdown-options-errors.js @@ -54,13 +54,7 @@ server.listen( 0, common.mustCall(() => { const client = http2.connect(`http://localhost:${server.address().port}`); - // On certain operating systems, an ECONNRESET may occur. We do not need - // to test for it here. Do not make this a mustCall - client.on('error', () => {}); const req = client.request(); - // On certain operating systems, an ECONNRESET may occur. We do not need - // to test for it here. Do not make this a mustCall - req.on('error', () => {}); req.resume(); req.on('close', common.mustCall(() => { client.close(); diff --git a/test/parallel/test-http2-server-socket-destroy.js b/test/parallel/test-http2-server-socket-destroy.js index 03afc1957b8af4..99595aeb63004d 100644 --- a/test/parallel/test-http2-server-socket-destroy.js +++ b/test/parallel/test-http2-server-socket-destroy.js @@ -41,14 +41,20 @@ server.on('listening', common.mustCall(() => { // The client may have an ECONNRESET error here depending on the operating // system, due mainly to differences in the timing of socket closing. Do // not wrap this in a common mustCall. - client.on('error', () => {}); + client.on('error', (err) => { + if (err.code !== 'ECONNRESET') + throw err; + }); client.on('close', common.mustCall()); const req = client.request({ ':method': 'POST' }); // The client may have an ECONNRESET error here depending on the operating // system, due mainly to differences in the timing of socket closing. Do // not wrap this in a common mustCall. - req.on('error', () => {}); + req.on('error', (err) => { + if (err.code !== 'ECONNRESET') + throw err; + }); req.on('aborted', common.mustCall()); req.resume(); diff --git a/test/parallel/test-http2-server-stream-session-destroy.js b/test/parallel/test-http2-server-stream-session-destroy.js index e70630fe0b1351..6d8de4ba5f7618 100644 --- a/test/parallel/test-http2-server-stream-session-destroy.js +++ b/test/parallel/test-http2-server-stream-session-destroy.js @@ -39,16 +39,8 @@ server.on('stream', common.mustCall((stream) => { server.listen(0, common.mustCall(() => { const client = h2.connect(`http://localhost:${server.address().port}`); - client.on('error', (err) => { - if (err.code !== 'ECONNRESET') - throw err; - }); const req = client.request(); req.resume(); req.on('end', common.mustCall()); req.on('close', common.mustCall(() => server.close(common.mustCall()))); - req.on('error', (err) => { - if (err.code !== 'ECONNRESET') - throw err; - }); })); diff --git a/test/parallel/test-http2-server-timeout.js b/test/parallel/test-http2-server-timeout.js index 581a409ce9171d..88fc4eab2e08c0 100755 --- a/test/parallel/test-http2-server-timeout.js +++ b/test/parallel/test-http2-server-timeout.js @@ -6,10 +6,10 @@ if (!common.hasCrypto) const http2 = require('http2'); const server = http2.createServer(); -server.setTimeout(common.platformTimeout(1)); +server.setTimeout(common.platformTimeout(50)); const onServerTimeout = common.mustCall((session) => { - session.close(() => session.destroy()); + session.close(); }); server.on('stream', common.mustNotCall()); @@ -18,14 +18,8 @@ server.once('timeout', onServerTimeout); server.listen(0, common.mustCall(() => { const url = `http://localhost:${server.address().port}`; const client = http2.connect(url); - // Because of the timeout, an ECONRESET error may or may not happen here. - // Keep this as a non-op and do not use common.mustCall() - client.on('error', () => {}); client.on('close', common.mustCall(() => { const client2 = http2.connect(url); - // Because of the timeout, an ECONRESET error may or may not happen here. - // Keep this as a non-op and do not use common.mustCall() - client2.on('error', () => {}); client2.on('close', common.mustCall(() => server.close())); })); })); diff --git a/test/parallel/test-http2-too-many-settings.js b/test/parallel/test-http2-too-many-settings.js index 0302fe623da07c..acfd73ada68416 100644 --- a/test/parallel/test-http2-too-many-settings.js +++ b/test/parallel/test-http2-too-many-settings.js @@ -29,9 +29,10 @@ function doTest(session) { server.listen(0, common.mustCall(() => { const client = h2.connect(`http://localhost:${server.address().port}`); - // On some operating systems, an ECONNRESET error may be emitted. - // On others it won't be. Do not make this a mustCall - client.on('error', () => {}); + client.on('error', common.expectsError({ + code: 'ERR_HTTP2_SESSION_ERROR', + message: 'Session closed with error code 2', + })); client.on('close', common.mustCall(() => server.close())); })); } diff --git a/test/parallel/test-http2-trailers.js b/test/parallel/test-http2-trailers.js index 2b7f6158ca3951..bdc0931157ad58 100644 --- a/test/parallel/test-http2-trailers.js +++ b/test/parallel/test-http2-trailers.js @@ -18,6 +18,7 @@ server.on('stream', common.mustCall(onStream)); function onStream(stream, headers, flags) { stream.on('trailers', common.mustCall((headers) => { assert.strictEqual(headers[trailerKey], trailerValue); + stream.end(body); })); stream.respond({ 'content-type': 'text/html', @@ -41,8 +42,6 @@ function onStream(stream, headers, flags) { type: Error } ); - - stream.end(body); } server.listen(0); From 5d29e2c6318c3cd6da4e3a7ac1b9699db1cf529d Mon Sep 17 00:00:00 2001 From: Anatoli Papirovski Date: Mon, 21 May 2018 11:42:32 +0400 Subject: [PATCH 048/129] test: fix flaky http2-session-unref It's possible for the connections to take too long and since the server is already unrefed, the process will just exit. Instead adjust the test so that server unref only happens after all sessions have been successfuly established and unrefed. That still tests the same condition but will not fail under load. Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/20772 Fixes: https://github.com/nodejs/node/issues/20705 Fixes: https://github.com/nodejs/node/issues/20750 Fixes: https://github.com/nodejs/node/issues/20850 Reviewed-By: Matteo Collina Reviewed-By: Rich Trott Reviewed-By: James M Snell --- test/parallel/test-http2-session-unref.js | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/test/parallel/test-http2-session-unref.js b/test/parallel/test-http2-session-unref.js index 465f01d0921f25..0381971c0eace5 100644 --- a/test/parallel/test-http2-session-unref.js +++ b/test/parallel/test-http2-session-unref.js @@ -9,16 +9,20 @@ const common = require('../common'); if (!common.hasCrypto) common.skip('missing crypto'); const http2 = require('http2'); +const Countdown = require('../common/countdown'); const makeDuplexPair = require('../common/duplexpair'); const server = http2.createServer(); const { clientSide, serverSide } = makeDuplexPair(); +const counter = new Countdown(3, () => server.unref()); + // 'session' event should be emitted 3 times: // - the vanilla client // - the destroyed client // - manual 'connection' event emission with generic Duplex stream server.on('session', common.mustCallAtLeast((session) => { + counter.dec(); session.unref(); }, 3)); @@ -54,6 +58,3 @@ server.listen(0, common.mustCall(() => { } })); server.emit('connection', serverSide); -server.unref(); - -setTimeout(common.mustNotCall(() => {}), 1000).unref(); From 91be1dc2a58787cf8885928c6eb0f21a013aface Mon Sep 17 00:00:00 2001 From: Anatoli Papirovski Date: Mon, 28 May 2018 09:20:51 +0200 Subject: [PATCH 049/129] http2: delay closing stream Delay automatically closing the stream with setImmediate in order to allow any pushStreams to be sent first. Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/20997 Fixes: https://github.com/nodejs/node/issues/20992 Reviewed-By: James M Snell --- lib/internal/http2/core.js | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/lib/internal/http2/core.js b/lib/internal/http2/core.js index 5a284a60771f26..4db1b9ea6e0720 100644 --- a/lib/internal/http2/core.js +++ b/lib/internal/http2/core.js @@ -1902,7 +1902,10 @@ class Http2Stream extends Duplex { !(state.flags & STREAM_FLAGS_HAS_TRAILERS) && !state.didRead && this._readableState.flowing === null) { - this.close(); + // By using setImmediate we allow pushStreams to make it through + // before the stream is officially closed. This prevents a bug + // in most browsers where those pushStreams would be rejected. + setImmediate(this.close.bind(this)); } } } @@ -2137,7 +2140,7 @@ class ServerHttp2Stream extends Http2Stream { let headRequest = false; if (headers[HTTP2_HEADER_METHOD] === HTTP2_METHOD_HEAD) headRequest = options.endStream = true; - options.readable = !options.endStream; + options.readable = false; const headersList = mapToHeaders(headers); if (!Array.isArray(headersList)) From b22266cc978b4e774f78266a318bd147e7875e71 Mon Sep 17 00:00:00 2001 From: Anatoli Papirovski Date: Tue, 29 May 2018 19:07:02 +0200 Subject: [PATCH 050/129] http2: force through RST_STREAM in destroy If still needed, force through RST_STREAM in Http2Stream#destroy calls, so that nghttp2 can wrap up properly and doesn't continue trying to read & write data to the stream. Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/21016 Fixes: https://github.com/nodejs/node/issues/21008 Reviewed-By: Benjamin Gruenbaum Reviewed-By: James M Snell Reviewed-By: Anna Henningsen --- lib/internal/http2/core.js | 15 ++++--- src/node_http2.cc | 2 + src/node_http2.h | 7 ++++ .../test-http2-large-write-destroy.js | 40 +++++++++++++++++++ 4 files changed, 59 insertions(+), 5 deletions(-) create mode 100644 test/parallel/test-http2-large-write-destroy.js diff --git a/lib/internal/http2/core.js b/lib/internal/http2/core.js index 4db1b9ea6e0720..39430edee6ff4f 100644 --- a/lib/internal/http2/core.js +++ b/lib/internal/http2/core.js @@ -286,7 +286,7 @@ function onStreamClose(code) { `${sessionName(stream[kSession][kType])}]: closed with code ${code}`); if (!stream.closed) - closeStream(stream, code, false); + closeStream(stream, code, kNoRstStream); if (stream[kState].fd !== undefined) tryClose(stream[kState].fd); @@ -1458,7 +1458,11 @@ function finishSendTrailers(stream, headersList) { stream[kMaybeDestroy](); } -function closeStream(stream, code, shouldSubmitRstStream = true) { +const kNoRstStream = 0; +const kSubmitRstStream = 1; +const kForceRstStream = 2; + +function closeStream(stream, code, rstStreamStatus = kSubmitRstStream) { const state = stream[kState]; state.flags |= STREAM_FLAGS_CLOSED; state.rstCode = code; @@ -1481,9 +1485,10 @@ function closeStream(stream, code, shouldSubmitRstStream = true) { stream.end(); } - if (shouldSubmitRstStream) { + if (rstStreamStatus !== kNoRstStream) { const finishFn = finishCloseStream.bind(stream, code); - if (!ending || finished || code !== NGHTTP2_NO_ERROR) + if (!ending || finished || code !== NGHTTP2_NO_ERROR || + rstStreamStatus === kForceRstStream) finishFn(); else stream.once('finish', finishFn); @@ -1845,7 +1850,7 @@ class Http2Stream extends Duplex { const hasHandle = handle !== undefined; if (!this.closed) - closeStream(this, code, hasHandle); + closeStream(this, code, hasHandle ? kForceRstStream : kNoRstStream); this.push(null); if (hasHandle) { diff --git a/src/node_http2.cc b/src/node_http2.cc index 4630e1228307c6..4eb35330fd42e9 100644 --- a/src/node_http2.cc +++ b/src/node_http2.cc @@ -1839,6 +1839,8 @@ inline void Http2Stream::Destroy() { // Do nothing if this stream instance is already destroyed if (IsDestroyed()) return; + if (session_->HasPendingRstStream(id_)) + FlushRstStream(); flags_ |= NGHTTP2_STREAM_FLAG_DESTROYED; DEBUG_HTTP2STREAM(this, "destroying stream"); diff --git a/src/node_http2.h b/src/node_http2.h index 009c05b731eee8..f054ff9e9d6a26 100644 --- a/src/node_http2.h +++ b/src/node_http2.h @@ -9,6 +9,7 @@ #include "stream_base-inl.h" #include "string_bytes.h" +#include #include namespace node { @@ -880,6 +881,12 @@ class Http2Session : public AsyncWrap { pending_rst_streams_.emplace_back(stream_id); } + inline bool HasPendingRstStream(int32_t stream_id) { + return pending_rst_streams_.end() != std::find(pending_rst_streams_.begin(), + pending_rst_streams_.end(), + stream_id); + } + static void OnStreamAllocImpl(size_t suggested_size, uv_buf_t* buf, void* ctx); diff --git a/test/parallel/test-http2-large-write-destroy.js b/test/parallel/test-http2-large-write-destroy.js new file mode 100644 index 00000000000000..24c0a055cc943f --- /dev/null +++ b/test/parallel/test-http2-large-write-destroy.js @@ -0,0 +1,40 @@ +'use strict'; +const common = require('../common'); +if (!common.hasCrypto) + common.skip('missing crypto'); +const fixtures = require('../common/fixtures'); +const http2 = require('http2'); + +// This test will result in a crash due to a missed CHECK in C++ or +// a straight-up segfault if the C++ doesn't send RST_STREAM through +// properly when calling destroy. + +const content = Buffer.alloc(60000, 0x44); + +const server = http2.createSecureServer({ + key: fixtures.readKey('agent1-key.pem'), + cert: fixtures.readKey('agent1-cert.pem') +}); +server.on('stream', common.mustCall((stream) => { + stream.respond({ + 'Content-Type': 'application/octet-stream', + 'Content-Length': (content.length.toString() * 2), + 'Vary': 'Accept-Encoding' + }, { waitForTrailers: true }); + + stream.write(content); + stream.destroy(); +})); + +server.listen(0, common.mustCall(() => { + const client = http2.connect(`https://localhost:${server.address().port}`, + { rejectUnauthorized: false }); + + const req = client.request({ ':path': '/' }); + req.end(); + + req.on('close', common.mustCall(() => { + client.close(); + server.close(); + })); +})); From 322f39d490f05cd3fc4472b727b02a5c6186e81c Mon Sep 17 00:00:00 2001 From: Anna Henningsen Date: Sat, 2 Jun 2018 16:01:58 +0200 Subject: [PATCH 051/129] test: minor adjustments to test-http2-respond-file Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/21098 Reviewed-By: Anatoli Papirovski Reviewed-By: Rich Trott Reviewed-By: Jon Moss Reviewed-By: Richard Lau --- test/parallel/test-http2-respond-file.js | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/test/parallel/test-http2-respond-file.js b/test/parallel/test-http2-respond-file.js index 9ad8e7a69648dc..1c10ceb4350723 100644 --- a/test/parallel/test-http2-respond-file.js +++ b/test/parallel/test-http2-respond-file.js @@ -19,7 +19,7 @@ const data = fs.readFileSync(fname); const stat = fs.statSync(fname); const server = http2.createServer(); -server.on('stream', (stream) => { +server.on('stream', common.mustCall((stream) => { stream.respondWithFile(fname, { [HTTP2_HEADER_CONTENT_TYPE]: 'text/plain' }, { @@ -28,9 +28,9 @@ server.on('stream', (stream) => { headers[HTTP2_HEADER_CONTENT_LENGTH] = stat.size; } }); -}); -server.listen(0, () => { +})); +server.listen(0, common.mustCall(() => { const client = http2.connect(`http://localhost:${server.address().port}`); const req = client.request(); @@ -49,4 +49,4 @@ server.listen(0, () => { server.close(); })); req.end(); -}); +})); From 3c8c53f4f4c35e87f73e9fdf472f06fd94f77f68 Mon Sep 17 00:00:00 2001 From: Anatoli Papirovski Date: Thu, 31 May 2018 10:39:19 +0200 Subject: [PATCH 052/129] http2: fix premature destroy Check stream._writableState.finished instead of stream.writable as the latter can lead to premature calls to destroy and dropped writes on busy processes. Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/21051 Fixes: https://github.com/nodejs/node/issues/20750 Reviewed-By: Anna Henningsen Reviewed-By: James M Snell --- lib/internal/http2/core.js | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/internal/http2/core.js b/lib/internal/http2/core.js index 39430edee6ff4f..8f6c0c20b5b36c 100644 --- a/lib/internal/http2/core.js +++ b/lib/internal/http2/core.js @@ -1892,7 +1892,7 @@ class Http2Stream extends Duplex { } // TODO(mcollina): remove usage of _*State properties - if (!this.writable) { + if (this._writableState.finished) { if (!this.readable && this.closed) { this.destroy(); return; From 0a8d0861f2165aa1a6139fb3a073a7b41312aa02 Mon Sep 17 00:00:00 2001 From: Anatoli Papirovski Date: Tue, 5 Jun 2018 20:34:47 -0400 Subject: [PATCH 053/129] http2: safer Http2Session destructor It's hypothetically (and with certain V8 flags) possible for the session to be garbage collected before all the streams are. In that case, trying to remove the stream from the session will lead to a segfault due to attempting to access no longer valid memory. Fix this by unsetting the session on any streams still around when destroying. Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/21194 Reviewed-By: Anna Henningsen Reviewed-By: Ujjwal Sharma --- src/node_http2.cc | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/src/node_http2.cc b/src/node_http2.cc index 4eb35330fd42e9..139d622e36781b 100644 --- a/src/node_http2.cc +++ b/src/node_http2.cc @@ -548,8 +548,8 @@ Http2Session::~Http2Session() { ClearWrap(object()); persistent().Reset(); CHECK(persistent().IsEmpty()); - for (const auto& iter : streams_) - iter.second->session_ = nullptr; + for (const auto& stream : streams_) + stream.second->session_ = nullptr; Unconsume(); DEBUG_HTTP2SESSION(this, "freeing nghttp2 session"); nghttp2_session_del(session_); @@ -1782,11 +1782,11 @@ Http2Stream::Http2Stream( Http2Stream::~Http2Stream() { - if (session_ != nullptr) { - DEBUG_HTTP2STREAM(this, "tearing down stream"); - session_->RemoveStream(this); - session_ = nullptr; - } + if (session_ == nullptr) + return; + DEBUG_HTTP2STREAM(this, "tearing down stream"); + session_->RemoveStream(this); + session_ = nullptr; persistent().Reset(); CHECK(persistent().IsEmpty()); @@ -1862,7 +1862,8 @@ inline void Http2Stream::Destroy() { // We can destroy the stream now if there are no writes for it // already on the socket. Otherwise, we'll wait for the garbage collector // to take care of cleaning up. - if (!stream->session()->HasWritesOnSocketForStream(stream)) + if (stream->session() == nullptr || + !stream->session()->HasWritesOnSocketForStream(stream)) delete stream; }, this, this->object()); From 0f3e65099da9b6104321cfd19407d5b13fc4db85 Mon Sep 17 00:00:00 2001 From: Anna Henningsen Date: Fri, 15 Jun 2018 02:15:04 +0200 Subject: [PATCH 054/129] http2: fix memory leak for uncommon headers Fix a memory leak that occurs with header names that are short and not present in the static table of default headers. Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/21336 Reviewed-By: Anatoli Papirovski Reviewed-By: James M Snell Reviewed-By: Minwoo Jung Reviewed-By: Richard Lau Reviewed-By: Tiancheng "Timothy" Gu --- src/node_http2.h | 1 + 1 file changed, 1 insertion(+) diff --git a/src/node_http2.h b/src/node_http2.h index f054ff9e9d6a26..67af832eff78b7 100644 --- a/src/node_http2.h +++ b/src/node_http2.h @@ -1299,6 +1299,7 @@ class ExternalHeader : } if (may_internalize && vec.len < 64) { + nghttp2_rcbuf_decref(buf); // This is a short header name, so there is a good chance V8 already has // it internalized. return GetInternalizedString(env, vec); From e9e4f434b3dcb90c80807e4dc9e72cbce258bf48 Mon Sep 17 00:00:00 2001 From: Anna Henningsen Date: Sat, 16 Jun 2018 23:07:31 +0200 Subject: [PATCH 055/129] http2: fix memory leak when headers are not emitted MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit When headers are not emitted to JS, e.g. because of an error before that could happen, we currently still have the vector of previously received headers lying around, each one holding a reference count of 1. To fix the resulting memory leak, release them in the `Http2Stream` destructor. Also, clear the vector of headers once they have been emitted – there’s not need to keep it around, wasting memory. Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/21373 Reviewed-By: Tiancheng "Timothy" Gu --- src/node_http2.cc | 13 ++++++++----- src/node_http2.h | 8 ++------ 2 files changed, 10 insertions(+), 11 deletions(-) diff --git a/src/node_http2.cc b/src/node_http2.cc index 139d622e36781b..e41cd3cf0315c1 100644 --- a/src/node_http2.cc +++ b/src/node_http2.cc @@ -1137,8 +1137,7 @@ inline void Http2Session::HandleHeadersFrame(const nghttp2_frame* frame) { if (stream->IsDestroyed()) return; - nghttp2_header* headers = stream->headers(); - size_t count = stream->headers_count(); + std::vector headers(stream->move_headers()); Local name_str; Local value_str; @@ -1155,9 +1154,9 @@ inline void Http2Session::HandleHeadersFrame(const nghttp2_frame* frame) { // this way for performance reasons (it's faster to generate and pass an // array than it is to generate and pass the object). size_t n = 0; - while (count > 0) { + while (n < headers.size()) { size_t j = 0; - while (count > 0 && j < arraysize(argv) / 2) { + while (n < headers.size() && j < arraysize(argv) / 2) { nghttp2_header item = headers[n++]; // The header name and value are passed as external one-byte strings name_str = @@ -1166,7 +1165,6 @@ inline void Http2Session::HandleHeadersFrame(const nghttp2_frame* frame) { ExternalHeader::New(env(), item.value).ToLocalChecked(); argv[j * 2] = name_str; argv[j * 2 + 1] = value_str; - count--; j++; } // For performance, we pass name and value pairs to array.protototype.push @@ -1782,6 +1780,11 @@ Http2Stream::Http2Stream( Http2Stream::~Http2Stream() { + for (nghttp2_header& header : current_headers_) { + nghttp2_rcbuf_decref(header.name); + nghttp2_rcbuf_decref(header.value); + } + if (session_ == nullptr) return; DEBUG_HTTP2STREAM(this, "tearing down stream"); diff --git a/src/node_http2.h b/src/node_http2.h index 67af832eff78b7..a35dbf468afa43 100644 --- a/src/node_http2.h +++ b/src/node_http2.h @@ -653,18 +653,14 @@ class Http2Stream : public AsyncWrap, nghttp2_rcbuf* value, uint8_t flags); - inline nghttp2_header* headers() { - return current_headers_.data(); + inline std::vector move_headers() { + return std::move(current_headers_); } inline nghttp2_headers_category headers_category() const { return current_headers_category_; } - inline size_t headers_count() const { - return current_headers_.size(); - } - void StartHeaders(nghttp2_headers_category category); // Required for StreamBase From 20b72fc94d35d2d1166a8cbaf827a0e29dbd9764 Mon Sep 17 00:00:00 2001 From: Anna Henningsen Date: Fri, 15 Jun 2018 00:52:35 +0200 Subject: [PATCH 056/129] http2: track memory allocated by nghttp2 Provide a custom memory allocator for nghttp2, and track memory allocated by the library with it. This makes the used-memory-per-session estimate more accurate, and allows us to track memory leaks either in nghttp2 itself or, more likely, through faulty usage on our end. It also allows us to make the per-session memory limit more accurate in the future; currently, we are not handling this in an ideal way, and instead let nghttp2 allocate what it wants, even if that goes over our limit. Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/21374 Refs: https://github.com/nodejs/node/pull/21373 Refs: https://github.com/nodejs/node/pull/21336 Reviewed-By: James M Snell Reviewed-By: Anatoli Papirovski Reviewed-By: Tiancheng "Timothy" Gu --- src/node_http2.cc | 99 ++++++++++++++++++++++++++++++++++++++++++++--- src/node_http2.h | 19 ++++++--- src/util-inl.h | 5 ++- src/util.h | 3 ++ 4 files changed, 113 insertions(+), 13 deletions(-) diff --git a/src/node_http2.cc b/src/node_http2.cc index e41cd3cf0315c1..b13632aa32d149 100644 --- a/src/node_http2.cc +++ b/src/node_http2.cc @@ -486,6 +486,92 @@ Http2Session::Callbacks::~Callbacks() { nghttp2_session_callbacks_del(callbacks); } +// Track memory allocated by nghttp2 using a custom allocator. +class Http2Session::MemoryAllocatorInfo { + public: + explicit MemoryAllocatorInfo(Http2Session* session) + : info({ session, H2Malloc, H2Free, H2Calloc, H2Realloc }) {} + + static void* H2Malloc(size_t size, void* user_data) { + return H2Realloc(nullptr, size, user_data); + } + + static void* H2Calloc(size_t nmemb, size_t size, void* user_data) { + size_t real_size = MultiplyWithOverflowCheck(nmemb, size); + void* mem = H2Malloc(real_size, user_data); + if (mem != nullptr) + memset(mem, 0, real_size); + return mem; + } + + static void H2Free(void* ptr, void* user_data) { + if (ptr == nullptr) return; // free(null); happens quite often. + void* result = H2Realloc(ptr, 0, user_data); + CHECK_EQ(result, nullptr); + } + + static void* H2Realloc(void* ptr, size_t size, void* user_data) { + Http2Session* session = static_cast(user_data); + size_t previous_size = 0; + char* original_ptr = nullptr; + + // We prepend each allocated buffer with a size_t containing the full + // size of the allocation. + if (size > 0) size += sizeof(size_t); + + if (ptr != nullptr) { + // We are free()ing or re-allocating. + original_ptr = static_cast(ptr) - sizeof(size_t); + previous_size = *reinterpret_cast(original_ptr); + // This means we called StopTracking() on this pointer before. + if (previous_size == 0) { + // Fall back to the standard Realloc() function. + char* ret = UncheckedRealloc(original_ptr, size); + if (ret != nullptr) + ret += sizeof(size_t); + return ret; + } + } + CHECK_GE(session->current_nghttp2_memory_, previous_size); + + // TODO(addaleax): Add the following, and handle NGHTTP2_ERR_NOMEM properly + // everywhere: + // + // if (size > previous_size && + // !session->IsAvailableSessionMemory(size - previous_size)) { + // return nullptr; + //} + + char* mem = UncheckedRealloc(original_ptr, size); + + if (mem != nullptr) { + // Adjust the memory info counter. + session->current_nghttp2_memory_ += size - previous_size; + *reinterpret_cast(mem) = size; + mem += sizeof(size_t); + } else if (size == 0) { + session->current_nghttp2_memory_ -= previous_size; + } + + return mem; + } + + static void StopTracking(Http2Session* session, void* ptr) { + size_t* original_ptr = reinterpret_cast( + static_cast(ptr) - sizeof(size_t)); + session->current_nghttp2_memory_ -= *original_ptr; + *original_ptr = 0; + } + + inline nghttp2_mem* operator*() { return &info; } + + nghttp2_mem info; +}; + +void Http2Session::StopTrackingRcbuf(nghttp2_rcbuf* buf) { + MemoryAllocatorInfo::StopTracking(this, buf); +} + Http2Session::Http2Session(Environment* env, Local wrap, nghttp2_session_type type) @@ -517,15 +603,17 @@ Http2Session::Http2Session(Environment* env, = callback_struct_saved[hasGetPaddingCallback ? 1 : 0].callbacks; auto fn = type == NGHTTP2_SESSION_SERVER ? - nghttp2_session_server_new2 : - nghttp2_session_client_new2; + nghttp2_session_server_new3 : + nghttp2_session_client_new3; + + MemoryAllocatorInfo allocator_info(this); // This should fail only if the system is out of memory, which // is going to cause lots of other problems anyway, or if any // of the options are out of acceptable range, which we should // be catching before it gets this far. Either way, crash if this // fails. - CHECK_EQ(fn(&session_, callbacks, this, *opts), 0); + CHECK_EQ(fn(&session_, callbacks, this, *opts, *allocator_info), 0); outgoing_storage_.reserve(4096); outgoing_buffers_.reserve(32); @@ -553,6 +641,7 @@ Http2Session::~Http2Session() { Unconsume(); DEBUG_HTTP2SESSION(this, "freeing nghttp2 session"); nghttp2_session_del(session_); + CHECK_EQ(current_nghttp2_memory_, 0); } inline bool HasHttp2Observer(Environment* env) { @@ -1160,9 +1249,9 @@ inline void Http2Session::HandleHeadersFrame(const nghttp2_frame* frame) { nghttp2_header item = headers[n++]; // The header name and value are passed as external one-byte strings name_str = - ExternalHeader::New(env(), item.name).ToLocalChecked(); + ExternalHeader::New(this, item.name).ToLocalChecked(); value_str = - ExternalHeader::New(env(), item.value).ToLocalChecked(); + ExternalHeader::New(this, item.value).ToLocalChecked(); argv[j * 2] = name_str; argv[j * 2 + 1] = value_str; j++; diff --git a/src/node_http2.h b/src/node_http2.h index a35dbf468afa43..09771862f53bae 100644 --- a/src/node_http2.h +++ b/src/node_http2.h @@ -797,6 +797,7 @@ class Http2Session : public AsyncWrap { class Http2Ping; class Http2Settings; + class MemoryAllocatorInfo; inline void EmitStatistics(); @@ -934,13 +935,15 @@ class Http2Session : public AsyncWrap { current_session_memory_ -= amount; } - // Returns the current session memory including the current size of both - // the inflate and deflate hpack headers, the current outbound storage - // queue, and pending writes. + // Tell our custom memory allocator that this rcbuf is independent of + // this session now, and may outlive it. + void StopTrackingRcbuf(nghttp2_rcbuf* buf); + + // Returns the current session memory including memory allocated by nghttp2, + // the current outbound storage queue, and pending writes. uint64_t GetCurrentSessionMemory() { uint64_t total = current_session_memory_ + sizeof(Http2Session); - total += nghttp2_session_get_hd_deflate_dynamic_table_size(session_); - total += nghttp2_session_get_hd_inflate_dynamic_table_size(session_); + total += current_nghttp2_memory_; total += outgoing_storage_.size(); return total; } @@ -1072,6 +1075,8 @@ class Http2Session : public AsyncWrap { // The maximum amount of memory allocated for this session uint64_t max_session_memory_ = DEFAULT_MAX_SESSION_MEMORY; uint64_t current_session_memory_ = 0; + // The amount of memory allocated by nghttp2 internals + uint64_t current_nghttp2_memory_ = 0; // The collection of active Http2Streams associated with this session std::unordered_map streams_; @@ -1274,7 +1279,8 @@ class ExternalHeader : } template - static MaybeLocal New(Environment* env, nghttp2_rcbuf* buf) { + static MaybeLocal New(Http2Session* session, nghttp2_rcbuf* buf) { + Environment* env = session->env(); if (nghttp2_rcbuf_is_static(buf)) { auto& static_str_map = env->isolate_data()->http2_static_strs; v8::Eternal& eternal = static_str_map[buf]; @@ -1301,6 +1307,7 @@ class ExternalHeader : return GetInternalizedString(env, vec); } + session->StopTrackingRcbuf(buf); ExternalHeader* h_str = new ExternalHeader(buf); MaybeLocal str = String::NewExternalOneByte(env->isolate(), h_str); if (str.IsEmpty()) diff --git a/src/util-inl.h b/src/util-inl.h index 56c94148d6e820..b24baece7a5561 100644 --- a/src/util-inl.h +++ b/src/util-inl.h @@ -345,8 +345,9 @@ bool StringEqualNoCaseN(const char* a, const char* b, size_t length) { return true; } -inline size_t MultiplyWithOverflowCheck(size_t a, size_t b) { - size_t ret = a * b; +template +inline T MultiplyWithOverflowCheck(T a, T b) { + auto ret = a * b; if (a != 0) CHECK_EQ(b, ret / a); diff --git a/src/util.h b/src/util.h index 1cf515bb40fe3f..bf5e515083d447 100644 --- a/src/util.h +++ b/src/util.h @@ -65,6 +65,9 @@ inline char* Calloc(size_t n); inline char* UncheckedMalloc(size_t n); inline char* UncheckedCalloc(size_t n); +template +inline T MultiplyWithOverflowCheck(T a, T b); + // Used by the allocation functions when allocation fails. // Thin wrapper around v8::Isolate::LowMemoryNotification() that checks // whether V8 is initialized. From 4228141012fff26dbe10d708e7beb5d12cf38c42 Mon Sep 17 00:00:00 2001 From: Gerhard Stoebich Date: Wed, 13 Jun 2018 00:26:42 +0200 Subject: [PATCH 057/129] doc: Improve doc for Http2 headers object Add more details regarding processing and data type of incoming headers in Http2. Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/21296 Reviewed-By: Anna Henningsen Reviewed-By: James M Snell Reviewed-By: Ujjwal Sharma --- doc/api/http2.md | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/doc/api/http2.md b/doc/api/http2.md index 3ffb41e1402ca2..c5c4cb4c95d352 100644 --- a/doc/api/http2.md +++ b/doc/api/http2.md @@ -2120,6 +2120,23 @@ prototype. This means that normal JavaScript object methods such as `Object.prototype.toString()` and `Object.prototype.hasOwnProperty()` will not work. +For incoming headers: +* The `:status` header is converted to `number`. +* Duplicates of `:status`, `:method`, `:authority`, `:scheme`, `:path`, +`age`, `authorization`, `access-control-allow-credentials`, +`access-control-max-age`, `access-control-request-method`, `content-encoding`, +`content-language`, `content-length`, `content-location`, `content-md5`, +`content-range`, `content-type`, `date`, `dnt`, `etag`, `expires`, `from`, +`if-match`, `if-modified-since`, `if-none-match`, `if-range`, +`if-unmodified-since`, `last-modified`, `location`, `max-forwards`, +`proxy-authorization`, `range`, `referer`,`retry-after`, `tk`, +`upgrade-insecure-requests`, `user-agent` or `x-content-type-options` are +discarded. +* `set-cookie` is a string if present once or an array in case duplicates +are present. +* `cookie`: the values are joined together with '; '. +* For all other headers, the values are joined together with ', '. + ```js const http2 = require('http2'); const server = http2.createServer(); From 1cdf93ecdcba25b9a115927b2b0038c73b6f46cd Mon Sep 17 00:00:00 2001 From: Gerhard Stoebich Date: Fri, 15 Jun 2018 23:37:46 +0200 Subject: [PATCH 058/129] http2: pass incoming set-cookie header as array Incoming set-cookie headers should be passed to user as array like in http module. Besides improving compatibility between http and http2 it avoids the need to check if the type is an array or not in user code. Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/21360 Reviewed-By: Tiancheng "Timothy" Gu Reviewed-By: Anatoli Papirovski Reviewed-By: James M Snell Reviewed-By: Trivikram Kamat Reviewed-By: Matteo Collina --- doc/api/http2.md | 3 +- lib/internal/http2/util.js | 7 +-- test/parallel/test-http2-util-headers-list.js | 44 +++++++++++++++++-- 3 files changed, 44 insertions(+), 10 deletions(-) diff --git a/doc/api/http2.md b/doc/api/http2.md index c5c4cb4c95d352..f180a7d297c372 100644 --- a/doc/api/http2.md +++ b/doc/api/http2.md @@ -2132,8 +2132,7 @@ For incoming headers: `proxy-authorization`, `range`, `referer`,`retry-after`, `tk`, `upgrade-insecure-requests`, `user-agent` or `x-content-type-options` are discarded. -* `set-cookie` is a string if present once or an array in case duplicates -are present. +* `set-cookie` is always an array. Duplicates are added to the array. * `cookie`: the values are joined together with '; '. * For all other headers, the values are joined together with ', '. diff --git a/lib/internal/http2/util.js b/lib/internal/http2/util.js index ef48b83d783af0..e7ef7db59077b3 100644 --- a/lib/internal/http2/util.js +++ b/lib/internal/http2/util.js @@ -504,7 +504,7 @@ function toHeaderObject(headers) { value |= 0; var existing = obj[name]; if (existing === undefined) { - obj[name] = value; + obj[name] = name === HTTP2_HEADER_SET_COOKIE ? [value] : value; } else if (!kSingleValueHeaders.has(name)) { switch (name) { case HTTP2_HEADER_COOKIE: @@ -523,10 +523,7 @@ function toHeaderObject(headers) { // fields with the same name. Since it cannot be combined into a // single field-value, recipients ought to handle "Set-Cookie" as a // special case while processing header fields." - if (Array.isArray(existing)) - existing.push(value); - else - obj[name] = [existing, value]; + existing.push(value); break; default: // https://tools.ietf.org/html/rfc7230#section-3.2.2 diff --git a/test/parallel/test-http2-util-headers-list.js b/test/parallel/test-http2-util-headers-list.js index 0ff6b558d9a51b..3b594e727cc5ef 100644 --- a/test/parallel/test-http2-util-headers-list.js +++ b/test/parallel/test-http2-util-headers-list.js @@ -1,14 +1,14 @@ // Flags: --expose-internals 'use strict'; -// Tests the internal utility function that is used to prepare headers -// to pass to the internal binding layer. +// Tests the internal utility functions that are used to prepare headers +// to pass to the internal binding layer and to build a header object. const common = require('../common'); if (!common.hasCrypto) common.skip('missing crypto'); const assert = require('assert'); -const { mapToHeaders } = require('internal/http2/util'); +const { mapToHeaders, toHeaderObject } = require('internal/http2/util'); const { HTTP2_HEADER_STATUS, @@ -302,3 +302,41 @@ common.expectsError({ assert(!(mapToHeaders({ te: 'trailers' }) instanceof Error)); assert(!(mapToHeaders({ te: ['trailers'] }) instanceof Error)); + + +{ + const rawHeaders = [ + ':status', '200', + 'cookie', 'foo', + 'set-cookie', 'sc1', + 'age', '10', + 'x-multi', 'first' + ]; + const headers = toHeaderObject(rawHeaders); + assert.strictEqual(headers[':status'], 200); + assert.strictEqual(headers.cookie, 'foo'); + assert.deepStrictEqual(headers['set-cookie'], ['sc1']); + assert.strictEqual(headers.age, '10'); + assert.strictEqual(headers['x-multi'], 'first'); +} + +{ + const rawHeaders = [ + ':status', '200', + ':status', '400', + 'cookie', 'foo', + 'cookie', 'bar', + 'set-cookie', 'sc1', + 'set-cookie', 'sc2', + 'age', '10', + 'age', '20', + 'x-multi', 'first', + 'x-multi', 'second' + ]; + const headers = toHeaderObject(rawHeaders); + assert.strictEqual(headers[':status'], 200); + assert.strictEqual(headers.cookie, 'foo; bar'); + assert.deepStrictEqual(headers['set-cookie'], ['sc1', 'sc2']); + assert.strictEqual(headers.age, '10'); + assert.strictEqual(headers['x-multi'], 'first, second'); +} From 2116ace0ad97fca3ca55cccb1c0e5133046234c3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=D0=A1=D0=BA=D0=BE=D0=B2=D0=BE=D1=80=D0=BE=D0=B4=D0=B0=20?= =?UTF-8?q?=D0=9D=D0=B8=D0=BA=D0=B8=D1=82=D0=B0=20=D0=90=D0=BD=D0=B4=D1=80?= =?UTF-8?q?=D0=B5=D0=B5=D0=B2=D0=B8=D1=87?= Date: Sat, 23 Jun 2018 14:48:54 +0300 Subject: [PATCH 059/129] doc: fix http2stream.pushStream error doc The old error code `ERR_HTTP2_STREAM_CLOSED` was removed in commit 0babd181a0c5d775e62a12b3b04fe4d7654fe80a (pull request #17406), and the testcase for http2stream.pushStream was changed accordingly, but the documentation change was overlooked. This commit fixes it and aligns the documentation with the testcase. This is a part of the fixes hinted by #21470, which includes some tests for error codes usage and documentation and enforces a stricter format. Refs: https://github.com/nodejs/node/pull/21470 Refs: https://github.com/nodejs/node/pull/17406 Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/21487 Reviewed-By: Colin Ihrig Reviewed-By: Trivikram Kamat Reviewed-By: Vse Mozhet Byt Reviewed-By: Luigi Pinca --- doc/api/http2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/api/http2.md b/doc/api/http2.md index f180a7d297c372..28e915519258f1 100644 --- a/doc/api/http2.md +++ b/doc/api/http2.md @@ -3145,7 +3145,7 @@ added: v8.4.0 Call [`http2stream.pushStream()`][] with the given headers, and wraps the given newly created [`Http2Stream`] on `Http2ServerResponse`. -The callback will be called with an error with code `ERR_HTTP2_STREAM_CLOSED` +The callback will be called with an error with code `ERR_HTTP2_INVALID_STREAM` if the stream is closed. ## Collecting HTTP/2 Performance Metrics From 071a022dbcf91093e7dca328a162d1930bf8d540 Mon Sep 17 00:00:00 2001 From: Rich Trott Date: Fri, 6 Jul 2018 09:48:59 -0700 Subject: [PATCH 060/129] http2: order declarations in core.js Order declarations: * public modules in alphabetical order * internal modules in alphabetical order * process.binding() calls in alphabetical order * exports in alphabetical order Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/21689 Reviewed-By: Gus Caplan Reviewed-By: Weijia Wang Reviewed-By: James M Snell --- lib/internal/http2/core.js | 70 +++++++++++++++++++++----------------- 1 file changed, 38 insertions(+), 32 deletions(-) diff --git a/lib/internal/http2/core.js b/lib/internal/http2/core.js index 8f6c0c20b5b36c..9beb8833a297fb 100644 --- a/lib/internal/http2/core.js +++ b/lib/internal/http2/core.js @@ -2,42 +2,38 @@ /* eslint-disable no-use-before-define */ -require('internal/util').assertCrypto(); +const { + assertCrypto, + customInspectSymbol: kInspect, + promisify +} = require('internal/util'); + +assertCrypto(); -const { async_id_symbol } = process.binding('async_wrap'); -const http = require('http'); -const binding = process.binding('http2'); const assert = require('assert'); const { Buffer } = require('buffer'); const EventEmitter = require('events'); +const fs = require('fs'); +const http = require('http'); const net = require('net'); +const { Duplex } = require('stream'); +const { + _unrefActive, + enroll, + unenroll +} = require('timers'); const tls = require('tls'); +const { URL } = require('url'); const util = require('util'); -const fs = require('fs'); -const errors = require('internal/errors'); + const { StreamWrap } = require('_stream_wrap'); -const { Duplex } = require('stream'); -const { URL } = require('url'); + +const errors = require('internal/errors'); +const { utcDate } = require('internal/http'); const { onServerStream, Http2ServerRequest, Http2ServerResponse, } = require('internal/http2/compat'); -const { utcDate } = require('internal/http'); -const { - promisify, - customInspectSymbol: kInspect -} = require('internal/util'); -const { isArrayBufferView } = require('internal/util/types'); -const { _connectionListener: httpConnectionListener } = http; -const { createPromise, promiseResolve } = process.binding('util'); -const debug = util.debuglog('http2'); - -const kMaxFrameSize = (2 ** 24) - 1; -const kMaxInt = (2 ** 32) - 1; -const kMaxStreams = (2 ** 31) - 1; - -// eslint-disable-next-line no-control-regex -const kQuotedString = /^[\x09\x20-\x5b\x5d-\x7e\x80-\xff]*$/; const { assertIsObject, @@ -58,13 +54,23 @@ const { updateSettingsBuffer } = require('internal/http2/util'); -const { - _unrefActive, - enroll, - unenroll -} = require('timers'); +const { isArrayBufferView } = require('internal/util/types'); +const { async_id_symbol } = process.binding('async_wrap'); +const binding = process.binding('http2'); const { ShutdownWrap, WriteWrap } = process.binding('stream_wrap'); +const { createPromise, promiseResolve } = process.binding('util'); + +const { _connectionListener: httpConnectionListener } = http; +const debug = util.debuglog('http2'); + +const kMaxFrameSize = (2 ** 24) - 1; +const kMaxInt = (2 ** 32) - 1; +const kMaxStreams = (2 ** 31) - 1; + +// eslint-disable-next-line no-control-regex +const kQuotedString = /^[\x09\x20-\x5b\x5d-\x7e\x80-\xff]*$/; + const { constants, nameForErrorCode } = binding; const NETServer = net.Server; @@ -2784,13 +2790,13 @@ function getUnpackedSettings(buf, options = {}) { // Exports module.exports = { + connect, constants, + createServer, + createSecureServer, getDefaultSettings, getPackedSettings, getUnpackedSettings, - createServer, - createSecureServer, - connect, Http2Session, Http2Stream, Http2ServerRequest, From 57618aae0abb1b3515f63c2a4e3c0f3060cd579b Mon Sep 17 00:00:00 2001 From: Shailesh Shekhawat Date: Wed, 27 Jun 2018 22:05:32 +0800 Subject: [PATCH 061/129] errors: fix undefined HTTP2 and tls errors Includes implementation of tls, HTTP2 error with documentation. Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/21564 Refs: https://github.com/nodejs/node/issues/21440 Reviewed-By: Matteo Collina Reviewed-By: Anatoli Papirovski --- doc/api/errors.md | 5 +++++ lib/internal/errors.js | 1 + 2 files changed, 6 insertions(+) diff --git a/doc/api/errors.md b/doc/api/errors.md index 1ab394580e7b76..13193f95ca9af8 100755 --- a/doc/api/errors.md +++ b/doc/api/errors.md @@ -826,6 +826,11 @@ send something other than a regular file. The `Http2Session` closed with a non-zero error code. + +### ERR_HTTP2_SETTINGS_CANCEL + +The `Http2Session` settings canceled. + ### ERR_HTTP2_SOCKET_BOUND diff --git a/lib/internal/errors.js b/lib/internal/errors.js index 7a26d863fe0818..d216e9c37cf0b0 100644 --- a/lib/internal/errors.js +++ b/lib/internal/errors.js @@ -333,6 +333,7 @@ E('ERR_HTTP2_PSEUDOHEADER_NOT_ALLOWED', 'Cannot set HTTP/2 pseudo-headers'); E('ERR_HTTP2_PUSH_DISABLED', 'HTTP/2 client has disabled push streams'); E('ERR_HTTP2_SEND_FILE', 'Only regular files can be sent'); E('ERR_HTTP2_SESSION_ERROR', 'Session closed with error code %s'); +E('ERR_HTTP2_SETTINGS_CANCEL', 'HTTP2 session settings canceled'); E('ERR_HTTP2_SOCKET_BOUND', 'The socket is already bound to an Http2Session'); E('ERR_HTTP2_STATUS_101', From e5175e65962c8dcd383dbbd6b8d4fcb0897dfe32 Mon Sep 17 00:00:00 2001 From: RidgeA Date: Wed, 11 Jul 2018 17:29:42 +0300 Subject: [PATCH 062/129] http2: remove `waitTrailers` listener after closing a stream When `writeHeader` of `Http2ServerResponse` instance are called with 204, 205 and 304 status codes an underlying stream closes. If call `end` method after sending any of these status codes it will cause an error `TypeError: Cannot read property 'Symbol(trailers)' of undefined` because a reference to `Http2ServerResponse` instance associated with Http2Stream already was deleted. The closing of stream causes emitting `waitTrailers` event and, when this event handles inside `onStreamTrailerReady` handler, there is no reference to Http2ServerResponse instance. Fixes: https://github.com/nodejs/node/issues/21740 Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/21764 Reviewed-By: James M Snell Reviewed-By: Matteo Collina Reviewed-By: Ruben Bridgewater Reviewed-By: Anatoli Papirovski --- lib/internal/http2/compat.js | 2 + ...esponse-end-after-statuses-without-body.js | 47 +++++++++++++++++++ 2 files changed, 49 insertions(+) create mode 100644 test/parallel/test-http2-compat-serverresponse-end-after-statuses-without-body.js diff --git a/lib/internal/http2/compat.js b/lib/internal/http2/compat.js index 9b2367c4c97a57..d0cfe94d48a281 100644 --- a/lib/internal/http2/compat.js +++ b/lib/internal/http2/compat.js @@ -377,6 +377,8 @@ function onStreamCloseResponse() { state.closed = true; this[kProxySocket] = null; + + this.removeListener('wantTrailers', onStreamTrailersReady); this[kResponse] = undefined; res.emit('finish'); diff --git a/test/parallel/test-http2-compat-serverresponse-end-after-statuses-without-body.js b/test/parallel/test-http2-compat-serverresponse-end-after-statuses-without-body.js new file mode 100644 index 00000000000000..83d5521bf2473f --- /dev/null +++ b/test/parallel/test-http2-compat-serverresponse-end-after-statuses-without-body.js @@ -0,0 +1,47 @@ +'use strict'; + +const common = require('../common'); +if (!common.hasCrypto) + common.skip('missing crypto'); +const h2 = require('http2'); + +// This test case ensures that calling of res.end after sending +// 204, 205 and 304 HTTP statuses will not cause an error +// See issue: https://github.com/nodejs/node/issues/21740 + +const { + HTTP_STATUS_NO_CONTENT, + HTTP_STATUS_RESET_CONTENT, + HTTP_STATUS_NOT_MODIFIED +} = h2.constants; + +const statusWithouBody = [ + HTTP_STATUS_NO_CONTENT, + HTTP_STATUS_RESET_CONTENT, + HTTP_STATUS_NOT_MODIFIED, +]; +const STATUS_CODES_COUNT = statusWithouBody.length; + +const server = h2.createServer(common.mustCall(function(req, res) { + res.writeHead(statusWithouBody.pop()); + res.end(); +}, STATUS_CODES_COUNT)); + +server.listen(0, common.mustCall(function() { + const url = `http://localhost:${server.address().port}`; + const client = h2.connect(url, common.mustCall(() => { + let responseCount = 0; + const closeAfterResponse = () => { + if (STATUS_CODES_COUNT === ++responseCount) { + client.destroy(); + server.close(); + } + }; + + for (let i = 0; i < STATUS_CODES_COUNT; i++) { + const request = client.request(); + request.on('response', common.mustCall(closeAfterResponse)); + } + + })); +})); From 32bfd7ebfbc705cd143b0a2816fdcb739810c0ad Mon Sep 17 00:00:00 2001 From: Kevin Simper Date: Tue, 17 Jul 2018 15:26:04 -0700 Subject: [PATCH 063/129] doc: add missing `require` to example in http2.md Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/21858 Reviewed-By: Gus Caplan Reviewed-By: Rich Trott Reviewed-By: Daijiro Wachi Reviewed-By: James M Snell Reviewed-By: Vse Mozhet Byt Reviewed-By: Trivikram Kamat --- doc/api/http2.md | 1 + 1 file changed, 1 insertion(+) diff --git a/doc/api/http2.md b/doc/api/http2.md index 28e915519258f1..dd201c9dd7dec2 100644 --- a/doc/api/http2.md +++ b/doc/api/http2.md @@ -1921,6 +1921,7 @@ instances. ```js const http2 = require('http2'); +const fs = require('fs'); const options = { key: fs.readFileSync('server-key.pem'), From 745e1e6192505611b03912e26091cfbd29b39e8d Mon Sep 17 00:00:00 2001 From: Anna Henningsen Date: Sun, 15 Jul 2018 23:58:13 +0200 Subject: [PATCH 064/129] http2: remove unused nghttp2 error list Remove a list of HTTP2 errors as well as `nghttp2_errname()` that converted an integer nghttp2 error code to a string representation. We already use `nghttp2_strerror()` for this, which is provided by nghttp2 returns a better error string anyway. Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/21827 Reviewed-By: Colin Ihrig Reviewed-By: Trivikram Kamat Reviewed-By: Matteo Collina Reviewed-By: James M Snell Reviewed-By: Luigi Pinca --- src/node_http2.h | 52 ------------------------------------------------ 1 file changed, 52 deletions(-) diff --git a/src/node_http2.h b/src/node_http2.h index 09771862f53bae..219d1d02527088 100644 --- a/src/node_http2.h +++ b/src/node_http2.h @@ -377,58 +377,6 @@ enum padding_strategy_type { PADDING_STRATEGY_CALLBACK }; -// These are the error codes provided by the underlying nghttp2 implementation. -#define NGHTTP2_ERROR_CODES(V) \ - V(NGHTTP2_ERR_INVALID_ARGUMENT) \ - V(NGHTTP2_ERR_BUFFER_ERROR) \ - V(NGHTTP2_ERR_UNSUPPORTED_VERSION) \ - V(NGHTTP2_ERR_WOULDBLOCK) \ - V(NGHTTP2_ERR_PROTO) \ - V(NGHTTP2_ERR_INVALID_FRAME) \ - V(NGHTTP2_ERR_EOF) \ - V(NGHTTP2_ERR_DEFERRED) \ - V(NGHTTP2_ERR_STREAM_ID_NOT_AVAILABLE) \ - V(NGHTTP2_ERR_STREAM_CLOSED) \ - V(NGHTTP2_ERR_STREAM_CLOSING) \ - V(NGHTTP2_ERR_STREAM_SHUT_WR) \ - V(NGHTTP2_ERR_INVALID_STREAM_ID) \ - V(NGHTTP2_ERR_INVALID_STREAM_STATE) \ - V(NGHTTP2_ERR_DEFERRED_DATA_EXIST) \ - V(NGHTTP2_ERR_START_STREAM_NOT_ALLOWED) \ - V(NGHTTP2_ERR_GOAWAY_ALREADY_SENT) \ - V(NGHTTP2_ERR_INVALID_HEADER_BLOCK) \ - V(NGHTTP2_ERR_INVALID_STATE) \ - V(NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE) \ - V(NGHTTP2_ERR_FRAME_SIZE_ERROR) \ - V(NGHTTP2_ERR_HEADER_COMP) \ - V(NGHTTP2_ERR_FLOW_CONTROL) \ - V(NGHTTP2_ERR_INSUFF_BUFSIZE) \ - V(NGHTTP2_ERR_PAUSE) \ - V(NGHTTP2_ERR_TOO_MANY_INFLIGHT_SETTINGS) \ - V(NGHTTP2_ERR_PUSH_DISABLED) \ - V(NGHTTP2_ERR_DATA_EXIST) \ - V(NGHTTP2_ERR_SESSION_CLOSING) \ - V(NGHTTP2_ERR_HTTP_HEADER) \ - V(NGHTTP2_ERR_HTTP_MESSAGING) \ - V(NGHTTP2_ERR_REFUSED_STREAM) \ - V(NGHTTP2_ERR_INTERNAL) \ - V(NGHTTP2_ERR_CANCEL) \ - V(NGHTTP2_ERR_FATAL) \ - V(NGHTTP2_ERR_NOMEM) \ - V(NGHTTP2_ERR_CALLBACK_FAILURE) \ - V(NGHTTP2_ERR_BAD_CLIENT_MAGIC) \ - V(NGHTTP2_ERR_FLOODED) - -const char* nghttp2_errname(int rv) { - switch (rv) { -#define V(code) case code: return #code; - NGHTTP2_ERROR_CODES(V) -#undef V - default: - return "NGHTTP2_UNKNOWN_ERROR"; - } -} - enum session_state_flags { SESSION_STATE_NONE = 0x0, SESSION_STATE_HAS_SCOPE = 0x1, From d602c7a2ed80c978f94bd63f76b54d0843bea6cf Mon Sep 17 00:00:00 2001 From: James Ide Date: Fri, 20 Jul 2018 17:25:02 -0700 Subject: [PATCH 065/129] http2: release request()'s "connect" event listener after it runs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The `Http2Session#request()` method internally listens to the "connect" event if the session has not yet established a connection so that the actual request can be sent after the connection has been established. This commit removes the event listener after it runs and carries out the request and is no longer needed. In practice this shouldn't affect the behavior of the session object since the "connect" event fires only once anyway, but removing the listener releases its references. The rest of this class subscribes to the "connect" event with `once` instead of `on` as well. Tested by adding a new test that ensures `Http2Session#request()` is called before the connection is established, indicated by a "connect" listener that is run. The test also ensures all "connect" listeners are removed after the connection is established. Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/21916 Reviewed-By: Michaël Zasso Reviewed-By: Trivikram Kamat Reviewed-By: Anna Henningsen Reviewed-By: Anatoli Papirovski Reviewed-By: Colin Ihrig Reviewed-By: James M Snell Reviewed-By: Matteo Collina --- lib/internal/http2/core.js | 2 +- ...t-http2-request-remove-connect-listener.js | 28 +++++++++++++++++++ 2 files changed, 29 insertions(+), 1 deletion(-) create mode 100644 test/parallel/test-http2-request-remove-connect-listener.js diff --git a/lib/internal/http2/core.js b/lib/internal/http2/core.js index 9beb8833a297fb..e496c3a87fd3ee 100644 --- a/lib/internal/http2/core.js +++ b/lib/internal/http2/core.js @@ -1384,7 +1384,7 @@ class ClientHttp2Session extends Http2Session { const onConnect = requestOnConnect.bind(stream, headersList, options); if (this.connecting) { - this.on('connect', onConnect); + this.once('connect', onConnect); } else { onConnect(); } diff --git a/test/parallel/test-http2-request-remove-connect-listener.js b/test/parallel/test-http2-request-remove-connect-listener.js new file mode 100644 index 00000000000000..61de140c225144 --- /dev/null +++ b/test/parallel/test-http2-request-remove-connect-listener.js @@ -0,0 +1,28 @@ +'use strict'; + +const common = require('../common'); +if (!common.hasCrypto) + common.skip('missing crypto'); +const assert = require('assert'); +const http2 = require('http2'); + + +const server = http2.createServer(); +server.on('stream', common.mustCall((stream) => { + stream.respond(); + stream.end(); +})); + +server.listen(0, common.mustCall(() => { + const client = http2.connect(`http://localhost:${server.address().port}`); + const req = client.request(); + client.once('connect', common.mustCall()); + + req.on('response', common.mustCall(() => { + assert.strictEqual(client.listenerCount('connect'), 0); + })); + req.on('close', common.mustCall(() => { + server.close(); + client.close(); + })); +})); From dcf04dc7df0bf0150c6a92c71bd23961f3f6f666 Mon Sep 17 00:00:00 2001 From: Anto Aravinth Date: Sat, 28 Jul 2018 10:00:32 +0530 Subject: [PATCH 066/129] test: refactor test-http2-compat-serverresponse-finished.js Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/21929 Reviewed-By: Luigi Pinca Reviewed-By: Jon Moss Reviewed-By: Trivikram Kamat Reviewed-By: Benjamin Gruenbaum Reviewed-By: Colin Ihrig Reviewed-By: James M Snell --- .../test-http2-compat-serverresponse-finished.js | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/test/parallel/test-http2-compat-serverresponse-finished.js b/test/parallel/test-http2-compat-serverresponse-finished.js index ceaa6eb5c3cf2c..4da592a5b354b0 100644 --- a/test/parallel/test-http2-compat-serverresponse-finished.js +++ b/test/parallel/test-http2-compat-serverresponse-finished.js @@ -9,14 +9,14 @@ const net = require('net'); // Http2ServerResponse.finished const server = h2.createServer(); -server.listen(0, common.mustCall(function() { +server.listen(0, common.mustCall(() => { const port = server.address().port; - server.once('request', common.mustCall(function(request, response) { + server.once('request', common.mustCall((request, response) => { assert.ok(response.socket instanceof net.Socket); assert.ok(response.connection instanceof net.Socket); assert.strictEqual(response.socket, response.connection); - response.on('finish', common.mustCall(function() { + response.on('finish', common.mustCall(() => { assert.strictEqual(response.socket, undefined); assert.strictEqual(response.connection, undefined); process.nextTick(common.mustCall(() => { @@ -30,7 +30,7 @@ server.listen(0, common.mustCall(function() { })); const url = `http://localhost:${port}`; - const client = h2.connect(url, common.mustCall(function() { + const client = h2.connect(url, common.mustCall(() => { const headers = { ':path': '/', ':method': 'GET', @@ -38,7 +38,7 @@ server.listen(0, common.mustCall(function() { ':authority': `localhost:${port}` }; const request = client.request(headers); - request.on('end', common.mustCall(function() { + request.on('end', common.mustCall(() => { client.close(); })); request.end(); From 7ea08eedaca67f39e9f6eb9ef33ae533400140eb Mon Sep 17 00:00:00 2001 From: Rich Trott Date: Sun, 29 Jul 2018 19:26:27 -0700 Subject: [PATCH 067/129] test: improve reliability in http2-session-timeout Use `setImmediate()` instead of `setTimeout()` to improve robustness of test-http2-session-timeout. Fixes: https://github.com/nodejs/node/issues/20628 Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/22026 Reviewed-By: Anatoli Papirovski Reviewed-By: Jon Moss Reviewed-By: James M Snell Reviewed-By: Trivikram Kamat --- test/sequential/test-http2-session-timeout.js | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/test/sequential/test-http2-session-timeout.js b/test/sequential/test-http2-session-timeout.js index 48e98998c700b6..5c4f047b338e9c 100644 --- a/test/sequential/test-http2-session-timeout.js +++ b/test/sequential/test-http2-session-timeout.js @@ -3,13 +3,12 @@ const common = require('../common'); if (!common.hasCrypto) common.skip('missing crypto'); -const h2 = require('http2'); +const http2 = require('http2'); const serverTimeout = common.platformTimeout(200); -const callTimeout = common.platformTimeout(20); const mustNotCall = common.mustNotCall(); -const server = h2.createServer(); +const server = http2.createServer(); server.timeout = serverTimeout; server.on('request', (req, res) => res.end()); @@ -19,7 +18,7 @@ server.listen(0, common.mustCall(() => { const port = server.address().port; const url = `http://localhost:${port}`; - const client = h2.connect(url); + const client = http2.connect(url); const startTime = process.hrtime(); makeReq(); @@ -37,7 +36,7 @@ server.listen(0, common.mustCall(() => { const diff = process.hrtime(startTime); const milliseconds = (diff[0] * 1e3 + diff[1] / 1e6); if (milliseconds < serverTimeout * 2) { - setTimeout(makeReq, callTimeout); + setImmediate(makeReq); } else { server.removeListener('timeout', mustNotCall); server.close(); From f2f66b4cfb54264480ff5212a46203f992803927 Mon Sep 17 00:00:00 2001 From: James M Snell Date: Fri, 10 Aug 2018 11:16:45 -0700 Subject: [PATCH 068/129] http2: remove `streamError` from docs `streamError` was removed quite some time ago but the docs and code comments weren't updated. Fix that. Fixes: https://github.com/nodejs/node/issues/20211 Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/22246 Reviewed-By: Luigi Pinca Reviewed-By: Jon Moss Reviewed-By: Colin Ihrig Reviewed-By: Matteo Collina Reviewed-By: Vse Mozhet Byt Reviewed-By: Trivikram Kamat --- doc/api/http2.md | 12 ------------ lib/internal/http2/compat.js | 4 +--- .../test-http2-compat-serverresponse-destroy.js | 3 +-- 3 files changed, 2 insertions(+), 17 deletions(-) diff --git a/doc/api/http2.md b/doc/api/http2.md index dd201c9dd7dec2..85b3ba4363ada5 100644 --- a/doc/api/http2.md +++ b/doc/api/http2.md @@ -1505,10 +1505,6 @@ added: v8.4.0 * Extends: {net.Server} -In `Http2Server`, there are no `'clientError'` events as there are in -HTTP1. However, there are `'sessionError'`, and `'streamError'` events for -errors emitted on the socket, or from `Http2Session` or `Http2Stream` instances. - #### Event: 'checkContinue' - -If a `ServerHttp2Stream` emits an `'error'` event, it will be forwarded here. -The stream will already be destroyed when this event is triggered. - #### Event: 'stream' +* `headers` {HTTP/2 Headers Object} An object describing the headers +* `callback` {Function} Called once `http2stream.pushStream()` is finished, + or either when the attempt to create the pushed `Http2Stream` has failed or + has been rejected, or the state of `Http2ServerRequest` is closed prior to + calling the `http2stream.pushStream()` method + * `err` {Error} + * `stream` {ServerHttp2Stream} The newly-created `ServerHttp2Stream` object -Call [`http2stream.pushStream()`][] with the given headers, and wraps the -given newly created [`Http2Stream`] on `Http2ServerResponse`. - -The callback will be called with an error with code `ERR_HTTP2_INVALID_STREAM` -if the stream is closed. +Call [`http2stream.pushStream()`][] with the given headers, and wrap the +given [`Http2Stream`] on a newly created `Http2ServerResponse` as the callback +parameter if successful. When `Http2ServerRequest` is closed, the callback is +called with an error `ERR_HTTP2_INVALID_STREAM`. ## Collecting HTTP/2 Performance Metrics From d0be932375c925df89f2434f1b38d2ad65ae328a Mon Sep 17 00:00:00 2001 From: Rich Trott Date: Sun, 26 Aug 2018 15:45:53 -0700 Subject: [PATCH 075/129] doc: simplify http2 wording and formatting Remove `It is important to note that` and italics from `waitForTrailers` sentences. Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/22541 Reviewed-By: Vse Mozhet Byt Reviewed-By: Luigi Pinca --- doc/api/http2.md | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/doc/api/http2.md b/doc/api/http2.md index ea53e7c48d50b9..7a8abcf4b58498 100644 --- a/doc/api/http2.md +++ b/doc/api/http2.md @@ -744,10 +744,10 @@ is emitted immediately after queuing the last chunk of payload data to be sent. The `http2stream.sendTrailers()` method can then be called to send trailing headers to the peer. -It is important to note that when `options.waitForTrailers` is set, the -`Http2Stream` will *not* automatically close when the final `DATA` frame is -transmitted. User code *must* call either `http2stream.sendTrailers()` or -`http2stream.close()` to close the `Http2Stream`. +When `options.waitForTrailers` is set, the `Http2Stream` will not automatically +close when the final `DATA` frame is transmitted. User code must call either +`http2stream.sendTrailers()` or `http2stream.close()` to close the +`Http2Stream`. The `:method` and `:path` pseudo-headers are not specified within `headers`, they respectively default to: @@ -1290,10 +1290,10 @@ will be emitted immediately after queuing the last chunk of payload data to be sent. The `http2stream.sendTrailers()` method can then be used to sent trailing header fields to the peer. -It is important to note that when `options.waitForTrailers` is set, the -`Http2Stream` will *not* automatically close when the final `DATA` frame is -transmitted. User code *must* call either `http2stream.sendTrailers()` or -`http2stream.close()` to close the `Http2Stream`. +When `options.waitForTrailers` is set, the `Http2Stream` will not automatically +close when the final `DATA` frame is transmitted. User code must call either +`http2stream.sendTrailers()` or `http2stream.close()` to close the +`Http2Stream`. ```js const http2 = require('http2'); @@ -1369,10 +1369,10 @@ will be emitted immediately after queuing the last chunk of payload data to be sent. The `http2stream.sendTrailers()` method can then be used to sent trailing header fields to the peer. -It is important to note that when `options.waitForTrailers` is set, the -`Http2Stream` will *not* automatically close when the final `DATA` frame is -transmitted. User code *must* call either `http2stream.sendTrailers()` or -`http2stream.close()` to close the `Http2Stream`. +When `options.waitForTrailers` is set, the `Http2Stream` will not automatically +close when the final `DATA` frame is transmitted. User code *must* call either +`http2stream.sendTrailers()` or `http2stream.close()` to close the +`Http2Stream`. ```js const http2 = require('http2'); @@ -1488,10 +1488,10 @@ will be emitted immediately after queuing the last chunk of payload data to be sent. The `http2stream.sendTrilers()` method can then be used to sent trailing header fields to the peer. -It is important to note that when `options.waitForTrailers` is set, the -`Http2Stream` will *not* automatically close when the final `DATA` frame is -transmitted. User code *must* call either `http2stream.sendTrailers()` or -`http2stream.close()` to close the `Http2Stream`. +When `options.waitForTrailers` is set, the `Http2Stream` will not automatically +close when the final `DATA` frame is transmitted. User code must call either +`http2stream.sendTrailers()` or `http2stream.close()` to close the +`Http2Stream`. ```js const http2 = require('http2'); From 6a396ff91153c55115e5b1cf39c04dbbbe0412c6 Mon Sep 17 00:00:00 2001 From: James M Snell Date: Thu, 23 Aug 2018 10:38:31 -0700 Subject: [PATCH 076/129] http2: throw better error when accessing unbound socket proxy Fixes: https://github.com/nodejs/node/issues/22268 Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/22486 Reviewed-By: Ruben Bridgewater Reviewed-By: Rich Trott Reviewed-By: Trivikram Kamat Reviewed-By: Matteo Collina --- doc/api/errors.md | 6 ++ lib/internal/errors.js | 2 + lib/internal/http2/core.js | 12 +++- .../test-http2-unbound-socket-proxy.js | 69 +++++++++++++++++++ 4 files changed, 87 insertions(+), 2 deletions(-) create mode 100644 test/parallel/test-http2-unbound-socket-proxy.js diff --git a/doc/api/errors.md b/doc/api/errors.md index 8632bb7be28484..d7c5da0ceb4f58 100755 --- a/doc/api/errors.md +++ b/doc/api/errors.md @@ -843,6 +843,12 @@ The `Http2Session` settings canceled. An attempt was made to connect a `Http2Session` object to a `net.Socket` or `tls.TLSSocket` that had already been bound to another `Http2Session` object. + +### ERR_HTTP2_SOCKET_UNBOUND + +An attempt was made to use the `socket` property of an `Http2Session` that +has already been closed. + ### ERR_HTTP2_STATUS_101 diff --git a/lib/internal/errors.js b/lib/internal/errors.js index 587fce65613374..0863f951830938 100644 --- a/lib/internal/errors.js +++ b/lib/internal/errors.js @@ -338,6 +338,8 @@ E('ERR_HTTP2_SESSION_ERROR', 'Session closed with error code %s'); E('ERR_HTTP2_SETTINGS_CANCEL', 'HTTP2 session settings canceled'); E('ERR_HTTP2_SOCKET_BOUND', 'The socket is already bound to an Http2Session'); +E('ERR_HTTP2_SOCKET_UNBOUND', + 'The socket has been disconnected from the Http2Session'); E('ERR_HTTP2_STATUS_101', 'HTTP status code 101 (Switching Protocols) is forbidden in HTTP/2'); E('ERR_HTTP2_STATUS_INVALID', 'Invalid status code: %s'); diff --git a/lib/internal/http2/core.js b/lib/internal/http2/core.js index 2dea50ea065556..9aa7a0e2a2f79e 100644 --- a/lib/internal/http2/core.js +++ b/lib/internal/http2/core.js @@ -641,12 +641,17 @@ const proxySocketHandler = { throw new errors.Error('ERR_HTTP2_NO_SOCKET_MANIPULATION'); default: const socket = session[kSocket]; + if (socket === undefined) + throw new errors.Error('ERR_HTTP2_SOCKET_UNBOUND'); const value = socket[prop]; return typeof value === 'function' ? value.bind(socket) : value; } }, getPrototypeOf(session) { - return Reflect.getPrototypeOf(session[kSocket]); + const socket = session[kSocket]; + if (socket === undefined) + throw new errors.Error('ERR_HTTP2_SOCKET_UNBOUND'); + return Reflect.getPrototypeOf(socket); }, set(session, prop, value) { switch (prop) { @@ -662,7 +667,10 @@ const proxySocketHandler = { case 'write': throw new errors.Error('ERR_HTTP2_NO_SOCKET_MANIPULATION'); default: - session[kSocket][prop] = value; + const socket = session[kSocket]; + if (socket === undefined) + throw new errors.Error('ERR_HTTP2_SOCKET_UNBOUND'); + socket[prop] = value; return true; } } diff --git a/test/parallel/test-http2-unbound-socket-proxy.js b/test/parallel/test-http2-unbound-socket-proxy.js new file mode 100644 index 00000000000000..44f113bac962f7 --- /dev/null +++ b/test/parallel/test-http2-unbound-socket-proxy.js @@ -0,0 +1,69 @@ +'use strict'; + +const common = require('../common'); +if (!common.hasCrypto) + common.skip('missing crypto'); +const http2 = require('http2'); +const net = require('net'); + +const server = http2.createServer(); +server.on('stream', common.mustCall((stream) => { + stream.respond(); + stream.end('ok'); +})); + +server.listen(0, common.mustCall(() => { + const client = http2.connect(`http://localhost:${server.address().port}`); + const socket = client.socket; + const req = client.request(); + req.resume(); + req.on('close', common.mustCall(() => { + client.close(); + server.close(); + + // Tests to make sure accessing the socket proxy fails with an + // informative error. + setImmediate(common.mustCall(() => { + common.expectsError(() => { + socket.example; + }, { + code: 'ERR_HTTP2_SOCKET_UNBOUND' + }); + common.expectsError(() => { + socket.example = 1; + }, { + code: 'ERR_HTTP2_SOCKET_UNBOUND' + }); + common.expectsError(() => { + socket instanceof net.Socket; + }, { + code: 'ERR_HTTP2_SOCKET_UNBOUND' + }); + common.expectsError(() => { + socket.ref(); + }, { + code: 'ERR_HTTP2_SOCKET_UNBOUND' + }); + common.expectsError(() => { + socket.unref(); + }, { + code: 'ERR_HTTP2_SOCKET_UNBOUND' + }); + common.expectsError(() => { + socket.setEncoding(); + }, { + code: 'ERR_HTTP2_SOCKET_UNBOUND' + }); + common.expectsError(() => { + socket.setKeepAlive(); + }, { + code: 'ERR_HTTP2_SOCKET_UNBOUND' + }); + common.expectsError(() => { + socket.setNoDelay(); + }, { + code: 'ERR_HTTP2_SOCKET_UNBOUND' + }); + })); + })); +})); From 60f7bfa4d7685b16b741df9d8b4bf21927747c7c Mon Sep 17 00:00:00 2001 From: Anna Henningsen Date: Sun, 2 Sep 2018 12:00:41 +0200 Subject: [PATCH 077/129] deps: update to nghttp2 1.33.0 Refs: https://github.com/nghttp2/nghttp2/releases/tag/v1.33.0 Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/22649 Reviewed-By: Colin Ihrig Reviewed-By: Matteo Collina Reviewed-By: Trivikram Kamat --- deps/nghttp2/lib/includes/nghttp2/nghttp2.h | 142 +++++++++++---- .../nghttp2/lib/includes/nghttp2/nghttp2ver.h | 4 +- deps/nghttp2/lib/nghttp2_buf.h | 2 +- deps/nghttp2/lib/nghttp2_callbacks.h | 2 +- deps/nghttp2/lib/nghttp2_debug.h | 10 +- deps/nghttp2/lib/nghttp2_frame.c | 130 ++++++++++++++ deps/nghttp2/lib/nghttp2_frame.h | 51 +++++- deps/nghttp2/lib/nghttp2_hd.h | 2 +- deps/nghttp2/lib/nghttp2_hd_huffman.h | 2 +- deps/nghttp2/lib/nghttp2_helper.h | 2 +- deps/nghttp2/lib/nghttp2_http.h | 2 +- deps/nghttp2/lib/nghttp2_int.h | 2 +- deps/nghttp2/lib/nghttp2_map.h | 2 +- deps/nghttp2/lib/nghttp2_mem.h | 2 +- deps/nghttp2/lib/nghttp2_net.h | 16 +- deps/nghttp2/lib/nghttp2_npn.h | 2 +- deps/nghttp2/lib/nghttp2_option.c | 4 + deps/nghttp2/lib/nghttp2_option.h | 2 +- deps/nghttp2/lib/nghttp2_outbound_item.c | 3 + deps/nghttp2/lib/nghttp2_outbound_item.h | 2 +- deps/nghttp2/lib/nghttp2_pq.h | 2 +- deps/nghttp2/lib/nghttp2_priority_spec.h | 2 +- deps/nghttp2/lib/nghttp2_queue.h | 2 +- deps/nghttp2/lib/nghttp2_rcbuf.h | 2 +- deps/nghttp2/lib/nghttp2_session.c | 161 +++++++++++++++++- deps/nghttp2/lib/nghttp2_session.h | 35 +++- deps/nghttp2/lib/nghttp2_stream.h | 2 +- deps/nghttp2/lib/nghttp2_submit.c | 83 +++++++++ deps/nghttp2/lib/nghttp2_submit.h | 2 +- deps/nghttp2/lib/nghttp2_version.c | 2 +- 30 files changed, 599 insertions(+), 78 deletions(-) diff --git a/deps/nghttp2/lib/includes/nghttp2/nghttp2.h b/deps/nghttp2/lib/includes/nghttp2/nghttp2.h index 14f8950bed8d15..8c54b9c8cc464d 100644 --- a/deps/nghttp2/lib/includes/nghttp2/nghttp2.h +++ b/deps/nghttp2/lib/includes/nghttp2/nghttp2.h @@ -28,7 +28,7 @@ /* Define WIN32 when build target is Win32 API (borrowed from libcurl) */ #if (defined(_WIN32) || defined(__WIN32__)) && !defined(WIN32) -#define WIN32 +# define WIN32 #endif #ifdef __cplusplus @@ -40,9 +40,9 @@ extern "C" { /* MSVC < 2013 does not have inttypes.h because it is not C99 compliant. See compiler macros and version number in https://sourceforge.net/p/predef/wiki/Compilers/ */ -#include +# include #else /* !defined(_MSC_VER) || (_MSC_VER >= 1800) */ -#include +# include #endif /* !defined(_MSC_VER) || (_MSC_VER >= 1800) */ #include #include @@ -50,20 +50,20 @@ extern "C" { #include #ifdef NGHTTP2_STATICLIB -#define NGHTTP2_EXTERN +# define NGHTTP2_EXTERN #elif defined(WIN32) -#ifdef BUILDING_NGHTTP2 -#define NGHTTP2_EXTERN __declspec(dllexport) -#else /* !BUILDING_NGHTTP2 */ -#define NGHTTP2_EXTERN __declspec(dllimport) -#endif /* !BUILDING_NGHTTP2 */ -#else /* !defined(WIN32) */ -#ifdef BUILDING_NGHTTP2 -#define NGHTTP2_EXTERN __attribute__((visibility("default"))) -#else /* !BUILDING_NGHTTP2 */ -#define NGHTTP2_EXTERN -#endif /* !BUILDING_NGHTTP2 */ -#endif /* !defined(WIN32) */ +# ifdef BUILDING_NGHTTP2 +# define NGHTTP2_EXTERN __declspec(dllexport) +# else /* !BUILDING_NGHTTP2 */ +# define NGHTTP2_EXTERN __declspec(dllimport) +# endif /* !BUILDING_NGHTTP2 */ +#else /* !defined(WIN32) */ +# ifdef BUILDING_NGHTTP2 +# define NGHTTP2_EXTERN __attribute__((visibility("default"))) +# else /* !BUILDING_NGHTTP2 */ +# define NGHTTP2_EXTERN +# endif /* !BUILDING_NGHTTP2 */ +#endif /* !defined(WIN32) */ /** * @macro @@ -611,7 +611,12 @@ typedef enum { * The ALTSVC frame, which is defined in `RFC 7383 * `_. */ - NGHTTP2_ALTSVC = 0x0a + NGHTTP2_ALTSVC = 0x0a, + /** + * The ORIGIN frame, which is defined by `RFC 8336 + * `_. + */ + NGHTTP2_ORIGIN = 0x0c } nghttp2_frame_type; /** @@ -2473,15 +2478,15 @@ nghttp2_option_set_no_auto_window_update(nghttp2_option *option, int val); * * This option sets the SETTINGS_MAX_CONCURRENT_STREAMS value of * remote endpoint as if it is received in SETTINGS frame. Without - * specifying this option, before the local endpoint receives - * SETTINGS_MAX_CONCURRENT_STREAMS in SETTINGS frame from remote - * endpoint, SETTINGS_MAX_CONCURRENT_STREAMS is unlimited. This may - * cause problem if local endpoint submits lots of requests initially - * and sending them at once to the remote peer may lead to the - * rejection of some requests. Specifying this option to the sensible - * value, say 100, may avoid this kind of issue. This value will be - * overwritten if the local endpoint receives - * SETTINGS_MAX_CONCURRENT_STREAMS from the remote endpoint. + * specifying this option, the maximum number of outgoing concurrent + * streams is initially limited to 100 to avoid issues when the local + * endpoint submits lots of requests before receiving initial SETTINGS + * frame from the remote endpoint, since sending them at once to the + * remote endpoint could lead to rejection of some of the requests. + * This value will be overwritten when the local endpoint receives + * initial SETTINGS frame from the remote endpoint, either to the + * value advertised in SETTINGS_MAX_CONCURRENT_STREAMS or to the + * default value (unlimited) if none was advertised. */ NGHTTP2_EXTERN void nghttp2_option_set_peer_max_concurrent_streams(nghttp2_option *option, @@ -3797,10 +3802,13 @@ nghttp2_priority_spec_check_default(const nghttp2_priority_spec *pri_spec); * .. warning:: * * This function returns assigned stream ID if it succeeds. But - * that stream is not opened yet. The application must not submit + * that stream is not created yet. The application must not submit * frame to that stream ID before * :type:`nghttp2_before_frame_send_callback` is called for this - * frame. + * frame. This means `nghttp2_session_get_stream_user_data()` does + * not work before the callback. But + * `nghttp2_session_set_stream_user_data()` handles this situation + * specially, and it can set data to a stream during this period. * */ NGHTTP2_EXTERN int32_t nghttp2_submit_request( @@ -4516,8 +4524,7 @@ typedef struct { * Submits ALTSVC frame. * * ALTSVC frame is a non-critical extension to HTTP/2, and defined in - * is defined in `RFC 7383 - * `_. + * `RFC 7383 `_. * * The |flags| is currently ignored and should be * :enum:`NGHTTP2_FLAG_NONE`. @@ -4551,6 +4558,81 @@ NGHTTP2_EXTERN int nghttp2_submit_altsvc(nghttp2_session *session, const uint8_t *field_value, size_t field_value_len); +/** + * @struct + * + * The single entry of an origin. + */ +typedef struct { + /** + * The pointer to origin. No validation is made against this field + * by the library. This is not necessarily NULL-terminated. + */ + uint8_t *origin; + /** + * The length of the |origin|. + */ + size_t origin_len; +} nghttp2_origin_entry; + +/** + * @struct + * + * The payload of ORIGIN frame. ORIGIN frame is a non-critical + * extension to HTTP/2 and defined by `RFC 8336 + * `_. + * + * If this frame is received, and + * `nghttp2_option_set_user_recv_extension_type()` is not set, and + * `nghttp2_option_set_builtin_recv_extension_type()` is set for + * :enum:`NGHTTP2_ORIGIN`, ``nghttp2_extension.payload`` will point to + * this struct. + * + * It has the following members: + */ +typedef struct { + /** + * The number of origins contained in |ov|. + */ + size_t nov; + /** + * The pointer to the array of origins contained in ORIGIN frame. + */ + nghttp2_origin_entry *ov; +} nghttp2_ext_origin; + +/** + * @function + * + * Submits ORIGIN frame. + * + * ORIGIN frame is a non-critical extension to HTTP/2 and defined by + * `RFC 8336 `_. + * + * The |flags| is currently ignored and should be + * :enum:`NGHTTP2_FLAG_NONE`. + * + * The |ov| points to the array of origins. The |nov| specifies the + * number of origins included in |ov|. This function creates copies + * of all elements in |ov|. + * + * The ORIGIN frame is only usable by a server. If this function is + * invoked with client side session, this function returns + * :enum:`NGHTTP2_ERR_INVALID_STATE`. + * + * :enum:`NGHTTP2_ERR_NOMEM` + * Out of memory + * :enum:`NGHTTP2_ERR_INVALID_STATE` + * The function is called from client side session. + * :enum:`NGHTTP2_ERR_INVALID_ARGUMENT` + * There are too many origins, or an origin is too large to fit + * into a default frame payload. + */ +NGHTTP2_EXTERN int nghttp2_submit_origin(nghttp2_session *session, + uint8_t flags, + const nghttp2_origin_entry *ov, + size_t nov); + /** * @function * diff --git a/deps/nghttp2/lib/includes/nghttp2/nghttp2ver.h b/deps/nghttp2/lib/includes/nghttp2/nghttp2ver.h index d32d2754441b25..1f1d4808ca27c0 100644 --- a/deps/nghttp2/lib/includes/nghttp2/nghttp2ver.h +++ b/deps/nghttp2/lib/includes/nghttp2/nghttp2ver.h @@ -29,7 +29,7 @@ * @macro * Version number of the nghttp2 library release */ -#define NGHTTP2_VERSION "1.32.0" +#define NGHTTP2_VERSION "1.33.0" /** * @macro @@ -37,6 +37,6 @@ * release. This is a 24 bit number with 8 bits for major number, 8 bits * for minor and 8 bits for patch. Version 1.2.3 becomes 0x010203. */ -#define NGHTTP2_VERSION_NUM 0x012000 +#define NGHTTP2_VERSION_NUM 0x012100 #endif /* NGHTTP2VER_H */ diff --git a/deps/nghttp2/lib/nghttp2_buf.h b/deps/nghttp2/lib/nghttp2_buf.h index 9f484a221acb5f..06cce67a11bdea 100644 --- a/deps/nghttp2/lib/nghttp2_buf.h +++ b/deps/nghttp2/lib/nghttp2_buf.h @@ -26,7 +26,7 @@ #define NGHTTP2_BUF_H #ifdef HAVE_CONFIG_H -#include +# include #endif /* HAVE_CONFIG_H */ #include diff --git a/deps/nghttp2/lib/nghttp2_callbacks.h b/deps/nghttp2/lib/nghttp2_callbacks.h index b607bbb58b8e3d..61e51fa53638de 100644 --- a/deps/nghttp2/lib/nghttp2_callbacks.h +++ b/deps/nghttp2/lib/nghttp2_callbacks.h @@ -26,7 +26,7 @@ #define NGHTTP2_CALLBACKS_H #ifdef HAVE_CONFIG_H -#include +# include #endif /* HAVE_CONFIG_H */ #include diff --git a/deps/nghttp2/lib/nghttp2_debug.h b/deps/nghttp2/lib/nghttp2_debug.h index 2e2cd0d145e5ba..cbb4dd57547234 100644 --- a/deps/nghttp2/lib/nghttp2_debug.h +++ b/deps/nghttp2/lib/nghttp2_debug.h @@ -26,18 +26,18 @@ #define NGHTTP2_DEBUG_H #ifdef HAVE_CONFIG_H -#include +# include #endif /* HAVE_CONFIG_H */ #include #ifdef DEBUGBUILD -#define DEBUGF(...) nghttp2_debug_vprintf(__VA_ARGS__) +# define DEBUGF(...) nghttp2_debug_vprintf(__VA_ARGS__) void nghttp2_debug_vprintf(const char *format, ...); #else -#define DEBUGF(...) \ - do { \ - } while (0) +# define DEBUGF(...) \ + do { \ + } while (0) #endif #endif /* NGHTTP2_DEBUG_H */ diff --git a/deps/nghttp2/lib/nghttp2_frame.c b/deps/nghttp2/lib/nghttp2_frame.c index fa7cb6953bc539..6e33f3c247f5cb 100644 --- a/deps/nghttp2/lib/nghttp2_frame.c +++ b/deps/nghttp2/lib/nghttp2_frame.c @@ -223,6 +223,36 @@ void nghttp2_frame_altsvc_free(nghttp2_extension *frame, nghttp2_mem *mem) { nghttp2_mem_free(mem, altsvc->origin); } +void nghttp2_frame_origin_init(nghttp2_extension *frame, + nghttp2_origin_entry *ov, size_t nov) { + nghttp2_ext_origin *origin; + size_t payloadlen = 0; + size_t i; + + for (i = 0; i < nov; ++i) { + payloadlen += 2 + ov[i].origin_len; + } + + nghttp2_frame_hd_init(&frame->hd, payloadlen, NGHTTP2_ORIGIN, + NGHTTP2_FLAG_NONE, 0); + + origin = frame->payload; + origin->ov = ov; + origin->nov = nov; +} + +void nghttp2_frame_origin_free(nghttp2_extension *frame, nghttp2_mem *mem) { + nghttp2_ext_origin *origin; + + origin = frame->payload; + if (origin == NULL) { + return; + } + /* We use the same buffer for all resources pointed by the field of + origin directly or indirectly. */ + nghttp2_mem_free(mem, origin->ov); +} + size_t nghttp2_frame_priority_len(uint8_t flags) { if (flags & NGHTTP2_FLAG_PRIORITY) { return NGHTTP2_PRIORITY_SPECLEN; @@ -746,6 +776,106 @@ int nghttp2_frame_unpack_altsvc_payload2(nghttp2_extension *frame, return 0; } +int nghttp2_frame_pack_origin(nghttp2_bufs *bufs, nghttp2_extension *frame) { + nghttp2_buf *buf; + nghttp2_ext_origin *origin; + nghttp2_origin_entry *orig; + size_t i; + + origin = frame->payload; + + buf = &bufs->head->buf; + + if (nghttp2_buf_avail(buf) < frame->hd.length) { + return NGHTTP2_ERR_FRAME_SIZE_ERROR; + } + + buf->pos -= NGHTTP2_FRAME_HDLEN; + + nghttp2_frame_pack_frame_hd(buf->pos, &frame->hd); + + for (i = 0; i < origin->nov; ++i) { + orig = &origin->ov[i]; + nghttp2_put_uint16be(buf->last, (uint16_t)orig->origin_len); + buf->last += 2; + buf->last = nghttp2_cpymem(buf->last, orig->origin, orig->origin_len); + } + + assert(nghttp2_buf_len(buf) == NGHTTP2_FRAME_HDLEN + frame->hd.length); + + return 0; +} + +int nghttp2_frame_unpack_origin_payload(nghttp2_extension *frame, + const uint8_t *payload, + size_t payloadlen, nghttp2_mem *mem) { + nghttp2_ext_origin *origin; + const uint8_t *p, *end; + uint8_t *dst; + size_t originlen; + nghttp2_origin_entry *ov; + size_t nov = 0; + size_t len = 0; + + origin = frame->payload; + p = payload; + end = p + payloadlen; + + for (; p != end;) { + if (end - p < 2) { + return NGHTTP2_ERR_FRAME_SIZE_ERROR; + } + originlen = nghttp2_get_uint16(p); + p += 2; + if (originlen == 0) { + continue; + } + if (originlen > (size_t)(end - p)) { + return NGHTTP2_ERR_FRAME_SIZE_ERROR; + } + p += originlen; + /* 1 for terminal NULL */ + len += originlen + 1; + ++nov; + } + + if (nov == 0) { + origin->ov = NULL; + origin->nov = 0; + + return 0; + } + + len += nov * sizeof(nghttp2_origin_entry); + + ov = nghttp2_mem_malloc(mem, len); + if (ov == NULL) { + return NGHTTP2_ERR_NOMEM; + } + + origin->ov = ov; + origin->nov = nov; + + dst = (uint8_t *)ov + nov * sizeof(nghttp2_origin_entry); + p = payload; + + for (; p != end;) { + originlen = nghttp2_get_uint16(p); + p += 2; + if (originlen == 0) { + continue; + } + ov->origin = dst; + ov->origin_len = originlen; + dst = nghttp2_cpymem(dst, p, originlen); + *dst++ = '\0'; + p += originlen; + ++ov; + } + + return 0; +} + nghttp2_settings_entry *nghttp2_frame_iv_copy(const nghttp2_settings_entry *iv, size_t niv, nghttp2_mem *mem) { nghttp2_settings_entry *iv_copy; diff --git a/deps/nghttp2/lib/nghttp2_frame.h b/deps/nghttp2/lib/nghttp2_frame.h index 35ca214a4a7a59..615bbf31f5d60d 100644 --- a/deps/nghttp2/lib/nghttp2_frame.h +++ b/deps/nghttp2/lib/nghttp2_frame.h @@ -26,7 +26,7 @@ #define NGHTTP2_FRAME_H #ifdef HAVE_CONFIG_H -#include +# include #endif /* HAVE_CONFIG_H */ #include @@ -72,6 +72,7 @@ /* Union of extension frame payload */ typedef union { nghttp2_ext_altsvc altsvc; + nghttp2_ext_origin origin; } nghttp2_ext_frame_payload; void nghttp2_frame_pack_frame_hd(uint8_t *buf, const nghttp2_frame_hd *hd); @@ -392,6 +393,36 @@ int nghttp2_frame_unpack_altsvc_payload2(nghttp2_extension *frame, const uint8_t *payload, size_t payloadlen, nghttp2_mem *mem); +/* + * Packs ORIGIN frame |frame| in wire frame format and store it in + * |bufs|. + * + * The caller must make sure that nghttp2_bufs_reset(bufs) is called + * before calling this function. + * + * This function returns 0 if it succeeds, or one of the following + * negative error codes: + * + * NGHTTP2_ERR_FRAME_SIZE_ERROR + * The length of the frame is too large. + */ +int nghttp2_frame_pack_origin(nghttp2_bufs *bufs, nghttp2_extension *ext); + +/* + * Unpacks ORIGIN wire format into |frame|. The |payload| of length + * |payloadlen| contains the frame payload. + * + * This function returns 0 if it succeeds, or one of the following + * negative error codes: + * + * NGHTTP2_ERR_NOMEM + * Out of memory. + * NGHTTP2_ERR_FRAME_SIZE_ERROR + * The payload is too small. + */ +int nghttp2_frame_unpack_origin_payload(nghttp2_extension *frame, + const uint8_t *payload, + size_t payloadlen, nghttp2_mem *mem); /* * Initializes HEADERS frame |frame| with given values. |frame| takes * ownership of |nva|, so caller must not free it. If |stream_id| is @@ -489,6 +520,24 @@ void nghttp2_frame_altsvc_init(nghttp2_extension *frame, int32_t stream_id, */ void nghttp2_frame_altsvc_free(nghttp2_extension *frame, nghttp2_mem *mem); +/* + * Initializes ORIGIN frame |frame| with given values. This function + * assumes that frame->payload points to nghttp2_ext_origin object. + * Also |ov| and the memory pointed by the field of its elements are + * allocated in single buffer, starting with |ov|. On success, this + * function takes ownership of |ov|, so caller must not free it. + */ +void nghttp2_frame_origin_init(nghttp2_extension *frame, + nghttp2_origin_entry *ov, size_t nov); + +/* + * Frees up resources under |frame|. This function does not free + * nghttp2_ext_origin object pointed by frame->payload. This function + * only frees nghttp2_ext_origin.ov. Therefore, other fields must be + * allocated in the same buffer with ov. + */ +void nghttp2_frame_origin_free(nghttp2_extension *frame, nghttp2_mem *mem); + /* * Returns the number of padding bytes after payload. The total * padding length is given in the |padlen|. The returned value does diff --git a/deps/nghttp2/lib/nghttp2_hd.h b/deps/nghttp2/lib/nghttp2_hd.h index 760bfbc357efdc..c64a1f2b9b406c 100644 --- a/deps/nghttp2/lib/nghttp2_hd.h +++ b/deps/nghttp2/lib/nghttp2_hd.h @@ -26,7 +26,7 @@ #define NGHTTP2_HD_H #ifdef HAVE_CONFIG_H -#include +# include #endif /* HAVE_CONFIG_H */ #include diff --git a/deps/nghttp2/lib/nghttp2_hd_huffman.h b/deps/nghttp2/lib/nghttp2_hd_huffman.h index 83323400313509..c6e3942e95f4fc 100644 --- a/deps/nghttp2/lib/nghttp2_hd_huffman.h +++ b/deps/nghttp2/lib/nghttp2_hd_huffman.h @@ -26,7 +26,7 @@ #define NGHTTP2_HD_HUFFMAN_H #ifdef HAVE_CONFIG_H -#include +# include #endif /* HAVE_CONFIG_H */ #include diff --git a/deps/nghttp2/lib/nghttp2_helper.h b/deps/nghttp2/lib/nghttp2_helper.h index 4a32564f39dfa3..b1f18ce541ac36 100644 --- a/deps/nghttp2/lib/nghttp2_helper.h +++ b/deps/nghttp2/lib/nghttp2_helper.h @@ -26,7 +26,7 @@ #define NGHTTP2_HELPER_H #ifdef HAVE_CONFIG_H -#include +# include #endif /* HAVE_CONFIG_H */ #include diff --git a/deps/nghttp2/lib/nghttp2_http.h b/deps/nghttp2/lib/nghttp2_http.h index ac684c4d9ecbb1..dd057cdb60757f 100644 --- a/deps/nghttp2/lib/nghttp2_http.h +++ b/deps/nghttp2/lib/nghttp2_http.h @@ -26,7 +26,7 @@ #define NGHTTP2_HTTP_H #ifdef HAVE_CONFIG_H -#include +# include #endif /* HAVE_CONFIG_H */ #include diff --git a/deps/nghttp2/lib/nghttp2_int.h b/deps/nghttp2/lib/nghttp2_int.h index 30cf7274bc4675..b23585ccb27da2 100644 --- a/deps/nghttp2/lib/nghttp2_int.h +++ b/deps/nghttp2/lib/nghttp2_int.h @@ -26,7 +26,7 @@ #define NGHTTP2_INT_H #ifdef HAVE_CONFIG_H -#include +# include #endif /* HAVE_CONFIG_H */ #include diff --git a/deps/nghttp2/lib/nghttp2_map.h b/deps/nghttp2/lib/nghttp2_map.h index 21262488d6d2d4..f6e29e35f2de3f 100644 --- a/deps/nghttp2/lib/nghttp2_map.h +++ b/deps/nghttp2/lib/nghttp2_map.h @@ -26,7 +26,7 @@ #define NGHTTP2_MAP_H #ifdef HAVE_CONFIG_H -#include +# include #endif /* HAVE_CONFIG_H */ #include diff --git a/deps/nghttp2/lib/nghttp2_mem.h b/deps/nghttp2/lib/nghttp2_mem.h index 2d1bd6a0f42396..f83dbcb8f9a588 100644 --- a/deps/nghttp2/lib/nghttp2_mem.h +++ b/deps/nghttp2/lib/nghttp2_mem.h @@ -26,7 +26,7 @@ #define NGHTTP2_MEM_H #ifdef HAVE_CONFIG_H -#include +# include #endif /* HAVE_CONFIG_H */ #include diff --git a/deps/nghttp2/lib/nghttp2_net.h b/deps/nghttp2/lib/nghttp2_net.h index 587f4189fdba4a..95ffee74a14fc9 100644 --- a/deps/nghttp2/lib/nghttp2_net.h +++ b/deps/nghttp2/lib/nghttp2_net.h @@ -26,15 +26,15 @@ #define NGHTTP2_NET_H #ifdef HAVE_CONFIG_H -#include +# include #endif /* HAVE_CONFIG_H */ #ifdef HAVE_ARPA_INET_H -#include +# include #endif /* HAVE_ARPA_INET_H */ #ifdef HAVE_NETINET_IN_H -#include +# include #endif /* HAVE_NETINET_IN_H */ #include @@ -44,11 +44,11 @@ define inline functions for those function so that we don't have dependeny on that lib. */ -#ifdef _MSC_VER -#define STIN static __inline -#else -#define STIN static inline -#endif +# ifdef _MSC_VER +# define STIN static __inline +# else +# define STIN static inline +# endif STIN uint32_t htonl(uint32_t hostlong) { uint32_t res; diff --git a/deps/nghttp2/lib/nghttp2_npn.h b/deps/nghttp2/lib/nghttp2_npn.h index a481bde3507ce5..c6f1c04b683594 100644 --- a/deps/nghttp2/lib/nghttp2_npn.h +++ b/deps/nghttp2/lib/nghttp2_npn.h @@ -26,7 +26,7 @@ #define NGHTTP2_NPN_H #ifdef HAVE_CONFIG_H -#include +# include #endif /* HAVE_CONFIG_H */ #include diff --git a/deps/nghttp2/lib/nghttp2_option.c b/deps/nghttp2/lib/nghttp2_option.c index aec5dcfa8ab542..8946d7dd38cfb8 100644 --- a/deps/nghttp2/lib/nghttp2_option.c +++ b/deps/nghttp2/lib/nghttp2_option.c @@ -86,6 +86,10 @@ void nghttp2_option_set_builtin_recv_extension_type(nghttp2_option *option, option->opt_set_mask |= NGHTTP2_OPT_BUILTIN_RECV_EXT_TYPES; option->builtin_recv_ext_types |= NGHTTP2_TYPEMASK_ALTSVC; return; + case NGHTTP2_ORIGIN: + option->opt_set_mask |= NGHTTP2_OPT_BUILTIN_RECV_EXT_TYPES; + option->builtin_recv_ext_types |= NGHTTP2_TYPEMASK_ORIGIN; + return; default: return; } diff --git a/deps/nghttp2/lib/nghttp2_option.h b/deps/nghttp2/lib/nghttp2_option.h index c743e33b8ed551..29e72aa321007a 100644 --- a/deps/nghttp2/lib/nghttp2_option.h +++ b/deps/nghttp2/lib/nghttp2_option.h @@ -26,7 +26,7 @@ #define NGHTTP2_OPTION_H #ifdef HAVE_CONFIG_H -#include +# include #endif /* HAVE_CONFIG_H */ #include diff --git a/deps/nghttp2/lib/nghttp2_outbound_item.c b/deps/nghttp2/lib/nghttp2_outbound_item.c index 1633cc36859da1..f651c8029ac024 100644 --- a/deps/nghttp2/lib/nghttp2_outbound_item.c +++ b/deps/nghttp2/lib/nghttp2_outbound_item.c @@ -86,6 +86,9 @@ void nghttp2_outbound_item_free(nghttp2_outbound_item *item, nghttp2_mem *mem) { case NGHTTP2_ALTSVC: nghttp2_frame_altsvc_free(&frame->ext, mem); break; + case NGHTTP2_ORIGIN: + nghttp2_frame_origin_free(&frame->ext, mem); + break; default: assert(0); break; diff --git a/deps/nghttp2/lib/nghttp2_outbound_item.h b/deps/nghttp2/lib/nghttp2_outbound_item.h index 89a8a92668dd5c..b5f503a312dd8c 100644 --- a/deps/nghttp2/lib/nghttp2_outbound_item.h +++ b/deps/nghttp2/lib/nghttp2_outbound_item.h @@ -26,7 +26,7 @@ #define NGHTTP2_OUTBOUND_ITEM_H #ifdef HAVE_CONFIG_H -#include +# include #endif /* HAVE_CONFIG_H */ #include diff --git a/deps/nghttp2/lib/nghttp2_pq.h b/deps/nghttp2/lib/nghttp2_pq.h index 71cf96a14e0c77..2d7b702ac18ad0 100644 --- a/deps/nghttp2/lib/nghttp2_pq.h +++ b/deps/nghttp2/lib/nghttp2_pq.h @@ -26,7 +26,7 @@ #define NGHTTP2_PQ_H #ifdef HAVE_CONFIG_H -#include +# include #endif /* HAVE_CONFIG_H */ #include diff --git a/deps/nghttp2/lib/nghttp2_priority_spec.h b/deps/nghttp2/lib/nghttp2_priority_spec.h index 98fac21060091e..92ece822a8f257 100644 --- a/deps/nghttp2/lib/nghttp2_priority_spec.h +++ b/deps/nghttp2/lib/nghttp2_priority_spec.h @@ -26,7 +26,7 @@ #define NGHTTP2_PRIORITY_SPEC_H #ifdef HAVE_CONFIG_H -#include +# include #endif /* HAVE_CONFIG_H */ #include diff --git a/deps/nghttp2/lib/nghttp2_queue.h b/deps/nghttp2/lib/nghttp2_queue.h index c7eb753ca92182..a06fa6c7a46fc7 100644 --- a/deps/nghttp2/lib/nghttp2_queue.h +++ b/deps/nghttp2/lib/nghttp2_queue.h @@ -26,7 +26,7 @@ #define NGHTTP2_QUEUE_H #ifdef HAVE_CONFIG_H -#include "config.h" +# include "config.h" #endif /* HAVE_CONFIG_H */ #include diff --git a/deps/nghttp2/lib/nghttp2_rcbuf.h b/deps/nghttp2/lib/nghttp2_rcbuf.h index 29d1543e2c5965..6814e709fb4148 100644 --- a/deps/nghttp2/lib/nghttp2_rcbuf.h +++ b/deps/nghttp2/lib/nghttp2_rcbuf.h @@ -26,7 +26,7 @@ #define NGHTTP2_RCBUF_H #ifdef HAVE_CONFIG_H -#include +# include #endif /* HAVE_CONFIG_H */ #include diff --git a/deps/nghttp2/lib/nghttp2_session.c b/deps/nghttp2/lib/nghttp2_session.c index a9e7a62390e56a..418ad6663585f5 100644 --- a/deps/nghttp2/lib/nghttp2_session.c +++ b/deps/nghttp2/lib/nghttp2_session.c @@ -348,6 +348,12 @@ static void session_inbound_frame_reset(nghttp2_session *session) { } nghttp2_frame_altsvc_free(&iframe->frame.ext, mem); break; + case NGHTTP2_ORIGIN: + if ((session->builtin_recv_ext_types & NGHTTP2_TYPEMASK_ORIGIN) == 0) { + break; + } + nghttp2_frame_origin_free(&iframe->frame.ext, mem); + break; } } @@ -1749,6 +1755,13 @@ static int session_predicate_altsvc_send(nghttp2_session *session, return 0; } +static int session_predicate_origin_send(nghttp2_session *session) { + if (session_is_closing(session)) { + return NGHTTP2_ERR_SESSION_CLOSING; + } + return 0; +} + /* Take into account settings max frame size and both connection-level flow control here */ static ssize_t @@ -2280,6 +2293,18 @@ static int session_prep_frame(nghttp2_session *session, nghttp2_frame_pack_altsvc(&session->aob.framebufs, &frame->ext); + return 0; + case NGHTTP2_ORIGIN: + rv = session_predicate_origin_send(session); + if (rv != 0) { + return rv; + } + + rv = nghttp2_frame_pack_origin(&session->aob.framebufs, &frame->ext); + if (rv != 0) { + return rv; + } + return 0; default: /* Unreachable here */ @@ -4385,6 +4410,12 @@ int nghttp2_session_on_settings_received(nghttp2_session *session, return session_call_on_frame_received(session, frame); } + if (!session->remote_settings_received) { + session->remote_settings.max_concurrent_streams = + NGHTTP2_DEFAULT_MAX_CONCURRENT_STREAMS; + session->remote_settings_received = 1; + } + for (i = 0; i < frame->settings.niv; ++i) { nghttp2_settings_entry *entry = &frame->settings.iv[i]; @@ -4821,6 +4852,11 @@ int nghttp2_session_on_altsvc_received(nghttp2_session *session, return session_call_on_frame_received(session, frame); } +int nghttp2_session_on_origin_received(nghttp2_session *session, + nghttp2_frame *frame) { + return session_call_on_frame_received(session, frame); +} + static int session_process_altsvc_frame(nghttp2_session *session) { nghttp2_inbound_frame *iframe = &session->iframe; nghttp2_frame *frame = &iframe->frame; @@ -4836,6 +4872,25 @@ static int session_process_altsvc_frame(nghttp2_session *session) { return nghttp2_session_on_altsvc_received(session, frame); } +static int session_process_origin_frame(nghttp2_session *session) { + nghttp2_inbound_frame *iframe = &session->iframe; + nghttp2_frame *frame = &iframe->frame; + nghttp2_mem *mem = &session->mem; + int rv; + + rv = nghttp2_frame_unpack_origin_payload(&frame->ext, iframe->lbuf.pos, + nghttp2_buf_len(&iframe->lbuf), mem); + if (rv != 0) { + if (nghttp2_is_fatal(rv)) { + return rv; + } + /* Ignore ORIGIN frame which cannot be parsed. */ + return 0; + } + + return nghttp2_session_on_origin_received(session, frame); +} + static int session_process_extension_frame(nghttp2_session *session) { int rv; nghttp2_inbound_frame *iframe = &session->iframe; @@ -5746,6 +5801,42 @@ ssize_t nghttp2_session_mem_recv(nghttp2_session *session, const uint8_t *in, iframe->state = NGHTTP2_IB_READ_NBYTE; inbound_frame_set_mark(iframe, 2); + break; + case NGHTTP2_ORIGIN: + if (!(session->builtin_recv_ext_types & NGHTTP2_TYPEMASK_ORIGIN)) { + busy = 1; + iframe->state = NGHTTP2_IB_IGN_PAYLOAD; + break; + } + + DEBUGF("recv: ORIGIN\n"); + + iframe->frame.ext.payload = &iframe->ext_frame_payload.origin; + + if (session->server || iframe->frame.hd.stream_id || + (iframe->frame.hd.flags & 0xf0)) { + busy = 1; + iframe->state = NGHTTP2_IB_IGN_PAYLOAD; + break; + } + + iframe->frame.hd.flags = NGHTTP2_FLAG_NONE; + + if (iframe->payloadleft) { + iframe->raw_lbuf = nghttp2_mem_malloc(mem, iframe->payloadleft); + + if (iframe->raw_lbuf == NULL) { + return NGHTTP2_ERR_NOMEM; + } + + nghttp2_buf_wrap_init(&iframe->lbuf, iframe->raw_lbuf, + iframe->payloadleft); + } else { + busy = 1; + } + + iframe->state = NGHTTP2_IB_READ_ORIGIN_PAYLOAD; + break; default: busy = 1; @@ -6583,7 +6674,6 @@ ssize_t nghttp2_session_mem_recv(nghttp2_session *session, const uint8_t *in, DEBUGF("recv: [IB_READ_ALTSVC_PAYLOAD]\n"); readlen = inbound_frame_payload_readlen(iframe, in, last); - if (readlen > 0) { iframe->lbuf.last = nghttp2_cpymem(iframe->lbuf.last, in, readlen); @@ -6601,11 +6691,44 @@ ssize_t nghttp2_session_mem_recv(nghttp2_session *session, const uint8_t *in, } rv = session_process_altsvc_frame(session); + if (nghttp2_is_fatal(rv)) { + return rv; + } + + session_inbound_frame_reset(session); + + break; + case NGHTTP2_IB_READ_ORIGIN_PAYLOAD: + DEBUGF("recv: [IB_READ_ORIGIN_PAYLOAD]\n"); + + readlen = inbound_frame_payload_readlen(iframe, in, last); + + if (readlen > 0) { + iframe->lbuf.last = nghttp2_cpymem(iframe->lbuf.last, in, readlen); + + iframe->payloadleft -= readlen; + in += readlen; + } + + DEBUGF("recv: readlen=%zu, payloadleft=%zu\n", readlen, + iframe->payloadleft); + + if (iframe->payloadleft) { + assert(nghttp2_buf_avail(&iframe->lbuf) > 0); + + break; + } + + rv = session_process_origin_frame(session); if (nghttp2_is_fatal(rv)) { return rv; } + if (iframe->state == NGHTTP2_IB_IGN_ALL) { + return (ssize_t)inlen; + } + session_inbound_frame_reset(session); break; @@ -7085,12 +7208,42 @@ int nghttp2_session_set_stream_user_data(nghttp2_session *session, int32_t stream_id, void *stream_user_data) { nghttp2_stream *stream; + nghttp2_frame *frame; + nghttp2_outbound_item *item; + stream = nghttp2_session_get_stream(session, stream_id); - if (!stream) { + if (stream) { + stream->stream_user_data = stream_user_data; + return 0; + } + + if (session->server || !nghttp2_session_is_my_stream_id(session, stream_id) || + !nghttp2_outbound_queue_top(&session->ob_syn)) { return NGHTTP2_ERR_INVALID_ARGUMENT; } - stream->stream_user_data = stream_user_data; - return 0; + + frame = &nghttp2_outbound_queue_top(&session->ob_syn)->frame; + assert(frame->hd.type == NGHTTP2_HEADERS); + + if (frame->hd.stream_id > stream_id || + (uint32_t)stream_id >= session->next_stream_id) { + return NGHTTP2_ERR_INVALID_ARGUMENT; + } + + for (item = session->ob_syn.head; item; item = item->qnext) { + if (item->frame.hd.stream_id < stream_id) { + continue; + } + + if (item->frame.hd.stream_id > stream_id) { + break; + } + + item->aux_data.headers.stream_user_data = stream_user_data; + return 0; + } + + return NGHTTP2_ERR_INVALID_ARGUMENT; } int nghttp2_session_resume_data(nghttp2_session *session, int32_t stream_id) { diff --git a/deps/nghttp2/lib/nghttp2_session.h b/deps/nghttp2/lib/nghttp2_session.h index c7cb27d77c1e25..5add50bc8bce16 100644 --- a/deps/nghttp2/lib/nghttp2_session.h +++ b/deps/nghttp2/lib/nghttp2_session.h @@ -26,7 +26,7 @@ #define NGHTTP2_SESSION_H #ifdef HAVE_CONFIG_H -#include +# include #endif /* HAVE_CONFIG_H */ #include @@ -61,7 +61,8 @@ typedef enum { */ typedef enum { NGHTTP2_TYPEMASK_NONE = 0, - NGHTTP2_TYPEMASK_ALTSVC = 1 << 0 + NGHTTP2_TYPEMASK_ALTSVC = 1 << 0, + NGHTTP2_TYPEMASK_ORIGIN = 1 << 1 } nghttp2_typemask; typedef enum { @@ -121,6 +122,7 @@ typedef enum { NGHTTP2_IB_IGN_DATA, NGHTTP2_IB_IGN_ALL, NGHTTP2_IB_READ_ALTSVC_PAYLOAD, + NGHTTP2_IB_READ_ORIGIN_PAYLOAD, NGHTTP2_IB_READ_EXTENSION_PAYLOAD } nghttp2_inbound_state; @@ -301,8 +303,10 @@ struct nghttp2_session { increased/decreased by submitting WINDOW_UPDATE. See nghttp2_submit_window_update(). */ int32_t local_window_size; - /* Settings value received from the remote endpoint. We just use ID - as index. The index = 0 is unused. */ + /* This flag is used to indicate that the local endpoint received initial + SETTINGS frame from the remote endpoint. */ + uint8_t remote_settings_received; + /* Settings value received from the remote endpoint. */ nghttp2_settings_storage remote_settings; /* Settings value of the local endpoint. */ nghttp2_settings_storage local_settings; @@ -698,7 +702,7 @@ int nghttp2_session_on_push_promise_received(nghttp2_session *session, * NGHTTP2_ERR_NOMEM * Out of memory. * NGHTTP2_ERR_CALLBACK_FAILURE - * The callback function failed. + * The callback function failed. * NGHTTP2_ERR_FLOODED * There are too many items in outbound queue, and this is most * likely caused by misbehaviour of peer. @@ -716,7 +720,7 @@ int nghttp2_session_on_ping_received(nghttp2_session *session, * NGHTTP2_ERR_NOMEM * Out of memory. * NGHTTP2_ERR_CALLBACK_FAILURE - * The callback function failed. + * The callback function failed. */ int nghttp2_session_on_goaway_received(nghttp2_session *session, nghttp2_frame *frame); @@ -731,7 +735,7 @@ int nghttp2_session_on_goaway_received(nghttp2_session *session, * NGHTTP2_ERR_NOMEM * Out of memory. * NGHTTP2_ERR_CALLBACK_FAILURE - * The callback function failed. + * The callback function failed. */ int nghttp2_session_on_window_update_received(nghttp2_session *session, nghttp2_frame *frame); @@ -744,11 +748,24 @@ int nghttp2_session_on_window_update_received(nghttp2_session *session, * negative error codes: * * NGHTTP2_ERR_CALLBACK_FAILURE - * The callback function failed. + * The callback function failed. */ int nghttp2_session_on_altsvc_received(nghttp2_session *session, nghttp2_frame *frame); +/* + * Called when ORIGIN is received, assuming |frame| is properly + * initialized. + * + * This function returns 0 if it succeeds, or one of the following + * negative error codes: + * + * NGHTTP2_ERR_CALLBACK_FAILURE + * The callback function failed. + */ +int nghttp2_session_on_origin_received(nghttp2_session *session, + nghttp2_frame *frame); + /* * Called when DATA is received, assuming |frame| is properly * initialized. @@ -759,7 +776,7 @@ int nghttp2_session_on_altsvc_received(nghttp2_session *session, * NGHTTP2_ERR_NOMEM * Out of memory. * NGHTTP2_ERR_CALLBACK_FAILURE - * The callback function failed. + * The callback function failed. */ int nghttp2_session_on_data_received(nghttp2_session *session, nghttp2_frame *frame); diff --git a/deps/nghttp2/lib/nghttp2_stream.h b/deps/nghttp2/lib/nghttp2_stream.h index da0e5d532c2f0b..d1d5856d800e76 100644 --- a/deps/nghttp2/lib/nghttp2_stream.h +++ b/deps/nghttp2/lib/nghttp2_stream.h @@ -26,7 +26,7 @@ #define NGHTTP2_STREAM_H #ifdef HAVE_CONFIG_H -#include +# include #endif /* HAVE_CONFIG_H */ #include diff --git a/deps/nghttp2/lib/nghttp2_submit.c b/deps/nghttp2/lib/nghttp2_submit.c index 6c15c82488e724..f604eff5c9017f 100644 --- a/deps/nghttp2/lib/nghttp2_submit.c +++ b/deps/nghttp2/lib/nghttp2_submit.c @@ -571,6 +571,89 @@ int nghttp2_submit_altsvc(nghttp2_session *session, uint8_t flags, return rv; } +int nghttp2_submit_origin(nghttp2_session *session, uint8_t flags, + const nghttp2_origin_entry *ov, size_t nov) { + nghttp2_mem *mem; + uint8_t *p; + nghttp2_outbound_item *item; + nghttp2_frame *frame; + nghttp2_ext_origin *origin; + nghttp2_origin_entry *ov_copy; + size_t len = 0; + size_t i; + int rv; + (void)flags; + + mem = &session->mem; + + if (!session->server) { + return NGHTTP2_ERR_INVALID_STATE; + } + + if (nov) { + for (i = 0; i < nov; ++i) { + len += ov[i].origin_len; + } + + if (2 * nov + len > NGHTTP2_MAX_PAYLOADLEN) { + return NGHTTP2_ERR_INVALID_ARGUMENT; + } + + /* The last nov is added for terminal NULL character. */ + ov_copy = + nghttp2_mem_malloc(mem, nov * sizeof(nghttp2_origin_entry) + len + nov); + if (ov_copy == NULL) { + return NGHTTP2_ERR_NOMEM; + } + + p = (uint8_t *)ov_copy + nov * sizeof(nghttp2_origin_entry); + + for (i = 0; i < nov; ++i) { + ov_copy[i].origin = p; + ov_copy[i].origin_len = ov[i].origin_len; + p = nghttp2_cpymem(p, ov[i].origin, ov[i].origin_len); + *p++ = '\0'; + } + + assert((size_t)(p - (uint8_t *)ov_copy) == + nov * sizeof(nghttp2_origin_entry) + len + nov); + } else { + ov_copy = NULL; + } + + item = nghttp2_mem_malloc(mem, sizeof(nghttp2_outbound_item)); + if (item == NULL) { + rv = NGHTTP2_ERR_NOMEM; + goto fail_item_malloc; + } + + nghttp2_outbound_item_init(item); + + item->aux_data.ext.builtin = 1; + + origin = &item->ext_frame_payload.origin; + + frame = &item->frame; + frame->ext.payload = origin; + + nghttp2_frame_origin_init(&frame->ext, ov_copy, nov); + + rv = nghttp2_session_add_item(session, item); + if (rv != 0) { + nghttp2_frame_origin_free(&frame->ext, mem); + nghttp2_mem_free(mem, item); + + return rv; + } + + return 0; + +fail_item_malloc: + free(ov_copy); + + return rv; +} + static uint8_t set_request_flags(const nghttp2_priority_spec *pri_spec, const nghttp2_data_provider *data_prd) { uint8_t flags = NGHTTP2_FLAG_NONE; diff --git a/deps/nghttp2/lib/nghttp2_submit.h b/deps/nghttp2/lib/nghttp2_submit.h index 545388cfa3bef4..74d702fbcf077e 100644 --- a/deps/nghttp2/lib/nghttp2_submit.h +++ b/deps/nghttp2/lib/nghttp2_submit.h @@ -26,7 +26,7 @@ #define NGHTTP2_SUBMIT_H #ifdef HAVE_CONFIG_H -#include +# include #endif /* HAVE_CONFIG_H */ #include diff --git a/deps/nghttp2/lib/nghttp2_version.c b/deps/nghttp2/lib/nghttp2_version.c index 8c5710d419c331..4211f2cf8f624d 100644 --- a/deps/nghttp2/lib/nghttp2_version.c +++ b/deps/nghttp2/lib/nghttp2_version.c @@ -23,7 +23,7 @@ * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. */ #ifdef HAVE_CONFIG_H -#include +# include #endif /* HAVE_CONFIG_H */ #include From 805bf40bfdc4747acd69b16112d693508685136c Mon Sep 17 00:00:00 2001 From: Szymon Marczak Date: Sun, 2 Sep 2018 12:07:20 +0200 Subject: [PATCH 078/129] http2: don't expose the original socket through the socket proxy Refs: https://github.com/nodejs/node/pull/22486 Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/22650 Reviewed-By: Matteo Collina Reviewed-By: Luigi Pinca Reviewed-By: Anatoli Papirovski Reviewed-By: James M Snell --- lib/internal/http2/core.js | 14 +++++++++-- test/parallel/test-http2-socket-proxy.js | 11 ++++++++ .../test-http2-unbound-socket-proxy.js | 25 ------------------- 3 files changed, 23 insertions(+), 27 deletions(-) diff --git a/lib/internal/http2/core.js b/lib/internal/http2/core.js index 9aa7a0e2a2f79e..575e4152c94228 100644 --- a/lib/internal/http2/core.js +++ b/lib/internal/http2/core.js @@ -630,7 +630,9 @@ const proxySocketHandler = { get(session, prop) { switch (prop) { case 'setTimeout': - return session.setTimeout.bind(session); + case 'ref': + case 'unref': + return session[prop].bind(session); case 'destroy': case 'emit': case 'end': @@ -638,6 +640,9 @@ const proxySocketHandler = { case 'read': case 'resume': case 'write': + case 'setEncoding': + case 'setKeepAlive': + case 'setNoDelay': throw new errors.Error('ERR_HTTP2_NO_SOCKET_MANIPULATION'); default: const socket = session[kSocket]; @@ -656,7 +661,9 @@ const proxySocketHandler = { set(session, prop, value) { switch (prop) { case 'setTimeout': - session.setTimeout = value; + case 'ref': + case 'unref': + session[prop] = value; return true; case 'destroy': case 'emit': @@ -665,6 +672,9 @@ const proxySocketHandler = { case 'read': case 'resume': case 'write': + case 'setEncoding': + case 'setKeepAlive': + case 'setNoDelay': throw new errors.Error('ERR_HTTP2_NO_SOCKET_MANIPULATION'); default: const socket = session[kSocket]; diff --git a/test/parallel/test-http2-socket-proxy.js b/test/parallel/test-http2-socket-proxy.js index 17830495addc63..209c49e719c676 100644 --- a/test/parallel/test-http2-socket-proxy.js +++ b/test/parallel/test-http2-socket-proxy.js @@ -38,6 +38,9 @@ server.on('stream', common.mustCall(function(stream, headers) { common.expectsError(() => socket.read, errMsg); common.expectsError(() => socket.resume, errMsg); common.expectsError(() => socket.write, errMsg); + common.expectsError(() => socket.setEncoding, errMsg); + common.expectsError(() => socket.setKeepAlive, errMsg); + common.expectsError(() => socket.setNoDelay, errMsg); common.expectsError(() => (socket.destroy = undefined), errMsg); common.expectsError(() => (socket.emit = undefined), errMsg); @@ -46,10 +49,18 @@ server.on('stream', common.mustCall(function(stream, headers) { common.expectsError(() => (socket.read = undefined), errMsg); common.expectsError(() => (socket.resume = undefined), errMsg); common.expectsError(() => (socket.write = undefined), errMsg); + common.expectsError(() => (socket.setEncoding = undefined), errMsg); + common.expectsError(() => (socket.setKeepAlive = undefined), errMsg); + common.expectsError(() => (socket.setNoDelay = undefined), errMsg); assert.doesNotThrow(() => (socket.on = socket.on)); assert.doesNotThrow(() => (socket.once = socket.once)); + socket.unref(); + assert.strictEqual(socket._handle.hasRef(), false); + socket.ref(); + assert.strictEqual(socket._handle.hasRef(), true); + stream.respond(); socket.writable = 0; diff --git a/test/parallel/test-http2-unbound-socket-proxy.js b/test/parallel/test-http2-unbound-socket-proxy.js index 44f113bac962f7..18881574f2c09b 100644 --- a/test/parallel/test-http2-unbound-socket-proxy.js +++ b/test/parallel/test-http2-unbound-socket-proxy.js @@ -39,31 +39,6 @@ server.listen(0, common.mustCall(() => { }, { code: 'ERR_HTTP2_SOCKET_UNBOUND' }); - common.expectsError(() => { - socket.ref(); - }, { - code: 'ERR_HTTP2_SOCKET_UNBOUND' - }); - common.expectsError(() => { - socket.unref(); - }, { - code: 'ERR_HTTP2_SOCKET_UNBOUND' - }); - common.expectsError(() => { - socket.setEncoding(); - }, { - code: 'ERR_HTTP2_SOCKET_UNBOUND' - }); - common.expectsError(() => { - socket.setKeepAlive(); - }, { - code: 'ERR_HTTP2_SOCKET_UNBOUND' - }); - common.expectsError(() => { - socket.setNoDelay(); - }, { - code: 'ERR_HTTP2_SOCKET_UNBOUND' - }); })); })); })); From bae7c608e253d880c6241d787e45cd58b555d3b4 Mon Sep 17 00:00:00 2001 From: Sagi Tsofan Date: Tue, 11 Sep 2018 01:14:10 +0300 Subject: [PATCH 079/129] doc: document http2 timeouts New default timeout values of "2 minutes" were added into documentation inside 2 classes under "Event: 'timeout'": 1) Class: Http2SecureServer 2) Class: Http2Server New sections for `.setTimeout()` method were added inside `Http2SecureServer` & `Http2Server` docs. Backport-PR-URL: https://github.com/nodejs/node/pull/22850 PR-URL: https://github.com/nodejs/node/pull/22798 Reviewed-By: Matteo Collina Reviewed-By: Colin Ihrig Reviewed-By: James M Snell --- doc/api/http2.md | 38 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/doc/api/http2.md b/doc/api/http2.md index 7a8abcf4b58498..81d1c8fa8e08e7 100644 --- a/doc/api/http2.md +++ b/doc/api/http2.md @@ -1604,6 +1604,7 @@ added: v8.4.0 The `'timeout'` event is emitted when there is no activity on the Server for a given number of milliseconds set using `http2server.setTimeout()`. +**Default:** 2 minutes. #### server.close([callback]) + +* `msecs` {number} **Default:** `120000` (2 minutes) +* `callback` {Function} +* Returns: {Http2Server} + +Used to set the timeout value for http2 server requests, +and sets a callback function that is called when there is no activity +on the `Http2Server` after `msecs` milliseconds. + +The given callback is registered as a listener on the `'timeout'` event. + +In case of no callback function were assigned, a new `ERR_INVALID_CALLBACK` +error will be thrown. + ### Class: Http2SecureServer + +* `msecs` {number} **Default:** `120000` (2 minutes) +* `callback` {Function} +* Returns: {Http2SecureServer} + +Used to set the timeout value for http2 secure server requests, +and sets a callback function that is called when there is no activity +on the `Http2SecureServer` after `msecs` milliseconds. + +The given callback is registered as a listener on the `'timeout'` event. + +In case of no callback function were assigned, a new `ERR_INVALID_CALLBACK` +error will be thrown. + ### http2.createServer(options[, onRequestHandler]) + +* {boolean} + +Set the `true` if the `END_STREAM` flag was set in the request or response +HEADERS frame received, indicating that no additional data should be received +and the readable side of the `Http2Stream` will be closed. + #### http2stream.pending + +* `origins` { string | URL | Object } One or more URL Strings passed as + separate arguments. + +Submits an `ORIGIN` frame (as defined by [RFC 8336][]) to the connected client +to advertise the set of origins for which the server is capable of providing +authoritative responses. + +```js +const http2 = require('http2'); +const options = getSecureOptionsSomehow(); +const server = http2.createSecureServer(options); +server.on('stream', (stream) => { + stream.respond(); + stream.end('ok'); +}); +server.on('session', (session) => { + session.origin('https://example.com', 'https://example.org'); +}); +``` + +When a string is passed as an `origin`, it will be parsed as a URL and the +origin will be derived. For instance, the origin for the HTTP URL +`'https://example.org/foo/bar'` is the ASCII string +`'https://example.org'`. An error will be thrown if either the given string +cannot be parsed as a URL or if a valid origin cannot be derived. + +A `URL` object, or any object with an `origin` property, may be passed as +an `origin`, in which case the value of the `origin` property will be +used. The value of the `origin` property *must* be a properly serialized +ASCII origin. + +Alternatively, the `origins` option may be used when creating a new HTTP/2 +server using the `http2.createSecureServer()` method: + +```js +const http2 = require('http2'); +const options = getSecureOptionsSomehow(); +options.origins = ['https://example.com', 'https://example.org']; +const server = http2.createSecureServer(options); +server.on('stream', (stream) => { + stream.respond(); + stream.end('ok'); +}); +``` + ### Class: ClientHttp2Session + +* `origins` {string[]} + +The `'origin'` event is emitted whenever an `ORIGIN` frame is received by +the client. The event is emitted with an array of `origin` strings. The +`http2session.originSet` will be updated to include the received +origins. + +```js +const http2 = require('http2'); +const client = http2.connect('https://example.org'); + +client.on('origin', (origins) => { + for (let n = 0; n < origins.length; n++) + console.log(origins[n]); +}); +``` + +The `'origin'` event is only emitted when using a secure TLS connection. + #### clienthttp2session.request(headers[, options]) + +* `payload` {Buffer} The `PING` frame 8-byte payload + +The `'ping'` event is emitted whenever a `PING` frame is received from the +connected peer. + #### Event: 'remoteSettings' -> Stability: 1 - Experimental +> Stability: 2 - Stable The `http2` module provides an implementation of the [HTTP/2][] protocol. It can be accessed using: diff --git a/lib/http2.js b/lib/http2.js index de06de1cc414cb..1f770ff4c734cd 100644 --- a/lib/http2.js +++ b/lib/http2.js @@ -1,11 +1,5 @@ 'use strict'; -process.emitWarning( - 'The http2 module is an experimental API.', - 'ExperimentalWarning', undefined, - 'See https://github.com/nodejs/http2' -); - const { constants, getDefaultSettings, From 5b9af6ea7379c61098a8b39b28f1b97f30e7ffaa Mon Sep 17 00:00:00 2001 From: Yuta Hiroto Date: Wed, 14 Mar 2018 14:40:31 +0900 Subject: [PATCH 088/129] doc: update username and email PR-URL: https://github.com/nodejs/node/pull/19338 Reviewed-By: Gireesh Punathil Reviewed-By: Yosuke Furukawa Reviewed-By: Rich Trott Reviewed-By: Myles Borins Reviewed-By: Daniel Bevenius Reviewed-By: Ruben Bridgewater Reviewed-By: James M Snell --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index b05014109e08db..7f0c69c4a3cbe6 100644 --- a/README.md +++ b/README.md @@ -291,8 +291,6 @@ For more information about the governance of the Node.js project, see ### Collaborators -* [abouthiroppy](https://github.com/abouthiroppy) - -**Yuta Hiroto** <hello@about-hiroppy.com> (he/him) * [addaleax](https://github.com/addaleax) - **Anna Henningsen** <anna@addaleax.net> (she/her) * [ak239](https://github.com/ak239) - @@ -367,6 +365,8 @@ For more information about the governance of the Node.js project, see **Guy Bedford** <guybedford@gmail.com> (he/him) * [hashseed](https://github.com/hashseed) - **Yang Guo** <yangguo@chromium.org> (he/him) +* [hiroppy](https://github.com/hiroppy) - +**Yuta Hiroto** <hello@hiroppy.me> (he/him) * [iarna](https://github.com/iarna) - **Rebecca Turner** <me@re-becca.org> * [imran-iq](https://github.com/imran-iq) - From 9446bb68eabb5323daa9c27e1e2442344fccc392 Mon Sep 17 00:00:00 2001 From: Rich Trott Date: Mon, 12 Mar 2018 21:04:55 -0700 Subject: [PATCH 089/129] doc: fix minor issues in async_hooks.md * easily -> easy * was -> is * add a missing comma PR-URL: https://github.com/nodejs/node/pull/19313 Reviewed-By: Vse Mozhet Byt Reviewed-By: Colin Ihrig Reviewed-By: Luigi Pinca --- doc/api/async_hooks.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/doc/api/async_hooks.md b/doc/api/async_hooks.md index cf9dc46a95d60d..2b92e4e6b5adbc 100644 --- a/doc/api/async_hooks.md +++ b/doc/api/async_hooks.md @@ -147,10 +147,10 @@ unintentional side effects. Because printing to the console is an asynchronous operation, `console.log()` will cause the AsyncHooks callbacks to be called. Using `console.log()` or similar asynchronous operations inside an AsyncHooks callback function will thus -cause an infinite recursion. An easily solution to this when debugging is -to use a synchronous logging operation such as `fs.writeSync(1, msg)`. This -will print to stdout because `1` is the file descriptor for stdout and will -not invoke AsyncHooks recursively because it is synchronous. +cause an infinite recursion. An easy solution to this when debugging is to use a +synchronous logging operation such as `fs.writeSync(1, msg)`. This will print to +stdout because `1` is the file descriptor for stdout and will not invoke +AsyncHooks recursively because it is synchronous. ```js const fs = require('fs'); @@ -593,8 +593,8 @@ JavaScript API so that all the appropriate callbacks are called. ### `class AsyncResource()` -The class `AsyncResource` was designed to be extended by the embedder's async -resources. Using this users can easily trigger the lifetime events of their +The class `AsyncResource` is designed to be extended by the embedder's async +resources. Using this, users can easily trigger the lifetime events of their own resources. The `init` hook will trigger when an `AsyncResource` is instantiated. From afcf059898cdd7b44c60c94d4165cebc3e72fbd2 Mon Sep 17 00:00:00 2001 From: Vse Mozhet Byt Date: Mon, 12 Mar 2018 05:09:14 +0200 Subject: [PATCH 090/129] build: do not cd on vcbuild help `vcbuild help` just outputs help info and exits. If a user calls this command not from a project root, the directory change can be unexpected and unwanted. PR-URL: https://github.com/nodejs/node/pull/19291 Reviewed-By: Richard Lau Reviewed-By: Daniel Bevenius Reviewed-By: James M Snell --- vcbuild.bat | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/vcbuild.bat b/vcbuild.bat index 51509b9cf4bfb5..43962324857bed 100644 --- a/vcbuild.bat +++ b/vcbuild.bat @@ -1,7 +1,5 @@ @echo off -cd %~dp0 - if /i "%1"=="help" goto help if /i "%1"=="--help" goto help if /i "%1"=="-help" goto help @@ -11,6 +9,8 @@ if /i "%1"=="-?" goto help if /i "%1"=="--?" goto help if /i "%1"=="/?" goto help +cd %~dp0 + @rem Process arguments. set config=Release set target=Build From 610297e2aba3e9d2118e5fcdc1cc0f1bd40fea77 Mon Sep 17 00:00:00 2001 From: Rich Trott Date: Mon, 12 Mar 2018 22:03:19 -0700 Subject: [PATCH 091/129] doc: improve best practices in onboarding-extras MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit PR-URL: https://github.com/nodejs/node/pull/19315 Reviewed-By: Daniel Bevenius Reviewed-By: Michaël Zasso Reviewed-By: Colin Ihrig Reviewed-By: Luigi Pinca Reviewed-By: Richard Lau --- doc/onboarding-extras.md | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/doc/onboarding-extras.md b/doc/onboarding-extras.md index 0da3b1b5259f3a..75f3010be48b2e 100644 --- a/doc/onboarding-extras.md +++ b/doc/onboarding-extras.md @@ -130,10 +130,7 @@ to update from nodejs/node: * `git remote update -p` OR `git fetch --all` (I prefer the former) * `git merge --ff-only upstream/master` (or `REMOTENAME/BRANCH`) +## Best practices -## best practices - -* commit often, out to your github fork (origin), open a PR -* when making PRs make sure to spend time on the description: - * every moment you spend writing a good description quarters the amount of time it takes to understand your code. -* usually prefer to only squash at the *end* of your work, depends on the change +* When making PRs, spend time writing a thorough description. +* Usually only squash at the end of your work. From 3ad10e5789acb755bb0c0d8ee0c2f5e87fa9c8b7 Mon Sep 17 00:00:00 2001 From: Daniel Bevenius Date: Tue, 13 Mar 2018 12:03:27 +0100 Subject: [PATCH 092/129] src: add extractPromiseWrap function Currently PromiseHook extracts the PromiseWrap from a Local in two places. This commit extracts that code into a function instead. PR-URL: https://github.com/nodejs/node/pull/19340 Reviewed-By: James M Snell --- src/async_wrap.cc | 21 +++++++++------------ 1 file changed, 9 insertions(+), 12 deletions(-) diff --git a/src/async_wrap.cc b/src/async_wrap.cc index 2ffc467ff8baaa..c5b3e48b8104b5 100644 --- a/src/async_wrap.cc +++ b/src/async_wrap.cc @@ -301,19 +301,20 @@ void PromiseWrap::getIsChainedPromise(Local property, info.Holder()->GetInternalField(kIsChainedPromiseField)); } -static void PromiseHook(PromiseHookType type, Local promise, - Local parent, void* arg) { - Environment* env = static_cast(arg); +static PromiseWrap* extractPromiseWrap(Local promise) { Local resource_object_value = promise->GetInternalField(0); - PromiseWrap* wrap = nullptr; if (resource_object_value->IsObject()) { - Local resource_object = resource_object_value.As(); - wrap = Unwrap(resource_object); + return Unwrap(resource_object_value.As()); } + return nullptr; +} +static void PromiseHook(PromiseHookType type, Local promise, + Local parent, void* arg) { + Environment* env = static_cast(arg); + PromiseWrap* wrap = extractPromiseWrap(promise); if (type == PromiseHookType::kInit || wrap == nullptr) { bool silent = type != PromiseHookType::kInit; - PromiseWrap* parent_wrap = nullptr; // set parent promise's async Id as this promise's triggerAsyncId if (parent->IsPromise()) { @@ -321,11 +322,7 @@ static void PromiseHook(PromiseHookType type, Local promise, // is a chained promise, so we set parent promise's id as // current promise's triggerAsyncId Local parent_promise = parent.As(); - Local parent_resource = parent_promise->GetInternalField(0); - if (parent_resource->IsObject()) { - parent_wrap = Unwrap(parent_resource.As()); - } - + PromiseWrap* parent_wrap = extractPromiseWrap(parent_promise); if (parent_wrap == nullptr) { parent_wrap = PromiseWrap::New(env, parent_promise, nullptr, true); } From 042434f9af2f1b2a5687cd8f492db312c34e9f5b Mon Sep 17 00:00:00 2001 From: Daniel Bevenius Date: Wed, 14 Mar 2018 08:10:59 +0100 Subject: [PATCH 093/129] src: fix indenting of wrap->EmitTraceEventBefore PR-URL: https://github.com/nodejs/node/pull/19340 Reviewed-By: James M Snell --- src/async_wrap.cc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/async_wrap.cc b/src/async_wrap.cc index c5b3e48b8104b5..5eda1d4b35e66e 100644 --- a/src/async_wrap.cc +++ b/src/async_wrap.cc @@ -338,7 +338,7 @@ static void PromiseHook(PromiseHookType type, Local promise, if (type == PromiseHookType::kBefore) { env->async_hooks()->push_async_ids( wrap->get_async_id(), wrap->get_trigger_async_id()); - wrap->EmitTraceEventBefore(); + wrap->EmitTraceEventBefore(); AsyncWrap::EmitBefore(wrap->env(), wrap->get_async_id()); } else if (type == PromiseHookType::kAfter) { wrap->EmitTraceEventAfter(); From 9e8f4e5047eb9ba1b0fd3de22f783392ee523e46 Mon Sep 17 00:00:00 2001 From: Daniel Bevenius Date: Wed, 14 Mar 2018 10:56:00 +0100 Subject: [PATCH 094/129] src: remove unused uv.h include from async_wrap.cc MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit PR-URL: https://github.com/nodejs/node/pull/19342 Reviewed-By: Michaël Zasso Reviewed-By: James M Snell Reviewed-By: Colin Ihrig Reviewed-By: Richard Lau Reviewed-By: Tobias Nießen Reviewed-By: Khaidi Chu --- src/async_wrap.cc | 1 - 1 file changed, 1 deletion(-) diff --git a/src/async_wrap.cc b/src/async_wrap.cc index 5eda1d4b35e66e..aba700ac91adef 100644 --- a/src/async_wrap.cc +++ b/src/async_wrap.cc @@ -24,7 +24,6 @@ #include "env-inl.h" #include "util-inl.h" -#include "uv.h" #include "v8.h" #include "v8-profiler.h" From b40706055672864d01cb4957277d5d87fdeed0ab Mon Sep 17 00:00:00 2001 From: Rich Trott Date: Wed, 14 Mar 2018 07:39:07 -0700 Subject: [PATCH 095/129] test: fix flaky test-http2-settings-flood The test is unreliable on some Windows platforms in its current form. Make it more robust by using `setInterval()` to repeat the flooding until an error is triggered. Fixes: https://github.com/nodejs/node/issues/18251 PR-URL: https://github.com/nodejs/node/pull/19349 Reviewed-By: James M Snell Reviewed-By: Anatoli Papirovski --- test/sequential/sequential.status | 1 - test/sequential/test-http2-settings-flood.js | 25 ++++++++++++-------- 2 files changed, 15 insertions(+), 11 deletions(-) diff --git a/test/sequential/sequential.status b/test/sequential/sequential.status index 8c6acd1f36dc8f..d037504a4e66a6 100644 --- a/test/sequential/sequential.status +++ b/test/sequential/sequential.status @@ -14,7 +14,6 @@ test-inspector-debug-end : PASS, FLAKY test-inspector-async-hook-setup-at-signal: PASS, FLAKY test-inspector-stop-profile-after-done: PASS, FLAKY test-http2-ping-flood : PASS, FLAKY -test-http2-settings-flood : PASS, FLAKY [$system==linux] diff --git a/test/sequential/test-http2-settings-flood.js b/test/sequential/test-http2-settings-flood.js index bad4cec9a8d509..15b3696b9c298e 100644 --- a/test/sequential/test-http2-settings-flood.js +++ b/test/sequential/test-http2-settings-flood.js @@ -4,8 +4,10 @@ const common = require('../common'); if (!common.hasCrypto) common.skip('missing crypto'); +const assert = require('assert'); const http2 = require('http2'); const net = require('net'); + const http2util = require('../common/http2'); // Test that settings flooding causes the session to be torn down @@ -14,13 +16,16 @@ const kSettings = new http2util.SettingsFrame(); const server = http2.createServer(); +let interval; + server.on('stream', common.mustNotCall()); server.on('session', common.mustCall((session) => { - session.on('error', common.expectsError({ - code: 'ERR_HTTP2_ERROR', - message: - 'Flooding was detected in this HTTP/2 session, and it must be closed' - })); + session.on('error', (e) => { + assert.strictEqual(e.code, 'ERR_HTTP2_ERROR'); + assert(e.message.includes('Flooding was detected')); + clearInterval(interval); + }); + session.on('close', common.mustCall(() => { server.close(); })); @@ -30,9 +35,7 @@ server.listen(0, common.mustCall(() => { const client = net.connect(server.address().port); // nghttp2 uses a limit of 10000 items in it's outbound queue. - // If this number is exceeded, a flooding error is raised. Set - // this lim higher to account for the ones that nghttp2 is - // successfully able to respond to. + // If this number is exceeded, a flooding error is raised. // TODO(jasnell): Unfortunately, this test is inherently flaky because // it is entirely dependent on how quickly the server is able to handle // the inbound frames and whether those just happen to overflow nghttp2's @@ -40,8 +43,10 @@ server.listen(0, common.mustCall(() => { // from one system to another, and from one test run to another. client.on('connect', common.mustCall(() => { client.write(http2util.kClientMagic, () => { - for (let n = 0; n < 35000; n++) - client.write(kSettings.data); + interval = setInterval(() => { + for (let n = 0; n < 10000; n++) + client.write(kSettings.data); + }, 1); }); })); From 5b2f6508f9c78df3d947b4e93ca34fb3a67b4d8f Mon Sep 17 00:00:00 2001 From: Daniel Bevenius Date: Thu, 15 Mar 2018 07:38:19 +0100 Subject: [PATCH 096/129] src: make AsyncWrap constructors delegate Currently, there is an AsyncWrap constructor that is only used by PromiseWrap. This constructor has a body which is very similar to the other AsyncWrap constructor. This commit suggests updating the private constructor that is used by PromiseWrap and also have the second constructor delegate to this one to avoid the code duplication. PR-URL: https://github.com/nodejs/node/pull/19366 Reviewed-By: James M Snell Reviewed-By: Minwoo Jung Reviewed-By: Anna Henningsen --- src/async_wrap.cc | 23 +++++++---------------- src/async_wrap.h | 7 +++++-- 2 files changed, 12 insertions(+), 18 deletions(-) diff --git a/src/async_wrap.cc b/src/async_wrap.cc index aba700ac91adef..9984985409a0b3 100644 --- a/src/async_wrap.cc +++ b/src/async_wrap.cc @@ -254,7 +254,7 @@ void AsyncWrap::EmitAfter(Environment* env, double async_id) { class PromiseWrap : public AsyncWrap { public: PromiseWrap(Environment* env, Local object, bool silent) - : AsyncWrap(env, object, silent) { + : AsyncWrap(env, object, PROVIDER_PROMISE, -1, silent) { MakeWeak(this); } size_t self_size() const override { return sizeof(*this); } @@ -626,32 +626,23 @@ AsyncWrap::AsyncWrap(Environment* env, Local object, ProviderType provider, double execution_async_id) - : BaseObject(env, object), - provider_type_(provider) { - CHECK_NE(provider, PROVIDER_NONE); - CHECK_GE(object->InternalFieldCount(), 1); + : AsyncWrap(env, object, provider, execution_async_id, false) {} - // Shift provider value over to prevent id collision. - persistent().SetWrapperClassId(NODE_ASYNC_ID_OFFSET + provider); - - // Use AsyncReset() call to execute the init() callbacks. - AsyncReset(execution_async_id); -} - - -// This is specifically used by the PromiseWrap constructor. AsyncWrap::AsyncWrap(Environment* env, Local object, + ProviderType provider, + double execution_async_id, bool silent) : BaseObject(env, object), - provider_type_(PROVIDER_PROMISE) { + provider_type_(provider) { + CHECK_NE(provider, PROVIDER_NONE); CHECK_GE(object->InternalFieldCount(), 1); // Shift provider value over to prevent id collision. persistent().SetWrapperClassId(NODE_ASYNC_ID_OFFSET + provider_type_); // Use AsyncReset() call to execute the init() callbacks. - AsyncReset(-1, silent); + AsyncReset(execution_async_id, silent); } diff --git a/src/async_wrap.h b/src/async_wrap.h index 8325d152ab09c4..2b2fe66c885905 100644 --- a/src/async_wrap.h +++ b/src/async_wrap.h @@ -173,8 +173,11 @@ class AsyncWrap : public BaseObject { private: friend class PromiseWrap; - // This is specifically used by the PromiseWrap constructor. - AsyncWrap(Environment* env, v8::Local promise, bool silent); + AsyncWrap(Environment* env, + v8::Local promise, + ProviderType provider, + double execution_async_id, + bool silent); inline AsyncWrap(); const ProviderType provider_type_; // Because the values may be Reset(), cannot be made const. From 38ed6c2b257a8b14c854c4877ac62499aed4e734 Mon Sep 17 00:00:00 2001 From: Rich Trott Date: Fri, 16 Mar 2018 10:50:03 -0700 Subject: [PATCH 097/129] test: fix flaky test-http2-ping-flood The test is unreliable on some Windows platforms in its current form. Make it more robust by using `setInterval()` to repeat the flooding until an error is triggered. PR-URL: https://github.com/nodejs/node/pull/19395 Reviewed-By: Anna Henningsen Reviewed-By: James M Snell --- test/sequential/sequential.status | 1 - test/sequential/test-http2-ping-flood.js | 24 ++++++++++++++---------- 2 files changed, 14 insertions(+), 11 deletions(-) diff --git a/test/sequential/sequential.status b/test/sequential/sequential.status index d037504a4e66a6..a3dd7b4ccf4b54 100644 --- a/test/sequential/sequential.status +++ b/test/sequential/sequential.status @@ -13,7 +13,6 @@ test-inspector-bindings : PASS, FLAKY test-inspector-debug-end : PASS, FLAKY test-inspector-async-hook-setup-at-signal: PASS, FLAKY test-inspector-stop-profile-after-done: PASS, FLAKY -test-http2-ping-flood : PASS, FLAKY [$system==linux] diff --git a/test/sequential/test-http2-ping-flood.js b/test/sequential/test-http2-ping-flood.js index 5b47d51be9c5a8..b414aca8a4703a 100644 --- a/test/sequential/test-http2-ping-flood.js +++ b/test/sequential/test-http2-ping-flood.js @@ -4,8 +4,10 @@ const common = require('../common'); if (!common.hasCrypto) common.skip('missing crypto'); +const assert = require('assert'); const http2 = require('http2'); const net = require('net'); + const http2util = require('../common/http2'); // Test that ping flooding causes the session to be torn down @@ -15,13 +17,15 @@ const kPing = new http2util.PingFrame(); const server = http2.createServer(); +let interval; + server.on('stream', common.mustNotCall()); server.on('session', common.mustCall((session) => { - session.on('error', common.expectsError({ - code: 'ERR_HTTP2_ERROR', - message: - 'Flooding was detected in this HTTP/2 session, and it must be closed' - })); + session.on('error', (e) => { + assert.strictEqual(e.code, 'ERR_HTTP2_ERROR'); + assert(e.message.includes('Flooding was detected')); + clearInterval(interval); + }); session.on('close', common.mustCall(() => { server.close(); })); @@ -31,9 +35,7 @@ server.listen(0, common.mustCall(() => { const client = net.connect(server.address().port); // nghttp2 uses a limit of 10000 items in it's outbound queue. - // If this number is exceeded, a flooding error is raised. Set - // this lim higher to account for the ones that nghttp2 is - // successfully able to respond to. + // If this number is exceeded, a flooding error is raised. // TODO(jasnell): Unfortunately, this test is inherently flaky because // it is entirely dependent on how quickly the server is able to handle // the inbound frames and whether those just happen to overflow nghttp2's @@ -42,8 +44,10 @@ server.listen(0, common.mustCall(() => { client.on('connect', common.mustCall(() => { client.write(http2util.kClientMagic, () => { client.write(kSettings.data, () => { - for (let n = 0; n < 35000; n++) - client.write(kPing.data); + interval = setInterval(() => { + for (let n = 0; n < 10000; n++) + client.write(kPing.data); + }, 1); }); }); })); From aba1ff202cd5d4a3377321626c3e2661a3564371 Mon Sep 17 00:00:00 2001 From: Rich Trott Date: Fri, 16 Mar 2018 22:12:37 -0700 Subject: [PATCH 098/129] test: refactor test-fs-readfile-tostring-fail The test uses both `assert()` and `assert.ok()`. Use just `assert.ok()`. Remove a comment that does not appear to match the code and does not seem to explain much beyond the bare code itself. PR-URL: https://github.com/nodejs/node/pull/19404 Reviewed-By: Richard Lau Reviewed-By: Luigi Pinca Reviewed-By: Shingo Inoue Reviewed-By: James M Snell Reviewed-By: Colin Ihrig --- test/sequential/test-fs-readfile-tostring-fail.js | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/test/sequential/test-fs-readfile-tostring-fail.js b/test/sequential/test-fs-readfile-tostring-fail.js index 88cf7347efbfdf..81c7a941eb9476 100644 --- a/test/sequential/test-fs-readfile-tostring-fail.js +++ b/test/sequential/test-fs-readfile-tostring-fail.js @@ -30,11 +30,10 @@ for (let i = 0; i < 201; i++) { stream.end(); stream.on('finish', common.mustCall(function() { - // make sure that the toString does not throw an error fs.readFile(file, 'utf8', common.mustCall(function(err, buf) { assert.ok(err instanceof Error); - assert(/^(Array buffer allocation failed|"toString\(\)" failed)$/ - .test(err.message)); + assert.ok(/^(Array buffer allocation failed|"toString\(\)" failed)$/ + .test(err.message)); assert.strictEqual(buf, undefined); })); })); From 9f15bc40b8849623f1d2ca9934e9b827d4e0e94d Mon Sep 17 00:00:00 2001 From: jn99 Date: Fri, 12 Oct 2018 10:36:32 -0700 Subject: [PATCH 099/129] test: skip failing tests for osx mojave MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Refs: https://github.com/nodejs/node/issues/21679 PR-URL: https://github.com/nodejs/node/pull/23550 Reviewed-By: Ruben Bridgewater Reviewed-By: George Adams Reviewed-By: Gus Caplan Reviewed-By: James M Snell Reviewed-By: Santiago Gimeno Reviewed-By: Anna Henningsen Reviewed-By: Colin Ihrig Reviewed-By: Trivikram Kamat Reviewed-By: Michael Dawson Reviewed-By: Сковорода Никита Андреевич --- test/common/index.js | 3 ++- .../test-cluster-bind-privileged-port.js | 25 +++++++++++++++++++ .../test-cluster-bind-privileged-port.js | 5 ++++ ...ster-shared-handle-bind-privileged-port.js | 5 ++++ 4 files changed, 37 insertions(+), 1 deletion(-) create mode 100644 test/known_issues/test-cluster-bind-privileged-port.js diff --git a/test/common/index.js b/test/common/index.js index 2ff682e95d1347..fafea8d17f8474 100644 --- a/test/common/index.js +++ b/test/common/index.js @@ -47,7 +47,8 @@ exports.isSunOS = process.platform === 'sunos'; exports.isFreeBSD = process.platform === 'freebsd'; exports.isOpenBSD = process.platform === 'openbsd'; exports.isLinux = process.platform === 'linux'; -exports.isOSX = process.platform === 'darwin'; +const isOSX = exports.isOSX = process.platform === 'darwin'; +exports.isOSXMojave = isOSX && (os.release().startsWith('18')); exports.enoughTestMem = os.totalmem() > 0x70000000; /* 1.75 Gb */ const cpus = os.cpus(); diff --git a/test/known_issues/test-cluster-bind-privileged-port.js b/test/known_issues/test-cluster-bind-privileged-port.js new file mode 100644 index 00000000000000..6ada04aa030b31 --- /dev/null +++ b/test/known_issues/test-cluster-bind-privileged-port.js @@ -0,0 +1,25 @@ +'use strict'; +const common = require('../common'); + +// This test should fail on macOS (10.14) due to an issue with privileged ports. + +const assert = require('assert'); +const cluster = require('cluster'); +const net = require('net'); + +if (!common.isOSXMojave) + assert.fail('Code should fail only on macOS Mojave.'); + + +if (cluster.isMaster) { + cluster.fork().on('exit', common.mustCall((exitCode) => { + assert.strictEqual(exitCode, 0); + })); +} else { + const s = net.createServer(common.mustNotCall()); + s.listen(42, common.mustNotCall('listen should have failed')); + s.on('error', common.mustCall((err) => { + assert.strictEqual(err.code, 'EACCES'); + process.disconnect(); + })); +} diff --git a/test/parallel/test-cluster-bind-privileged-port.js b/test/parallel/test-cluster-bind-privileged-port.js index 99f7a20c4946be..9d155259c3789e 100644 --- a/test/parallel/test-cluster-bind-privileged-port.js +++ b/test/parallel/test-cluster-bind-privileged-port.js @@ -21,6 +21,11 @@ 'use strict'; const common = require('../common'); + +// Skip on OS X Mojave. https://github.com/nodejs/node/issues/21679 +if (common.isOSXMojave) + common.skip('bypass test for Mojave due to OSX issue'); + if (common.isWindows) common.skip('not reliable on Windows.'); diff --git a/test/parallel/test-cluster-shared-handle-bind-privileged-port.js b/test/parallel/test-cluster-shared-handle-bind-privileged-port.js index 8f05b28fed3308..1edece30af6de9 100644 --- a/test/parallel/test-cluster-shared-handle-bind-privileged-port.js +++ b/test/parallel/test-cluster-shared-handle-bind-privileged-port.js @@ -21,6 +21,11 @@ 'use strict'; const common = require('../common'); + +// Skip on OS X Mojave. https://github.com/nodejs/node/issues/21679 +if (common.isOSXMojave) + common.skip('bypass test for Mojave due to OSX issue'); + if (common.isWindows) common.skip('not reliable on Windows'); From 104fbc64eda5483fa19a304dfa6f2510095f4f00 Mon Sep 17 00:00:00 2001 From: Gibson Fahnestock Date: Tue, 6 Mar 2018 08:20:44 +0000 Subject: [PATCH 100/129] build: update arm64 minimum supported platform This is already true in practice. Backport-PR-URL: https://github.com/nodejs/node/pull/23275 PR-URL: https://github.com/nodejs/node/pull/19164 Fixes: https://github.com/nodejs/build/issues/1164 Reviewed-By: Rod Vagg Reviewed-By: Michael Dawson --- BUILDING.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/BUILDING.md b/BUILDING.md index 66c0ffc5aed226..1ecdc5abb17adc 100644 --- a/BUILDING.md +++ b/BUILDING.md @@ -39,7 +39,8 @@ in production. | System | Support type | Version | Architectures | Notes | |--------------|--------------|----------------------------------|----------------------|------------------| -| GNU/Linux | Tier 1 | kernel >= 2.6.32, glibc >= 2.12 | x86, x64, arm, arm64 | | +| GNU/Linux | Tier 1 | kernel >= 2.6.32, glibc >= 2.12 | x86, x64, arm | | +| GNU/Linux | Tier 1 | kernel >= 3.10, glibc >= 2.17 | arm64 | | | macOS | Tier 1 | >= 10.10 | x64 | | | Windows | Tier 1 | >= Windows 7 / 2008 R2 | x86, x64 | vs2015 or vs2017 | | SmartOS | Tier 2 | >= 15 < 16.4 | x86, x64 | see note1 | From 7ab253f62e7917b052c297025e6ce33de35d495e Mon Sep 17 00:00:00 2001 From: Yang Guo Date: Fri, 5 Oct 2018 12:35:05 +0200 Subject: [PATCH 101/129] deps: V8: cherry-pick 64-bit hash seed commits MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This serves as mitigation for the so-called HashWick vulnerability. Original commit messages: commit d5686a74d56fbb6985b22663ddadd66eb7b91519 Author: Yang Guo Date: Mon Jul 16 11:19:42 2018 Extend hash seed to 64 bits R=bmeurer@chromium.org, ulan@chromium.org Bug: chromium:680662 Change-Id: I5e1486ad2a42db2998d5485a0c4e711378678e6c Reviewed-on: https://chromium-review.googlesource.com/1136034 Reviewed-by: Marja Hölttä Reviewed-by: Ulan Degenbaev Reviewed-by: Benedikt Meurer Commit-Queue: Yang Guo Cr-Commit-Position: refs/heads/master@{#54460} commit 3833fef57368c53c6170559ffa524c8c69f16ee5 Author: Yang Guo Date: Thu Sep 20 11:43:13 2018 Refactor integer hashing function names We now clearly differentiate between: - unseeded hash for 32-bit integers - unseeded hash for 64-bit integers - seeded hash for 32-bit integers - seeded hash for strings R=bmeurer@chromium.org Bug: chromium:680662 Change-Id: I7459958c4158ee3501c962943dff8f33258bb5ce Reviewed-on: https://chromium-review.googlesource.com/1235973 Commit-Queue: Yang Guo Reviewed-by: Benedikt Meurer Cr-Commit-Position: refs/heads/master@{#56068} commit 95a979e02d7154e45b293261a6998c99d71fc238 Author: Yang Guo Date: Thu Sep 20 14:34:48 2018 Call into C++ to compute seeded integer hash R=bmeurer@chromium.org Bug: chromium:680662 Change-Id: I8dace89d576dfcc5833fd539ce698a9ade1cb5a0 Reviewed-on: https://chromium-review.googlesource.com/1235928 Commit-Queue: Yang Guo Reviewed-by: Benedikt Meurer Cr-Commit-Position: refs/heads/master@{#56091} commit 2c2af0022d5feb9e525a00a76cb15db9f3e38dba Author: Yang Guo Date: Thu Sep 27 16:37:57 2018 Use 64-bit for seeded integer hashes R=petermarshall@chromium.org Bug: chromium:680662 Change-Id: If48d1043dbe1e1bb695ec890c23e103a6cacf2d4 Reviewed-on: https://chromium-review.googlesource.com/1244220 Commit-Queue: Yang Guo Reviewed-by: Peter Marshall Cr-Commit-Position: refs/heads/master@{#56271} Refs: https://github.com/nodejs/node/issues/23259 PR-URL: https://github.com/nodejs/node/pull/23274 Reviewed-By: Anna Henningsen Reviewed-By: Matteo Collina --- deps/v8/include/v8-version.h | 2 +- deps/v8/src/assembler.cc | 10 +++++ deps/v8/src/assembler.h | 2 +- deps/v8/src/ast/ast-value-factory.cc | 1 - deps/v8/src/ast/ast-value-factory.h | 10 ++--- .../src/builtins/builtins-collections-gen.cc | 11 +++-- deps/v8/src/code-stub-assembler.cc | 44 ++++++++++++------- deps/v8/src/code-stub-assembler.h | 5 +-- deps/v8/src/external-reference-table.cc | 2 + deps/v8/src/flag-definitions.h | 7 +-- deps/v8/src/flags.cc | 39 ++++++++++++++++ deps/v8/src/frames.cc | 3 +- deps/v8/src/heap/heap-inl.h | 6 +-- deps/v8/src/heap/heap.cc | 15 ++++--- deps/v8/src/heap/heap.h | 5 ++- deps/v8/src/json-parser.cc | 4 +- deps/v8/src/objects-inl.h | 11 +++-- deps/v8/src/objects.cc | 16 +++---- deps/v8/src/objects/hash-table.h | 2 +- deps/v8/src/objects/string-inl.h | 8 ++-- deps/v8/src/parsing/parse-info.h | 6 +-- deps/v8/src/profiler/allocation-tracker.cc | 2 +- deps/v8/src/profiler/heap-profiler.cc | 4 +- .../src/profiler/heap-snapshot-generator.cc | 2 +- .../v8/src/profiler/heap-snapshot-generator.h | 4 +- deps/v8/src/profiler/profile-generator.cc | 18 ++++---- deps/v8/src/profiler/profile-generator.h | 2 +- deps/v8/src/profiler/profiler-listener.cc | 3 +- deps/v8/src/profiler/strings-storage.cc | 5 +-- deps/v8/src/profiler/strings-storage.h | 3 +- deps/v8/src/string-hasher-inl.h | 10 ++--- deps/v8/src/string-hasher.h | 10 ++--- deps/v8/src/utils.h | 19 ++++---- deps/v8/test/cctest/heap/test-heap.cc | 2 +- .../test/cctest/test-code-stub-assembler.cc | 12 +++-- deps/v8/test/cctest/test-serialize.cc | 3 +- 36 files changed, 180 insertions(+), 128 deletions(-) diff --git a/deps/v8/include/v8-version.h b/deps/v8/include/v8-version.h index f2531abb21f0aa..1e44f2b4a266a9 100644 --- a/deps/v8/include/v8-version.h +++ b/deps/v8/include/v8-version.h @@ -11,7 +11,7 @@ #define V8_MAJOR_VERSION 6 #define V8_MINOR_VERSION 2 #define V8_BUILD_NUMBER 414 -#define V8_PATCH_LEVEL 68 +#define V8_PATCH_LEVEL 69 // Use 1 for candidates and 0 otherwise. // (Boolean macro values are not supported by all preprocessors.) diff --git a/deps/v8/src/assembler.cc b/deps/v8/src/assembler.cc index 35238081f9864f..cb5fc84500506d 100644 --- a/deps/v8/src/assembler.cc +++ b/deps/v8/src/assembler.cc @@ -1432,6 +1432,16 @@ ExternalReference ExternalReference::check_object_type(Isolate* isolate) { return ExternalReference(Redirect(isolate, FUNCTION_ADDR(CheckObjectType))); } +static uint32_t ComputeSeededIntegerHash(Isolate* isolate, uint32_t key) { + DisallowHeapAllocation no_gc; + return ComputeSeededHash(key, isolate->heap()->HashSeed()); +} + +ExternalReference ExternalReference::compute_integer_hash(Isolate* isolate) { + return ExternalReference( + Redirect(isolate, FUNCTION_ADDR(ComputeSeededIntegerHash))); +} + #ifdef V8_INTL_SUPPORT ExternalReference ExternalReference::intl_convert_one_byte_to_lower( Isolate* isolate) { diff --git a/deps/v8/src/assembler.h b/deps/v8/src/assembler.h index aeecaa167c94de..17c8d1115478f0 100644 --- a/deps/v8/src/assembler.h +++ b/deps/v8/src/assembler.h @@ -979,7 +979,7 @@ class ExternalReference BASE_EMBEDDED { static ExternalReference try_internalize_string_function(Isolate* isolate); static ExternalReference check_object_type(Isolate* isolate); - + static ExternalReference compute_integer_hash(Isolate* isolate); #ifdef V8_INTL_SUPPORT static ExternalReference intl_convert_one_byte_to_lower(Isolate* isolate); static ExternalReference intl_to_latin1_lower_table(Isolate* isolate); diff --git a/deps/v8/src/ast/ast-value-factory.cc b/deps/v8/src/ast/ast-value-factory.cc index c9c89d7745adde..b7bcb84e18e9ae 100644 --- a/deps/v8/src/ast/ast-value-factory.cc +++ b/deps/v8/src/ast/ast-value-factory.cc @@ -202,7 +202,6 @@ bool AstValue::BooleanValue() const { UNREACHABLE(); } - void AstValue::Internalize(Isolate* isolate) { switch (type_) { case STRING: diff --git a/deps/v8/src/ast/ast-value-factory.h b/deps/v8/src/ast/ast-value-factory.h index b72e34a36c17e8..9f10307555d9d4 100644 --- a/deps/v8/src/ast/ast-value-factory.h +++ b/deps/v8/src/ast/ast-value-factory.h @@ -361,7 +361,7 @@ class AstValue : public ZoneObject { class AstStringConstants final { public: - AstStringConstants(Isolate* isolate, uint32_t hash_seed) + AstStringConstants(Isolate* isolate, uint64_t hash_seed) : zone_(isolate->allocator(), ZONE_NAME), string_table_(AstRawString::Compare), hash_seed_(hash_seed) { @@ -391,7 +391,7 @@ class AstStringConstants final { STRING_CONSTANTS(F) #undef F - uint32_t hash_seed() const { return hash_seed_; } + uint64_t hash_seed() const { return hash_seed_; } const base::CustomMatcherHashMap* string_table() const { return &string_table_; } @@ -399,7 +399,7 @@ class AstStringConstants final { private: Zone zone_; base::CustomMatcherHashMap string_table_; - uint32_t hash_seed_; + uint64_t hash_seed_; #define F(name, str) AstRawString* name##_string_; STRING_CONSTANTS(F) @@ -418,7 +418,7 @@ class AstStringConstants final { class AstValueFactory { public: AstValueFactory(Zone* zone, const AstStringConstants* string_constants, - uint32_t hash_seed) + uint64_t hash_seed) : string_table_(string_constants->string_table()), values_(nullptr), strings_(nullptr), @@ -535,7 +535,7 @@ class AstValueFactory { Zone* zone_; - uint32_t hash_seed_; + uint64_t hash_seed_; #define F(name) AstValue* name##_; OTHER_CONSTANTS(F) diff --git a/deps/v8/src/builtins/builtins-collections-gen.cc b/deps/v8/src/builtins/builtins-collections-gen.cc index acb4c949ae11a8..9c8ce5ab1d4d6e 100644 --- a/deps/v8/src/builtins/builtins-collections-gen.cc +++ b/deps/v8/src/builtins/builtins-collections-gen.cc @@ -92,7 +92,7 @@ class CollectionsBuiltinsAssembler : public CodeStubAssembler { Node* key_tagged, Variable* result, Label* entry_found, Label* not_found); - Node* ComputeIntegerHashForString(Node* context, Node* string_key); + Node* ComputeStringHash(Node* context, Node* string_key); void SameValueZeroString(Node* context, Node* key_string, Node* candidate_key, Label* if_same, Label* if_not_same); @@ -515,8 +515,7 @@ void CollectionsBuiltinsAssembler::FindOrderedHashTableEntryForSmiKey( Node* table, Node* smi_key, Variable* result, Label* entry_found, Label* not_found) { Node* const key_untagged = SmiUntag(smi_key); - Node* const hash = - ChangeInt32ToIntPtr(ComputeIntegerHash(key_untagged, Int32Constant(0))); + Node* const hash = ChangeInt32ToIntPtr(ComputeUnseededHash(key_untagged)); CSA_ASSERT(this, IntPtrGreaterThanOrEqual(hash, IntPtrConstant(0))); result->Bind(hash); FindOrderedHashTableEntry( @@ -531,7 +530,7 @@ template void CollectionsBuiltinsAssembler::FindOrderedHashTableEntryForStringKey( Node* context, Node* table, Node* key_tagged, Variable* result, Label* entry_found, Label* not_found) { - Node* const hash = ComputeIntegerHashForString(context, key_tagged); + Node* const hash = ComputeStringHash(context, key_tagged); CSA_ASSERT(this, IntPtrGreaterThanOrEqual(hash, IntPtrConstant(0))); result->Bind(hash); FindOrderedHashTableEntry( @@ -573,8 +572,8 @@ void CollectionsBuiltinsAssembler::FindOrderedHashTableEntryForOtherKey( result, entry_found, not_found); } -Node* CollectionsBuiltinsAssembler::ComputeIntegerHashForString( - Node* context, Node* string_key) { +Node* CollectionsBuiltinsAssembler::ComputeStringHash(Node* context, + Node* string_key) { VARIABLE(var_result, MachineType::PointerRepresentation()); Label hash_not_computed(this), done(this, &var_result); diff --git a/deps/v8/src/code-stub-assembler.cc b/deps/v8/src/code-stub-assembler.cc index 915f507b12689c..b48e1afae5274f 100644 --- a/deps/v8/src/code-stub-assembler.cc +++ b/deps/v8/src/code-stub-assembler.cc @@ -200,10 +200,6 @@ HEAP_CONSTANT_LIST(HEAP_CONSTANT_ACCESSOR); HEAP_CONSTANT_LIST(HEAP_CONSTANT_TEST); #undef HEAP_CONSTANT_TEST -Node* CodeStubAssembler::HashSeed() { - return LoadAndUntagToWord32Root(Heap::kHashSeedRootIndex); -} - Node* CodeStubAssembler::StaleRegisterConstant() { return LoadRoot(Heap::kStaleRegisterRootIndex); } @@ -5389,22 +5385,32 @@ template void CodeStubAssembler::NameDictionaryLookup( template void CodeStubAssembler::NameDictionaryLookup( Node*, Node*, Label*, Variable*, Label*, int, LookupMode); -Node* CodeStubAssembler::ComputeIntegerHash(Node* key) { - return ComputeIntegerHash(key, IntPtrConstant(kZeroHashSeed)); -} - -Node* CodeStubAssembler::ComputeIntegerHash(Node* key, Node* seed) { - // See v8::internal::ComputeIntegerHash() +Node* CodeStubAssembler::ComputeUnseededHash(Node* key) { + // See v8::internal::ComputeUnseededHash() Node* hash = TruncateWordToWord32(key); - hash = Word32Xor(hash, seed); - hash = Int32Add(Word32Xor(hash, Int32Constant(0xffffffff)), + hash = Int32Add(Word32Xor(hash, Int32Constant(0xFFFFFFFF)), Word32Shl(hash, Int32Constant(15))); hash = Word32Xor(hash, Word32Shr(hash, Int32Constant(12))); hash = Int32Add(hash, Word32Shl(hash, Int32Constant(2))); hash = Word32Xor(hash, Word32Shr(hash, Int32Constant(4))); hash = Int32Mul(hash, Int32Constant(2057)); hash = Word32Xor(hash, Word32Shr(hash, Int32Constant(16))); - return Word32And(hash, Int32Constant(0x3fffffff)); + return Word32And(hash, Int32Constant(0x3FFFFFFF)); +} + +Node* CodeStubAssembler::ComputeSeededHash(Node* key) { + Node* const function_addr = + ExternalConstant(ExternalReference::compute_integer_hash(isolate())); + Node* const isolate_ptr = + ExternalConstant(ExternalReference::isolate_address(isolate())); + + MachineType type_ptr = MachineType::Pointer(); + MachineType type_uint32 = MachineType::Uint32(); + + Node* const result = + CallCFunction2(type_uint32, type_ptr, type_uint32, function_addr, + isolate_ptr, TruncateWordToWord32(key)); + return result; } template @@ -5420,10 +5426,14 @@ void CodeStubAssembler::NumberDictionaryLookup(Node* dictionary, Node* capacity = SmiUntag(GetCapacity(dictionary)); Node* mask = IntPtrSub(capacity, IntPtrConstant(1)); - Node* int32_seed = std::is_same::value - ? HashSeed() - : Int32Constant(kZeroHashSeed); - Node* hash = ChangeUint32ToWord(ComputeIntegerHash(intptr_index, int32_seed)); + Node* hash; + + if (std::is_same::value) { + hash = ChangeUint32ToWord(ComputeSeededHash(intptr_index)); + } else { + hash = ChangeUint32ToWord(ComputeUnseededHash(intptr_index)); + } + Node* key_as_float64 = RoundIntPtrToFloat64(intptr_index); // See Dictionary::FirstProbe(). diff --git a/deps/v8/src/code-stub-assembler.h b/deps/v8/src/code-stub-assembler.h index 8379663297d6e7..4ee6fd8c1f5bb5 100644 --- a/deps/v8/src/code-stub-assembler.h +++ b/deps/v8/src/code-stub-assembler.h @@ -170,7 +170,6 @@ class V8_EXPORT_PRIVATE CodeStubAssembler : public compiler::CodeAssembler { HEAP_CONSTANT_LIST(HEAP_CONSTANT_TEST) #undef HEAP_CONSTANT_TEST - Node* HashSeed(); Node* StaleRegisterConstant(); Node* IntPtrOrSmiConstant(int value, ParameterMode mode); @@ -1276,8 +1275,8 @@ class V8_EXPORT_PRIVATE CodeStubAssembler : public compiler::CodeAssembler { int inlined_probes = kInlinedDictionaryProbes, LookupMode mode = kFindExisting); - Node* ComputeIntegerHash(Node* key); - Node* ComputeIntegerHash(Node* key, Node* seed); + Node* ComputeUnseededHash(Node* key); + Node* ComputeSeededHash(Node* key); template void NumberDictionaryLookup(Node* dictionary, Node* intptr_index, diff --git a/deps/v8/src/external-reference-table.cc b/deps/v8/src/external-reference-table.cc index 18edc4289bf04b..b9a49524d0840e 100644 --- a/deps/v8/src/external-reference-table.cc +++ b/deps/v8/src/external-reference-table.cc @@ -232,6 +232,8 @@ void ExternalReferenceTable::AddReferences(Isolate* isolate) { "try_internalize_string_function"); Add(ExternalReference::check_object_type(isolate).address(), "check_object_type"); + Add(ExternalReference::compute_integer_hash(isolate).address(), + "ComputeSeededHash"); #ifdef V8_INTL_SUPPORT Add(ExternalReference::intl_convert_one_byte_to_lower(isolate).address(), "intl_convert_one_byte_to_lower"); diff --git a/deps/v8/src/flag-definitions.h b/deps/v8/src/flag-definitions.h index 7fc25bcc91b66e..1576202c9a031e 100644 --- a/deps/v8/src/flag-definitions.h +++ b/deps/v8/src/flag-definitions.h @@ -160,6 +160,7 @@ struct MaybeBoolFlag { FLAG(MAYBE_BOOL, MaybeBoolFlag, nam, {false COMMA false}, cmt) #define DEFINE_INT(nam, def, cmt) FLAG(INT, int, nam, def, cmt) #define DEFINE_UINT(nam, def, cmt) FLAG(UINT, unsigned int, nam, def, cmt) +#define DEFINE_UINT64(nam, def, cmt) FLAG(UINT64, uint64_t, nam, def, cmt) #define DEFINE_FLOAT(nam, def, cmt) FLAG(FLOAT, double, nam, def, cmt) #define DEFINE_STRING(nam, def, cmt) FLAG(STRING, const char*, nam, def, cmt) #define DEFINE_ARGS(nam, cmt) FLAG(ARGS, JSArguments, nam, {0 COMMA NULL}, cmt) @@ -914,9 +915,9 @@ DEFINE_BOOL(randomize_hashes, true, "(with snapshots this option cannot override the baked-in seed)") DEFINE_BOOL(rehash_snapshot, true, "rehash strings from the snapshot to override the baked-in seed") -DEFINE_INT(hash_seed, 0, - "Fixed seed to use to hash property keys (0 means random)" - "(with snapshots this option cannot override the baked-in seed)") +DEFINE_UINT64(hash_seed, 0, + "Fixed seed to use to hash property keys (0 means random)" + "(with snapshots this option cannot override the baked-in seed)") DEFINE_INT(random_seed, 0, "Default seed for initializing random generator " "(0, the default, means to use system random).") diff --git a/deps/v8/src/flags.cc b/deps/v8/src/flags.cc index 9fdc5d04be84dd..edf0d4681253a3 100644 --- a/deps/v8/src/flags.cc +++ b/deps/v8/src/flags.cc @@ -39,6 +39,7 @@ struct Flag { TYPE_MAYBE_BOOL, TYPE_INT, TYPE_UINT, + TYPE_UINT64, TYPE_FLOAT, TYPE_STRING, TYPE_ARGS @@ -77,6 +78,11 @@ struct Flag { return reinterpret_cast(valptr_); } + uint64_t* uint64_variable() const { + DCHECK(type_ == TYPE_UINT64); + return reinterpret_cast(valptr_); + } + double* float_variable() const { DCHECK(type_ == TYPE_FLOAT); return reinterpret_cast(valptr_); @@ -115,6 +121,11 @@ struct Flag { return *reinterpret_cast(defptr_); } + uint64_t uint64_default() const { + DCHECK(type_ == TYPE_UINT64); + return *reinterpret_cast(defptr_); + } + double float_default() const { DCHECK(type_ == TYPE_FLOAT); return *reinterpret_cast(defptr_); @@ -141,6 +152,8 @@ struct Flag { return *int_variable() == int_default(); case TYPE_UINT: return *uint_variable() == uint_default(); + case TYPE_UINT64: + return *uint64_variable() == uint64_default(); case TYPE_FLOAT: return *float_variable() == float_default(); case TYPE_STRING: { @@ -171,6 +184,9 @@ struct Flag { case TYPE_UINT: *uint_variable() = uint_default(); break; + case TYPE_UINT64: + *uint64_variable() = uint64_default(); + break; case TYPE_FLOAT: *float_variable() = float_default(); break; @@ -201,6 +217,8 @@ static const char* Type2String(Flag::FlagType type) { case Flag::TYPE_INT: return "int"; case Flag::TYPE_UINT: return "uint"; + case Flag::TYPE_UINT64: + return "uint64"; case Flag::TYPE_FLOAT: return "float"; case Flag::TYPE_STRING: return "string"; case Flag::TYPE_ARGS: return "arguments"; @@ -225,6 +243,9 @@ std::ostream& operator<<(std::ostream& os, const Flag& flag) { // NOLINT case Flag::TYPE_UINT: os << *flag.uint_variable(); break; + case Flag::TYPE_UINT64: + os << *flag.uint64_variable(); + break; case Flag::TYPE_FLOAT: os << *flag.float_variable(); break; @@ -443,6 +464,24 @@ int FlagList::SetFlagsFromCommandLine(int* argc, *flag->uint_variable() = static_cast(val); break; } + case Flag::TYPE_UINT64: { + // We do not use strtoul because it accepts negative numbers. + int64_t val = static_cast(strtoll(value, &endp, 10)); + if (val < 0 || val > std::numeric_limits::max()) { + PrintF(stderr, + "Error: Value for flag %s of type %s is out of bounds " + "[0-%" PRIu64 + "]\n" + "Try --help for options\n", + arg, Type2String(flag->type()), + static_cast( + std::numeric_limits::max())); + return_code = j; + break; + } + *flag->uint64_variable() = static_cast(val); + break; + } case Flag::TYPE_FLOAT: *flag->float_variable() = strtod(value, &endp); break; diff --git a/deps/v8/src/frames.cc b/deps/v8/src/frames.cc index 37f5b4d9cf1673..665cd04d3e4d02 100644 --- a/deps/v8/src/frames.cc +++ b/deps/v8/src/frames.cc @@ -2132,7 +2132,8 @@ InnerPointerToCodeCache::InnerPointerToCodeCacheEntry* InnerPointerToCodeCache::GetCacheEntry(Address inner_pointer) { isolate_->counters()->pc_to_code()->Increment(); DCHECK(base::bits::IsPowerOfTwo(kInnerPointerToCodeCacheSize)); - uint32_t hash = ComputeIntegerHash(ObjectAddressForHashing(inner_pointer)); + uint32_t hash = ComputeUnseededHash( + ObjectAddressForHashing(reinterpret_cast(inner_pointer))); uint32_t index = hash & (kInnerPointerToCodeCacheSize - 1); InnerPointerToCodeCacheEntry* entry = cache(index); if (entry->inner_pointer == inner_pointer) { diff --git a/deps/v8/src/heap/heap-inl.h b/deps/v8/src/heap/heap-inl.h index 49d69c80e23642..7abad16bf98cba 100644 --- a/deps/v8/src/heap/heap-inl.h +++ b/deps/v8/src/heap/heap-inl.h @@ -614,13 +614,13 @@ Oddball* Heap::ToBoolean(bool condition) { return condition ? true_value() : false_value(); } -uint32_t Heap::HashSeed() { - uint32_t seed = static_cast(hash_seed()->value()); +uint64_t Heap::HashSeed() { + uint64_t seed; + hash_seed()->copy_out(0, reinterpret_cast(&seed), kInt64Size); DCHECK(FLAG_randomize_hashes || seed == 0); return seed; } - int Heap::NextScriptId() { int last_id = last_script_id()->value(); if (last_id == Smi::kMaxValue) { diff --git a/deps/v8/src/heap/heap.cc b/deps/v8/src/heap/heap.cc index f736977d227e7e..502b679dd80030 100644 --- a/deps/v8/src/heap/heap.cc +++ b/deps/v8/src/heap/heap.cc @@ -2825,6 +2825,9 @@ void Heap::CreateInitialObjects() { set_minus_infinity_value( *factory->NewHeapNumber(-V8_INFINITY, IMMUTABLE, TENURED)); + set_hash_seed(*factory->NewByteArray(kInt64Size, TENURED)); + InitializeHashSeed(); + // Allocate initial string table. set_string_table(*StringTable::New(isolate(), kInitialStringTableSize)); @@ -5929,10 +5932,6 @@ bool Heap::SetUp() { space_[LO_SPACE] = lo_space_ = new LargeObjectSpace(this, LO_SPACE); if (!lo_space_->SetUp()) return false; - // Set up the seed that is used to randomize the string hash function. - DCHECK(hash_seed() == 0); - if (FLAG_randomize_hashes) InitializeHashSeed(); - for (int i = 0; i < static_cast(v8::Isolate::kUseCounterFeatureCount); i++) { deferred_counters_[i] = 0; @@ -5970,12 +5969,14 @@ bool Heap::SetUp() { } void Heap::InitializeHashSeed() { + uint64_t new_hash_seed; if (FLAG_hash_seed == 0) { - int rnd = isolate()->random_number_generator()->NextInt(); - set_hash_seed(Smi::FromInt(rnd & Name::kHashBitMask)); + int64_t rnd = isolate()->random_number_generator()->NextInt64(); + new_hash_seed = static_cast(rnd); } else { - set_hash_seed(Smi::FromInt(FLAG_hash_seed)); + new_hash_seed = static_cast(FLAG_hash_seed); } + hash_seed()->copy_in(0, reinterpret_cast(&new_hash_seed), kInt64Size); } bool Heap::CreateHeapObjects() { diff --git a/deps/v8/src/heap/heap.h b/deps/v8/src/heap/heap.h index 9b0bbc6293ff73..bcae0e5130cbfb 100644 --- a/deps/v8/src/heap/heap.h +++ b/deps/v8/src/heap/heap.h @@ -228,6 +228,8 @@ using v8::MemoryPressureLevel; V(FixedArray, serialized_templates, SerializedTemplates) \ V(FixedArray, serialized_global_proxy_sizes, SerializedGlobalProxySizes) \ V(TemplateList, message_listeners, MessageListeners) \ + /* Hash seed */ \ + V(ByteArray, hash_seed, HashSeed) \ /* per-Isolate map for JSPromiseCapability. */ \ /* TODO(caitp): Make this a Struct */ \ V(Map, js_promise_capability_map, JSPromiseCapabilityMap) \ @@ -240,7 +242,6 @@ using v8::MemoryPressureLevel; V(Smi, stack_limit, StackLimit) \ V(Smi, real_stack_limit, RealStackLimit) \ V(Smi, last_script_id, LastScriptId) \ - V(Smi, hash_seed, HashSeed) \ /* To distinguish the function templates, so that we can find them in the */ \ /* function cache of the native context. */ \ V(Smi, next_template_serial_number, NextTemplateSerialNumber) \ @@ -814,7 +815,7 @@ class Heap { void IncrementDeferredCount(v8::Isolate::UseCounterFeature feature); - inline uint32_t HashSeed(); + inline uint64_t HashSeed(); inline int NextScriptId(); diff --git a/deps/v8/src/json-parser.cc b/deps/v8/src/json-parser.cc index 32e4187c8edb3d..f5b0363f290301 100644 --- a/deps/v8/src/json-parser.cc +++ b/deps/v8/src/json-parser.cc @@ -793,9 +793,11 @@ Handle JsonParser::ScanJsonString() { // We intentionally use local variables instead of fields, compute hash // while we are iterating a string and manually inline StringTable lookup // here. - uint32_t running_hash = isolate()->heap()->HashSeed(); + uint32_t running_hash = + static_cast(isolate()->heap()->HashSeed()); int position = position_; uc32 c0 = c0_; + do { if (c0 == '\\') { c0_ = c0; diff --git a/deps/v8/src/objects-inl.h b/deps/v8/src/objects-inl.h index 91fd08a2dd58c8..b02f59bbe7d79a 100644 --- a/deps/v8/src/objects-inl.h +++ b/deps/v8/src/objects-inl.h @@ -5844,13 +5844,13 @@ bool NumberDictionaryShape::IsMatch(uint32_t key, Object* other) { } uint32_t UnseededNumberDictionaryShape::Hash(Isolate* isolate, uint32_t key) { - return ComputeIntegerHash(key); + return ComputeUnseededHash(key); } uint32_t UnseededNumberDictionaryShape::HashForObject(Isolate* isolate, Object* other) { DCHECK(other->IsNumber()); - return ComputeIntegerHash(static_cast(other->Number())); + return ComputeUnseededHash(static_cast(other->Number())); } Map* UnseededNumberDictionaryShape::GetMap(Isolate* isolate) { @@ -5858,14 +5858,14 @@ Map* UnseededNumberDictionaryShape::GetMap(Isolate* isolate) { } uint32_t SeededNumberDictionaryShape::Hash(Isolate* isolate, uint32_t key) { - return ComputeIntegerHash(key, isolate->heap()->HashSeed()); + return ComputeSeededHash(key, isolate->heap()->HashSeed()); } uint32_t SeededNumberDictionaryShape::HashForObject(Isolate* isolate, Object* other) { DCHECK(other->IsNumber()); - return ComputeIntegerHash(static_cast(other->Number()), - isolate->heap()->HashSeed()); + return ComputeSeededHash(static_cast(other->Number()), + isolate->heap()->HashSeed()); } @@ -5936,7 +5936,6 @@ uint32_t ObjectHashTableShape::HashForObject(Isolate* isolate, Object* other) { return Smi::ToInt(other->GetHash()); } - Handle ObjectHashTableShape::AsHandle(Isolate* isolate, Handle key) { return key; diff --git a/deps/v8/src/objects.cc b/deps/v8/src/objects.cc index 25bf0650245626..3dabc3cf0bd72d 100644 --- a/deps/v8/src/objects.cc +++ b/deps/v8/src/objects.cc @@ -2305,7 +2305,7 @@ Object* GetSimpleHash(Object* object) { // The object is either a Smi, a HeapNumber, a name, an odd-ball, a real JS // object, or a Harmony proxy. if (object->IsSmi()) { - uint32_t hash = ComputeIntegerHash(Smi::ToInt(object)); + uint32_t hash = ComputeUnseededHash(Smi::ToInt(object)); return Smi::FromInt(hash & Smi::kMaxValue); } if (object->IsHeapNumber()) { @@ -2313,7 +2313,7 @@ Object* GetSimpleHash(Object* object) { if (std::isnan(num)) return Smi::FromInt(Smi::kMaxValue); if (i::IsMinusZero(num)) num = 0; if (IsSmiDouble(num)) { - return Smi::FromInt(FastD2I(num))->GetHash(); + return Smi::FromInt(ComputeUnseededHash(FastD2I(num))); } uint32_t hash = ComputeLongHash(double_to_uint64(num)); return Smi::FromInt(hash & Smi::kMaxValue); @@ -12152,9 +12152,7 @@ uint32_t StringHasher::GetHashField() { } } - -uint32_t StringHasher::ComputeUtf8Hash(Vector chars, - uint32_t seed, +uint32_t StringHasher::ComputeUtf8Hash(Vector chars, uint64_t seed, int* utf16_length_out) { int vector_length = chars.length(); // Handle some edge cases @@ -16838,7 +16836,7 @@ Handle JSGlobalObject::EnsureEmptyPropertyCell( // algorithm. class TwoCharHashTableKey : public StringTableKey { public: - TwoCharHashTableKey(uint16_t c1, uint16_t c2, uint32_t seed) + TwoCharHashTableKey(uint16_t c1, uint16_t c2, uint64_t seed) : StringTableKey(ComputeHashField(c1, c2, seed)), c1_(c1), c2_(c2) {} bool IsMatch(Object* o) override { @@ -16855,9 +16853,9 @@ class TwoCharHashTableKey : public StringTableKey { } private: - uint32_t ComputeHashField(uint16_t c1, uint16_t c2, uint32_t seed) { + uint32_t ComputeHashField(uint16_t c1, uint16_t c2, uint64_t seed) { // Char 1. - uint32_t hash = seed; + uint32_t hash = static_cast(seed); hash += c1; hash += hash << 10; hash ^= hash >> 6; @@ -17030,7 +17028,7 @@ namespace { class StringTableNoAllocateKey : public StringTableKey { public: - StringTableNoAllocateKey(String* string, uint32_t seed) + StringTableNoAllocateKey(String* string, uint64_t seed) : StringTableKey(0), string_(string) { StringShape shape(string); one_byte_ = shape.HasOnlyOneByteChars(); diff --git a/deps/v8/src/objects/hash-table.h b/deps/v8/src/objects/hash-table.h index 6b5682535ab4ce..68ea45115f9e79 100644 --- a/deps/v8/src/objects/hash-table.h +++ b/deps/v8/src/objects/hash-table.h @@ -466,7 +466,7 @@ class OrderedHashTable : public OrderedHashTableBase { // This special cases for Smi, so that we avoid the HandleScope // creation below. if (key->IsSmi()) { - uint32_t hash = ComputeIntegerHash(Smi::ToInt(key)); + uint32_t hash = ComputeUnseededHash(Smi::ToInt(key)); return HashToEntry(hash & Smi::kMaxValue); } HandleScope scope(isolate); diff --git a/deps/v8/src/objects/string-inl.h b/deps/v8/src/objects/string-inl.h index a30b65da956c62..7208ef28dcc9f0 100644 --- a/deps/v8/src/objects/string-inl.h +++ b/deps/v8/src/objects/string-inl.h @@ -195,7 +195,7 @@ Char FlatStringReader::Get(int index) { template class SequentialStringKey : public StringTableKey { public: - explicit SequentialStringKey(Vector string, uint32_t seed) + explicit SequentialStringKey(Vector string, uint64_t seed) : StringTableKey(StringHasher::HashSequentialString( string.start(), string.length(), seed)), string_(string) {} @@ -205,7 +205,7 @@ class SequentialStringKey : public StringTableKey { class OneByteStringKey : public SequentialStringKey { public: - OneByteStringKey(Vector str, uint32_t seed) + OneByteStringKey(Vector str, uint64_t seed) : SequentialStringKey(str, seed) {} bool IsMatch(Object* string) override { @@ -250,7 +250,7 @@ class SeqOneByteSubStringKey : public StringTableKey { class TwoByteStringKey : public SequentialStringKey { public: - explicit TwoByteStringKey(Vector str, uint32_t seed) + explicit TwoByteStringKey(Vector str, uint64_t seed) : SequentialStringKey(str, seed) {} bool IsMatch(Object* string) override { @@ -263,7 +263,7 @@ class TwoByteStringKey : public SequentialStringKey { // Utf8StringKey carries a vector of chars as key. class Utf8StringKey : public StringTableKey { public: - explicit Utf8StringKey(Vector string, uint32_t seed) + explicit Utf8StringKey(Vector string, uint64_t seed) : StringTableKey(StringHasher::ComputeUtf8Hash(string, seed, &chars_)), string_(string) {} diff --git a/deps/v8/src/parsing/parse-info.h b/deps/v8/src/parsing/parse-info.h index 909f4c58e008bb..8aa0dfd700793b 100644 --- a/deps/v8/src/parsing/parse-info.h +++ b/deps/v8/src/parsing/parse-info.h @@ -148,8 +148,8 @@ class V8_EXPORT_PRIVATE ParseInfo { uintptr_t stack_limit() const { return stack_limit_; } void set_stack_limit(uintptr_t stack_limit) { stack_limit_ = stack_limit; } - uint32_t hash_seed() const { return hash_seed_; } - void set_hash_seed(uint32_t hash_seed) { hash_seed_ = hash_seed; } + uint64_t hash_seed() const { return hash_seed_; } + void set_hash_seed(uint64_t hash_seed) { hash_seed_ = hash_seed; } int compiler_hints() const { return compiler_hints_; } void set_compiler_hints(int compiler_hints) { @@ -266,7 +266,7 @@ class V8_EXPORT_PRIVATE ParseInfo { DeclarationScope* asm_function_scope_; UnicodeCache* unicode_cache_; uintptr_t stack_limit_; - uint32_t hash_seed_; + uint64_t hash_seed_; int compiler_hints_; int start_position_; int end_position_; diff --git a/deps/v8/src/profiler/allocation-tracker.cc b/deps/v8/src/profiler/allocation-tracker.cc index 8d8a3c7e1d4429..8d14031f640ccb 100644 --- a/deps/v8/src/profiler/allocation-tracker.cc +++ b/deps/v8/src/profiler/allocation-tracker.cc @@ -252,7 +252,7 @@ void AllocationTracker::AllocationEvent(Address addr, int size) { static uint32_t SnapshotObjectIdHash(SnapshotObjectId id) { - return ComputeIntegerHash(static_cast(id)); + return ComputeUnseededHash(static_cast(id)); } diff --git a/deps/v8/src/profiler/heap-profiler.cc b/deps/v8/src/profiler/heap-profiler.cc index f587adda497daf..61ffb84e8e0446 100644 --- a/deps/v8/src/profiler/heap-profiler.cc +++ b/deps/v8/src/profiler/heap-profiler.cc @@ -16,7 +16,7 @@ namespace internal { HeapProfiler::HeapProfiler(Heap* heap) : ids_(new HeapObjectsMap(heap)), - names_(new StringsStorage(heap->HashSeed())), + names_(new StringsStorage()), is_tracking_object_moves_(false), get_retainer_infos_callback_(nullptr) {} @@ -34,7 +34,7 @@ HeapProfiler::~HeapProfiler() { void HeapProfiler::DeleteAllSnapshots() { snapshots_.Iterate(DeleteHeapSnapshot); snapshots_.Clear(); - names_.reset(new StringsStorage(heap()->HashSeed())); + names_.reset(new StringsStorage()); } diff --git a/deps/v8/src/profiler/heap-snapshot-generator.cc b/deps/v8/src/profiler/heap-snapshot-generator.cc index 98e4600cd39893..0674a58cd6c511 100644 --- a/deps/v8/src/profiler/heap-snapshot-generator.cc +++ b/deps/v8/src/profiler/heap-snapshot-generator.cc @@ -526,7 +526,7 @@ SnapshotObjectId HeapObjectsMap::GenerateId(v8::RetainedObjectInfo* info) { heap_->HashSeed()); intptr_t element_count = info->GetElementCount(); if (element_count != -1) { - id ^= ComputeIntegerHash(static_cast(element_count)); + id ^= ComputeUnseededHash(static_cast(element_count)); } return id << 1; } diff --git a/deps/v8/src/profiler/heap-snapshot-generator.h b/deps/v8/src/profiler/heap-snapshot-generator.h index b40400aa7d45df..6baaca2cb4f8e9 100644 --- a/deps/v8/src/profiler/heap-snapshot-generator.h +++ b/deps/v8/src/profiler/heap-snapshot-generator.h @@ -288,7 +288,7 @@ class HeapEntriesMap { private: static uint32_t Hash(HeapThing thing) { - return ComputeIntegerHash( + return ComputeUnseededHash( static_cast(reinterpret_cast(thing))); } @@ -497,7 +497,7 @@ class NativeObjectsExplorer { void VisitSubtreeWrapper(Object** p, uint16_t class_id); static uint32_t InfoHash(v8::RetainedObjectInfo* info) { - return ComputeIntegerHash(static_cast(info->GetHash())); + return ComputeUnseededHash(static_cast(info->GetHash())); } static bool RetainedInfosMatch(void* key1, void* key2) { return key1 == key2 || diff --git a/deps/v8/src/profiler/profile-generator.cc b/deps/v8/src/profiler/profile-generator.cc index c01904b77801b1..23bf67a8d2ee5d 100644 --- a/deps/v8/src/profiler/profile-generator.cc +++ b/deps/v8/src/profiler/profile-generator.cc @@ -83,16 +83,16 @@ CodeEntry* CodeEntry::UnresolvedEntryCreateTrait::Create() { } uint32_t CodeEntry::GetHash() const { - uint32_t hash = ComputeIntegerHash(tag()); + uint32_t hash = ComputeUnseededHash(tag()); if (script_id_ != v8::UnboundScript::kNoScriptId) { - hash ^= ComputeIntegerHash(static_cast(script_id_)); - hash ^= ComputeIntegerHash(static_cast(position_)); + hash ^= ComputeUnseededHash(static_cast(script_id_)); + hash ^= ComputeUnseededHash(static_cast(position_)); } else { - hash ^= ComputeIntegerHash( + hash ^= ComputeUnseededHash( static_cast(reinterpret_cast(name_))); - hash ^= ComputeIntegerHash( + hash ^= ComputeUnseededHash( static_cast(reinterpret_cast(resource_name_))); - hash ^= ComputeIntegerHash(line_number_); + hash ^= ComputeUnseededHash(line_number_); } return hash; } @@ -533,7 +533,7 @@ void CodeMap::AddCode(Address addr, CodeEntry* entry, unsigned size) { ClearCodesInRange(addr, addr + size); unsigned index = AddCodeEntry(addr, entry); code_map_.emplace(addr, CodeEntryMapInfo{index, size}); - DCHECK(entry->instruction_start() == kNullAddress || + DCHECK(entry->instruction_start() == 0 || addr == entry->instruction_start()); } @@ -605,9 +605,7 @@ void CodeMap::Print() { } CpuProfilesCollection::CpuProfilesCollection(Isolate* isolate) - : resource_names_(isolate->heap()->HashSeed()), - profiler_(nullptr), - current_profiles_semaphore_(1) {} + : profiler_(nullptr), current_profiles_semaphore_(1) {} static void DeleteCpuProfile(CpuProfile** profile_ptr) { delete *profile_ptr; diff --git a/deps/v8/src/profiler/profile-generator.h b/deps/v8/src/profiler/profile-generator.h index f5b8ef39f26e67..740e9f27ea5077 100644 --- a/deps/v8/src/profiler/profile-generator.h +++ b/deps/v8/src/profiler/profile-generator.h @@ -241,7 +241,7 @@ class ProfileNode { }; struct Hasher { std::size_t operator()(CodeEntryAndLineNumber pair) const { - return pair.code_entry->GetHash() ^ ComputeIntegerHash(pair.line_number); + return pair.code_entry->GetHash() ^ ComputeUnseededHash(pair.line_number); } }; diff --git a/deps/v8/src/profiler/profiler-listener.cc b/deps/v8/src/profiler/profiler-listener.cc index 0feb16dc875346..cba4e018d4717a 100644 --- a/deps/v8/src/profiler/profiler-listener.cc +++ b/deps/v8/src/profiler/profiler-listener.cc @@ -16,8 +16,7 @@ namespace internal { ProfilerListener::ProfilerListener(Isolate* isolate, CodeEventObserver* observer) : isolate_(isolate), - observer_(observer), - function_and_resource_names_(isolate->heap()->HashSeed()) {} + observer_(observer) {} ProfilerListener::~ProfilerListener() = default; diff --git a/deps/v8/src/profiler/strings-storage.cc b/deps/v8/src/profiler/strings-storage.cc index cce5ad176560e5..bc79c48250cf42 100644 --- a/deps/v8/src/profiler/strings-storage.cc +++ b/deps/v8/src/profiler/strings-storage.cc @@ -17,8 +17,7 @@ bool StringsStorage::StringsMatch(void* key1, void* key2) { 0; } -StringsStorage::StringsStorage(uint32_t hash_seed) - : hash_seed_(hash_seed), names_(StringsMatch) {} +StringsStorage::StringsStorage() : names_(StringsMatch) {} StringsStorage::~StringsStorage() { for (base::HashMap::Entry* p = names_.Start(); p != NULL; @@ -116,7 +115,7 @@ const char* StringsStorage::GetFunctionName(const char* name) { } base::HashMap::Entry* StringsStorage::GetEntry(const char* str, int len) { - uint32_t hash = StringHasher::HashSequentialString(str, len, hash_seed_); + uint32_t hash = StringHasher::HashSequentialString(str, len, kZeroHashSeed); return names_.LookupOrInsert(const_cast(str), hash); } diff --git a/deps/v8/src/profiler/strings-storage.h b/deps/v8/src/profiler/strings-storage.h index 246a4033d5ee57..88d12c02b4fe3c 100644 --- a/deps/v8/src/profiler/strings-storage.h +++ b/deps/v8/src/profiler/strings-storage.h @@ -19,7 +19,7 @@ class Name; // forever, even if they disappear from JS heap or external storage. class StringsStorage { public: - explicit StringsStorage(uint32_t hash_seed); + StringsStorage(); ~StringsStorage(); const char* GetCopy(const char* src); @@ -39,7 +39,6 @@ class StringsStorage { const char* AddOrDisposeString(char* str, int len); base::CustomMatcherHashMap::Entry* GetEntry(const char* str, int len); - uint32_t hash_seed_; base::CustomMatcherHashMap names_; DISALLOW_COPY_AND_ASSIGN(StringsStorage); diff --git a/deps/v8/src/string-hasher-inl.h b/deps/v8/src/string-hasher-inl.h index 7d1f106e0227bd..62f50f050a33ee 100644 --- a/deps/v8/src/string-hasher-inl.h +++ b/deps/v8/src/string-hasher-inl.h @@ -12,9 +12,9 @@ namespace v8 { namespace internal { -StringHasher::StringHasher(int length, uint32_t seed) +StringHasher::StringHasher(int length, uint64_t seed) : length_(length), - raw_running_hash_(seed), + raw_running_hash_(static_cast(seed)), array_index_(0), is_array_index_(0 < length_ && length_ <= String::kMaxArrayIndexSize), is_first_char_(true) { @@ -113,16 +113,16 @@ inline void StringHasher::AddCharacters(const Char* chars, int length) { template uint32_t StringHasher::HashSequentialString(const schar* chars, int length, - uint32_t seed) { + uint64_t seed) { StringHasher hasher(length, seed); if (!hasher.has_trivial_hash()) hasher.AddCharacters(chars, length); return hasher.GetHashField(); } -IteratingStringHasher::IteratingStringHasher(int len, uint32_t seed) +IteratingStringHasher::IteratingStringHasher(int len, uint64_t seed) : StringHasher(len, seed) {} -uint32_t IteratingStringHasher::Hash(String* string, uint32_t seed) { +uint32_t IteratingStringHasher::Hash(String* string, uint64_t seed) { IteratingStringHasher hasher(string->length(), seed); // Nothing to do. if (hasher.has_trivial_hash()) return hasher.GetHashField(); diff --git a/deps/v8/src/string-hasher.h b/deps/v8/src/string-hasher.h index 867a480a4187c1..35cca259294d21 100644 --- a/deps/v8/src/string-hasher.h +++ b/deps/v8/src/string-hasher.h @@ -18,14 +18,14 @@ class Vector; class V8_EXPORT_PRIVATE StringHasher { public: - explicit inline StringHasher(int length, uint32_t seed); + explicit inline StringHasher(int length, uint64_t seed); template static inline uint32_t HashSequentialString(const schar* chars, int length, - uint32_t seed); + uint64_t seed); // Reads all the data, even for long strings and computes the utf16 length. - static uint32_t ComputeUtf8Hash(Vector chars, uint32_t seed, + static uint32_t ComputeUtf8Hash(Vector chars, uint64_t seed, int* utf16_length_out); // Calculated hash value for a string consisting of 1 to @@ -74,12 +74,12 @@ class V8_EXPORT_PRIVATE StringHasher { class IteratingStringHasher : public StringHasher { public: - static inline uint32_t Hash(String* string, uint32_t seed); + static inline uint32_t Hash(String* string, uint64_t seed); inline void VisitOneByteString(const uint8_t* chars, int length); inline void VisitTwoByteString(const uint16_t* chars, int length); private: - inline IteratingStringHasher(int len, uint32_t seed); + inline IteratingStringHasher(int len, uint64_t seed); void VisitConsString(ConsString* cons_string); DISALLOW_COPY_AND_ASSIGN(IteratingStringHasher); }; diff --git a/deps/v8/src/utils.h b/deps/v8/src/utils.h index 292ff535cb5700..20ef89d8ebcfcc 100644 --- a/deps/v8/src/utils.h +++ b/deps/v8/src/utils.h @@ -393,13 +393,12 @@ class BitSetComputer { // ---------------------------------------------------------------------------- // Hash function. -static const uint32_t kZeroHashSeed = 0; +static const uint64_t kZeroHashSeed = 0; // Thomas Wang, Integer Hash Functions. -// http://www.concentric.net/~Ttwang/tech/inthash.htm -inline uint32_t ComputeIntegerHash(uint32_t key, uint32_t seed) { +// http://www.concentric.net/~Ttwang/tech/inthash.htm` +inline uint32_t ComputeUnseededHash(uint32_t key) { uint32_t hash = key; - hash = hash ^ seed; hash = ~hash + (hash << 15); // hash = (hash << 15) - hash - 1; hash = hash ^ (hash >> 12); hash = hash + (hash << 2); @@ -409,10 +408,6 @@ inline uint32_t ComputeIntegerHash(uint32_t key, uint32_t seed) { return hash & 0x3fffffff; } -inline uint32_t ComputeIntegerHash(uint32_t key) { - return ComputeIntegerHash(key, kZeroHashSeed); -} - inline uint32_t ComputeLongHash(uint64_t key) { uint64_t hash = key; hash = ~hash + (hash << 18); // hash = (hash << 18) - hash - 1; @@ -421,16 +416,18 @@ inline uint32_t ComputeLongHash(uint64_t key) { hash = hash ^ (hash >> 11); hash = hash + (hash << 6); hash = hash ^ (hash >> 22); - return static_cast(hash); + return static_cast(hash & 0x3fffffff); } +inline uint32_t ComputeSeededHash(uint32_t key, uint64_t seed) { + return ComputeLongHash(static_cast(key) ^ seed); +} inline uint32_t ComputePointerHash(void* ptr) { - return ComputeIntegerHash( + return ComputeUnseededHash( static_cast(reinterpret_cast(ptr))); } - // ---------------------------------------------------------------------------- // Generated memcpy/memmove diff --git a/deps/v8/test/cctest/heap/test-heap.cc b/deps/v8/test/cctest/heap/test-heap.cc index 46927cdc887554..74a44f7371d2c6 100644 --- a/deps/v8/test/cctest/heap/test-heap.cc +++ b/deps/v8/test/cctest/heap/test-heap.cc @@ -6094,7 +6094,7 @@ UNINITIALIZED_TEST(ReinitializeStringHashSeed) { v8::Isolate* isolate = v8::Isolate::New(create_params); { v8::Isolate::Scope isolate_scope(isolate); - CHECK_EQ(1337 * i, + CHECK_EQ(static_cast(1337 * i), reinterpret_cast(isolate)->heap()->HashSeed()); v8::HandleScope handle_scope(isolate); v8::Local context = v8::Context::New(isolate); diff --git a/deps/v8/test/cctest/test-code-stub-assembler.cc b/deps/v8/test/cctest/test-code-stub-assembler.cc index e34b2322449756..af4116ae059985 100644 --- a/deps/v8/test/cctest/test-code-stub-assembler.cc +++ b/deps/v8/test/cctest/test-code-stub-assembler.cc @@ -262,15 +262,13 @@ TEST(JSFunction) { TEST(ComputeIntegerHash) { Isolate* isolate(CcTest::InitIsolateOnce()); - const int kNumParams = 2; + const int kNumParams = 1; CodeAssemblerTester asm_tester(isolate, kNumParams); CodeStubAssembler m(asm_tester.state()); - m.Return(m.SmiFromWord32(m.ComputeIntegerHash( - m.SmiUntag(m.Parameter(0)), m.SmiToWord32(m.Parameter(1))))); - FunctionTester ft(asm_tester.GenerateCode(), kNumParams); + m.Return(m.SmiFromWord32(m.ComputeSeededHash(m.SmiUntag(m.Parameter(0))))); - Handle hash_seed = isolate->factory()->hash_seed(); + FunctionTester ft(asm_tester.GenerateCode(), kNumParams); base::RandomNumberGenerator rand_gen(FLAG_random_seed); @@ -278,9 +276,9 @@ TEST(ComputeIntegerHash) { int k = rand_gen.NextInt(Smi::kMaxValue); Handle key(Smi::FromInt(k), isolate); - Handle result = ft.Call(key, hash_seed).ToHandleChecked(); + Handle result = ft.Call(key).ToHandleChecked(); - uint32_t hash = ComputeIntegerHash(k, hash_seed->value()); + uint32_t hash = ComputeSeededHash(k, isolate->heap()->HashSeed()); Smi* expected = Smi::FromInt(hash & Smi::kMaxValue); CHECK_EQ(expected, Smi::cast(*result)); } diff --git a/deps/v8/test/cctest/test-serialize.cc b/deps/v8/test/cctest/test-serialize.cc index 1f1d3e6010f1d6..5a3b9fc1b16c91 100644 --- a/deps/v8/test/cctest/test-serialize.cc +++ b/deps/v8/test/cctest/test-serialize.cc @@ -2465,7 +2465,8 @@ UNINITIALIZED_TEST(ReinitializeStringHashSeedNotRehashable) { v8::Isolate* isolate = v8::Isolate::New(create_params); { // Check that no rehashing has been performed. - CHECK_EQ(42, reinterpret_cast(isolate)->heap()->HashSeed()); + CHECK_EQ(static_cast(42), + reinterpret_cast(isolate)->heap()->HashSeed()); v8::Isolate::Scope isolate_scope(isolate); v8::HandleScope handle_scope(isolate); v8::Local context = v8::Context::New(isolate); From ce65d84537d9fc1bb9fcba1e45cce0e9364988bd Mon Sep 17 00:00:00 2001 From: Ben Newman Date: Sat, 4 Aug 2018 11:28:29 -0400 Subject: [PATCH 102/129] deps: backport a8f6869 from upstream V8 The upstream V8 commit a8f68691 was originally cherry-picked onto nodejs/node master as commit bb357524, then backported to v10.x-staging and released in Node.js v10.10.0 as 5e9ed6d9. This commit cherry-picks that commit back to the v8.x-staging branch. Additionally this commit supports optional boolean argument to DisableBreak constructor. This allows a stack-allocated DisableBreak object to re-enable breakpoints temporarily, rather than always disabling them. This functionality anticipates an upstream change that will be introduced in V8 6.7.176: https://github.com/v8/v8/commit/cc9736a1c0f409bf26d35653358a7b14212d2435 Original commit message: [debug] Fully implement Debug::ArchiveDebug and Debug::RestoreDebug. I have a project that embeds V8 and uses a single `Isolate` from multiple threads. The program runs just fine, but sometimes the inspector doesn't stop on the correct line after stepping over a statement that switches threads behind the scenes, even though the original thread is restored by the time the next statement is executed. After some digging, I discovered that the `Debug::ArchiveDebug` and `Debug::RestoreDebug` methods, which should be responsible for saving/restoring this `ThreadLocal` information when switching threads, currently don't do anything. This commit implements those methods using MemCopy, in the style of other Archive/Restore methods in the V8 codebase. Related: https://groups.google.com/forum/#!topic/v8-users/_Qf2rwljRk8 R=yangguo@chromium.org,jgruber@chromium.org CC=info@bnoordhuis.nl Bug: v8:7230 Change-Id: Id517c873eb81cd53f7216c7efd441b956cf7f943 Reviewed-on: https://chromium-review.googlesource.com/833260 Commit-Queue: Yang Guo Reviewed-by: Yang Guo Cr-Commit-Position: refs/heads/master@{#54902} PR-URL: https://github.com/nodejs/node/pull/22714 Refs: https://github.com/v8/v8/commit/a8f6869177685cfb9c199c454a86f4698c260515 Refs: https://github.com/nodejs/node/pull/22122 Refs: https://github.com/nodejs/node/commit/bb3575242cc87f59882bbcefa253353313f5606b Refs: https://github.com/nodejs/node/commit/5e9ed6d924db97462fb208e7ad8f32acce2a9bf3 Reviewed-By: James M Snell Reviewed-By: Myles Borins Reviewed-By: Yang Guo --- deps/v8/AUTHORS | 2 + deps/v8/include/v8-version.h | 2 +- deps/v8/src/debug/debug.cc | 29 +++++-- deps/v8/src/debug/debug.h | 5 +- deps/v8/src/v8threads.cc | 8 +- deps/v8/src/v8threads.h | 1 + deps/v8/test/cctest/test-debug.cc | 130 ++++++++++++++++++++++++++++++ 7 files changed, 166 insertions(+), 11 deletions(-) diff --git a/deps/v8/AUTHORS b/deps/v8/AUTHORS index 3bc767945cfdee..1a4af86d9ef399 100644 --- a/deps/v8/AUTHORS +++ b/deps/v8/AUTHORS @@ -30,6 +30,7 @@ Yandex LLC <*@yandex-team.ru> StrongLoop, Inc. <*@strongloop.com> Facebook, Inc. <*@fb.com> Facebook, Inc. <*@oculus.com> +Meteor Development Group <*@meteor.com> Aaron Bieber Abdulla Kamar @@ -44,6 +45,7 @@ Andrew Paprocki Andrei Kashcha Anna Henningsen Bangfu Tao +Ben Newman Ben Noordhuis Benjamin Tan Bert Belder diff --git a/deps/v8/include/v8-version.h b/deps/v8/include/v8-version.h index 1e44f2b4a266a9..43abf735d38d3e 100644 --- a/deps/v8/include/v8-version.h +++ b/deps/v8/include/v8-version.h @@ -11,7 +11,7 @@ #define V8_MAJOR_VERSION 6 #define V8_MINOR_VERSION 2 #define V8_BUILD_NUMBER 414 -#define V8_PATCH_LEVEL 69 +#define V8_PATCH_LEVEL 70 // Use 1 for candidates and 0 otherwise. // (Boolean macro values are not supported by all preprocessors.) diff --git a/deps/v8/src/debug/debug.cc b/deps/v8/src/debug/debug.cc index 966be62e63617f..00cdd5ad30defa 100644 --- a/deps/v8/src/debug/debug.cc +++ b/deps/v8/src/debug/debug.cc @@ -264,19 +264,36 @@ void Debug::ThreadInit() { char* Debug::ArchiveDebug(char* storage) { - // Simply reset state. Don't archive anything. - ThreadInit(); + MemCopy(storage, reinterpret_cast(&thread_local_), + ArchiveSpacePerThread()); return storage + ArchiveSpacePerThread(); } - char* Debug::RestoreDebug(char* storage) { - // Simply reset state. Don't restore anything. - ThreadInit(); + MemCopy(reinterpret_cast(&thread_local_), storage, + ArchiveSpacePerThread()); + + if (in_debug_scope()) { + // If this thread was in a DebugScope when we archived it, restore the + // previous debugging state now. Note that in_debug_scope() returns + // true when thread_local_.current_debug_scope_ (restored by MemCopy + // above) is non-null. + + // Clear any one-shot breakpoints that may have been set by the other + // thread, and reapply breakpoints for this thread. + HandleScope scope(isolate_); + ClearOneShot(); + + if (thread_local_.last_step_action_ != StepNone) { + // Reset the previous step action for this thread. + PrepareStep(thread_local_.last_step_action_); + } + } + return storage + ArchiveSpacePerThread(); } -int Debug::ArchiveSpacePerThread() { return 0; } +int Debug::ArchiveSpacePerThread() { return sizeof(ThreadLocal); } void Debug::Iterate(RootVisitor* v) { v->VisitRootPointer(Root::kDebug, &thread_local_.return_value_); diff --git a/deps/v8/src/debug/debug.h b/deps/v8/src/debug/debug.h index 9601fa7899fe20..f4d923b0626a2b 100644 --- a/deps/v8/src/debug/debug.h +++ b/deps/v8/src/debug/debug.h @@ -309,6 +309,7 @@ class Debug { static int ArchiveSpacePerThread(); void FreeThreadResources() { } void Iterate(RootVisitor* v); + void InitThread(const ExecutionAccess& lock) { ThreadInit(); } bool CheckExecutionState(int id) { return CheckExecutionState() && break_id() == id; @@ -690,9 +691,9 @@ class ReturnValueScope { // Stack allocated class for disabling break. class DisableBreak BASE_EMBEDDED { public: - explicit DisableBreak(Debug* debug) + explicit DisableBreak(Debug* debug, bool disable = true) : debug_(debug), previous_break_disabled_(debug->break_disabled_) { - debug_->break_disabled_ = true; + debug_->break_disabled_ = disable; } ~DisableBreak() { debug_->break_disabled_ = previous_break_disabled_; diff --git a/deps/v8/src/v8threads.cc b/deps/v8/src/v8threads.cc index 202323ec0de809..a2b02446e5aa9c 100644 --- a/deps/v8/src/v8threads.cc +++ b/deps/v8/src/v8threads.cc @@ -45,7 +45,7 @@ void Locker::Initialize(v8::Isolate* isolate) { } else { internal::ExecutionAccess access(isolate_); isolate_->stack_guard()->ClearThread(access); - isolate_->stack_guard()->InitThread(access); + isolate_->thread_manager()->InitThread(access); } } DCHECK(isolate_->thread_manager()->IsLockedByCurrentThread()); @@ -95,6 +95,10 @@ Unlocker::~Unlocker() { namespace internal { +void ThreadManager::InitThread(const ExecutionAccess& lock) { + isolate_->stack_guard()->InitThread(lock); + isolate_->debug()->InitThread(lock); +} bool ThreadManager::RestoreThread() { DCHECK(IsLockedByCurrentThread()); @@ -127,7 +131,7 @@ bool ThreadManager::RestoreThread() { isolate_->FindPerThreadDataForThisThread(); if (per_thread == NULL || per_thread->thread_state() == NULL) { // This is a new thread. - isolate_->stack_guard()->InitThread(access); + InitThread(access); return false; } ThreadState* state = per_thread->thread_state(); diff --git a/deps/v8/src/v8threads.h b/deps/v8/src/v8threads.h index 8fc6f0c62f6ad6..1f39f473cf6914 100644 --- a/deps/v8/src/v8threads.h +++ b/deps/v8/src/v8threads.h @@ -67,6 +67,7 @@ class ThreadManager { void Lock(); void Unlock(); + void InitThread(const ExecutionAccess&); void ArchiveThread(); bool RestoreThread(); void FreeThreadResources(); diff --git a/deps/v8/test/cctest/test-debug.cc b/deps/v8/test/cctest/test-debug.cc index 8c3818e8e9ec35..682d9517f1a26e 100644 --- a/deps/v8/test/cctest/test-debug.cc +++ b/deps/v8/test/cctest/test-debug.cc @@ -6207,6 +6207,136 @@ TEST(DebugBreakOffThreadTerminate) { } +class ArchiveRestoreThread : public v8::base::Thread, + public v8::debug::DebugDelegate { + public: + ArchiveRestoreThread(v8::Isolate* isolate, int spawn_count) + : Thread(Options("ArchiveRestoreThread")), + isolate_(isolate), + debug_(reinterpret_cast(isolate_)->debug()), + spawn_count_(spawn_count), + break_count_(0) {} + + virtual void Run() { + v8::Locker locker(isolate_); + isolate_->Enter(); + + v8::HandleScope scope(isolate_); + v8::Local context = v8::Context::New(isolate_); + v8::Context::Scope context_scope(context); + + v8::Local test = CompileFunction(isolate_, + "function test(n) {\n" + " debugger;\n" + " return n + 1;\n" + "}\n", + "test"); + + debug_->SetDebugDelegate(this, false); + v8::internal::DisableBreak enable_break(debug_, false); + + v8::Local args[1] = {v8::Integer::New(isolate_, spawn_count_)}; + + int result = test->Call(context, context->Global(), 1, args) + .ToLocalChecked() + ->Int32Value(context) + .FromJust(); + + // Verify that test(spawn_count_) returned spawn_count_ + 1. + CHECK_EQ(spawn_count_ + 1, result); + + isolate_->Exit(); + } + + void BreakProgramRequested(v8::Local context, + v8::Local exec_state, + v8::Local break_points_hit, + const std::vector&) { + auto stack_traces = v8::debug::StackTraceIterator::Create(isolate_); + if (!stack_traces->Done()) { + v8::debug::Location location = stack_traces->GetSourceLocation(); + + i::PrintF("ArchiveRestoreThread #%d hit breakpoint at line %d\n", + spawn_count_, location.GetLineNumber()); + + switch (location.GetLineNumber()) { + case 1: // debugger; + CHECK_EQ(break_count_, 0); + + // Attempt to stop on the next line after the first debugger + // statement. If debug->{Archive,Restore}Debug() improperly reset + // thread-local debug information, the debugger will fail to stop + // before the test function returns. + debug_->PrepareStep(StepNext); + + // Spawning threads while handling the current breakpoint verifies + // that the parent thread correctly archived and restored the + // state necessary to stop on the next line. If not, then control + // will simply continue past the `return n + 1` statement. + MaybeSpawnChildThread(); + + break; + + case 2: // return n + 1; + CHECK_EQ(break_count_, 1); + break; + + default: + CHECK(false); + } + } + + ++break_count_; + } + + void MaybeSpawnChildThread() { + if (spawn_count_ > 1) { + v8::Unlocker unlocker(isolate_); + + // Spawn a thread that spawns a thread that spawns a thread (and so + // on) so that the ThreadManager is forced to archive and restore + // the current thread. + ArchiveRestoreThread child(isolate_, spawn_count_ - 1); + child.Start(); + child.Join(); + + // The child thread sets itself as the debug delegate, so we need to + // usurp it after the child finishes, or else future breakpoints + // will be delegated to a destroyed ArchiveRestoreThread object. + debug_->SetDebugDelegate(this, false); + + // This is the most important check in this test, since + // child.GetBreakCount() will return 1 if the debugger fails to stop + // on the `return n + 1` line after the grandchild thread returns. + CHECK_EQ(child.GetBreakCount(), 2); + } + } + + int GetBreakCount() { return break_count_; } + + private: + v8::Isolate* isolate_; + v8::internal::Debug* debug_; + const int spawn_count_; + int break_count_; +}; + +TEST(DebugArchiveRestore) { + v8::Isolate::CreateParams create_params; + create_params.array_buffer_allocator = CcTest::array_buffer_allocator(); + v8::Isolate* isolate = v8::Isolate::New(create_params); + + ArchiveRestoreThread thread(isolate, 5); + // Instead of calling thread.Start() and thread.Join() here, we call + // thread.Run() directly, to make sure we exercise archive/restore + // logic on the *current* thread as well as other threads. + thread.Run(); + CHECK_EQ(thread.GetBreakCount(), 2); + + isolate->Dispose(); +} + + static void DebugEventExpectNoException( const v8::Debug::EventDetails& event_details) { v8::DebugEvent event = event_details.GetEvent(); From e3bddeec18f9b36c1314cdd5fbfa59119ba6d619 Mon Sep 17 00:00:00 2001 From: Anatoli Papirovski Date: Sat, 14 Apr 2018 14:01:13 +0200 Subject: [PATCH 103/129] http: fix undefined error in parser event The current check for socket.server[kIncomingMessage] does not account for the possibility of a socket.server that doesn't have that property defined. Fix it. Backport-PR-URL: https://github.com/nodejs/node/pull/22880 PR-URL: https://github.com/nodejs/node/pull/20029 Fixes: https://github.com/nodejs/node/issues/19231 Reviewed-By: Ben Noordhuis Reviewed-By: Luigi Pinca Reviewed-By: Trivikram Kamat Reviewed-By: James M Snell Reviewed-By: Ruben Bridgewater Reviewed-By: Khaidi Chu --- lib/_http_common.js | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/lib/_http_common.js b/lib/_http_common.js index 6fc283e607633e..cacc04eaf7364e 100644 --- a/lib/_http_common.js +++ b/lib/_http_common.js @@ -62,7 +62,8 @@ function parserOnHeaders(headers, url) { function parserOnHeadersComplete(versionMajor, versionMinor, headers, method, url, statusCode, statusMessage, upgrade, shouldKeepAlive) { - var parser = this; + const parser = this; + const { socket } = parser; if (!headers) { headers = parser._headers; @@ -75,10 +76,11 @@ function parserOnHeadersComplete(versionMajor, versionMinor, headers, method, } // Parser is also used by http client - var ParserIncomingMessage = parser.socket && parser.socket.server ? - parser.socket.server[kIncomingMessage] : IncomingMessage; + const ParserIncomingMessage = (socket && socket.server && + socket.server[kIncomingMessage]) || + IncomingMessage; - parser.incoming = new ParserIncomingMessage(parser.socket); + parser.incoming = new ParserIncomingMessage(socket); parser.incoming.httpVersionMajor = versionMajor; parser.incoming.httpVersionMinor = versionMinor; parser.incoming.httpVersion = `${versionMajor}.${versionMinor}`; From c6d94f8fa5781b3741b6cf347c429e7037864b94 Mon Sep 17 00:00:00 2001 From: Ruben Bridgewater Date: Mon, 13 Nov 2017 18:15:24 -0200 Subject: [PATCH 104/129] assert: add strict functionality export Requireing the strict version will allow to use `assert.equal`, `assert.deepEqual` and there negated counterparts to be used with strict comparison instead of using e.g. `assert.strictEqual`. The API is identical to the regular assert export and only differs in the way that all functions use strict compairson. Backport-PR-URL: https://github.com/nodejs/node/pull/23223 PR-URL: https://github.com/nodejs/node/pull/17002 Reviewed-By: Refael Ackermann Reviewed-By: James M Snell Reviewed-By: Vse Mozhet Byt Reviewed-By: Anna Henningsen --- doc/api/assert.md | 123 +++++++++++++++++++++++++++++++---- doc/api/deprecations.md | 9 +++ lib/assert.js | 12 ++++ test/parallel/test-assert.js | 19 ++++++ 4 files changed, 150 insertions(+), 13 deletions(-) diff --git a/doc/api/assert.md b/doc/api/assert.md index d2b541111586be..d337a09b328921 100644 --- a/doc/api/assert.md +++ b/doc/api/assert.md @@ -7,6 +7,57 @@ The `assert` module provides a simple set of assertion tests that can be used to test invariants. +A `strict` and a `legacy` mode exist, while it is recommended to only use +[`strict mode`][]. + +For more information about the used equality comparisons see +[MDN's guide on equality comparisons and sameness][mdn-equality-guide]. + +## Strict mode + + +When using the `strict mode`, any `assert` function will use the equality used in +the strict function mode. So [`assert.deepEqual()`][] will, for example, work the +same as [`assert.deepStrictEqual()`][]. + +It can be accessed using: + +```js +const assert = require('assert').strict; +``` + +## Legacy mode + +> Stability: 0 - Deprecated: Use strict mode instead. + +When accessing `assert` directly instead of using the `strict` property, the +[Abstract Equality Comparison][] will be used for any function without a +"strict" in its name (e.g. [`assert.deepEqual()`][]). + +It can be accessed using: + +```js +const assert = require('assert'); +``` + +It is recommended to use the [`strict mode`][] instead as the +[Abstract Equality Comparison][] can often have surprising results. Especially +in case of [`assert.deepEqual()`][] as the used comparison rules there are very +lax. + +E.g. + +```js +// WARNING: This does not throw an AssertionError! +assert.deepEqual(/a/gi, new Date()); +``` + ## assert(value[, message]) ```js -// THIS IS A MISTAKE! DO NOT DO THIS! -assert.throws(myFunction, 'missing foo', 'did not throw with expected message'); - -// Do this instead. -assert.throws(myFunction, /missing foo/, 'did not throw with expected message'); +function throwingFirst() { + throw new Error('First'); +} +function throwingSecond() { + throw new Error('Second'); +} +function notThrowing() {} + +// The second argument is a string and the input function threw an Error. +// In that case both cases do not throw as neither is going to try to +// match for the error message thrown by the input function! +assert.throws(throwingFirst, 'Second'); +assert.throws(throwingSecond, 'Second'); + +// The string is only used (as message) in case the function does not throw: +assert.throws(notThrowing, 'Second'); +// AssertionError [ERR_ASSERTION]: Missing expected exception: Second + +// If it was intended to match for the error message do this instead: +assert.throws(throwingSecond, /Second$/); +// Does not throw because the error messages match. +assert.throws(throwingFirst, /Second$/); +// Throws a error: +// Error: First +// at throwingFirst (repl:2:9) ``` +Due to the confusing notation, it is recommended not to use a string as the +second argument. This might lead to difficult-to-spot errors. + ## Caveats For the following cases, consider using ES2015 [`Object.is()`][], diff --git a/lib/assert.js b/lib/assert.js index 79c61fa7559a16..b0513c9795bed5 100644 --- a/lib/assert.js +++ b/lib/assert.js @@ -675,7 +675,11 @@ function expectedException(actual, expected) { return expected.call({}, actual) === true; } -function tryBlock(block) { +function getActual(block) { + if (typeof block !== 'function') { + throw new errors.TypeError('ERR_INVALID_ARG_TYPE', 'block', 'Function', + block); + } try { block(); } catch (e) { @@ -683,60 +687,61 @@ function tryBlock(block) { } } -function innerThrows(shouldThrow, block, expected, message) { - var details = ''; +// Expected to throw an error. +assert.throws = function throws(block, error, message) { + const actual = getActual(block); - if (typeof block !== 'function') { - throw new errors.TypeError('ERR_INVALID_ARG_TYPE', 'block', 'function', - block); - } + if (typeof error === 'string') { + if (arguments.length === 3) + throw new errors.TypeError('ERR_INVALID_ARG_TYPE', + 'error', + ['Function', 'RegExp'], + error); - if (typeof expected === 'string') { - message = expected; - expected = null; + message = error; + error = null; } - const actual = tryBlock(block); - - if (shouldThrow === true) { - if (actual === undefined) { - if (expected && expected.name) { - details += ` (${expected.name})`; - } - details += message ? `: ${message}` : '.'; - innerFail({ - actual, - expected, - operator: 'throws', - message: `Missing expected exception${details}`, - stackStartFn: assert.throws - }); - } - if (expected && expectedException(actual, expected) === false) { - throw actual; - } - } else if (actual !== undefined) { - if (!expected || expectedException(actual, expected)) { - details = message ? `: ${message}` : '.'; - innerFail({ - actual, - expected, - operator: 'doesNotThrow', - message: `Got unwanted exception${details}`, - stackStartFn: assert.doesNotThrow - }); + if (actual === undefined) { + let details = ''; + if (error && error.name) { + details += ` (${error.name})`; } + details += message ? `: ${message}` : '.'; + innerFail({ + actual, + expected: error, + operator: 'throws', + message: `Missing expected exception${details}`, + stackStartFn: throws + }); + } + if (error && expectedException(actual, error) === false) { throw actual; } -} - -// Expected to throw an error. -assert.throws = function throws(block, error, message) { - innerThrows(true, block, error, message); }; assert.doesNotThrow = function doesNotThrow(block, error, message) { - innerThrows(false, block, error, message); + const actual = getActual(block); + if (actual === undefined) + return; + + if (typeof error === 'string') { + message = error; + error = null; + } + + if (!error || expectedException(actual, error)) { + const details = message ? `: ${message}` : '.'; + innerFail({ + actual, + expected: error, + operator: 'doesNotThrow', + message: `Got unwanted exception${details}\n${actual.message}`, + stackStartFn: doesNotThrow + }); + } + throw actual; }; assert.ifError = function ifError(err) { if (err) throw err; }; diff --git a/test/parallel/test-assert.js b/test/parallel/test-assert.js index a8ba1bf5d1fc6d..72606a911e5a15 100644 --- a/test/parallel/test-assert.js +++ b/test/parallel/test-assert.js @@ -640,7 +640,7 @@ try { common.expectsError({ code: 'ERR_INVALID_ARG_TYPE', type: TypeError, - message: 'The "block" argument must be of type function. Received ' + + message: 'The "block" argument must be of type Function. Received ' + `type ${typeName(block)}` })(e); } @@ -731,3 +731,14 @@ common.expectsError( message: 'null == true' } ); + +common.expectsError( + // eslint-disable-next-line no-restricted-syntax + () => assert.throws(() => {}, 'Error message', 'message'), + { + code: 'ERR_INVALID_ARG_TYPE', + type: TypeError, + message: 'The "error" argument must be one of type Function or RegExp. ' + + 'Received type string' + } +); From f2af930ebbd1ab4e4b8b929701d110c0c1983832 Mon Sep 17 00:00:00 2001 From: Ruben Bridgewater Date: Sat, 9 Dec 2017 22:54:44 -0200 Subject: [PATCH 109/129] assert: .throws accept objects From now on it is possible to use a validation object in throws instead of the other possibilites. Backport-PR-URL: https://github.com/nodejs/node/pull/23223 PR-URL: https://github.com/nodejs/node/pull/17584 Refs: https://github.com/nodejs/node/pull/17557 Reviewed-By: Benjamin Gruenbaum Reviewed-By: Luigi Pinca Reviewed-By: James M Snell Reviewed-By: Matteo Collina Reviewed-By: Ron Korving Reviewed-By: Yuta Hiroto --- doc/api/assert.md | 26 +++++++++-- lib/assert.js | 43 ++++++++++++++++-- test/parallel/test-assert.js | 86 +++++++++++++++++++++++++++++++----- 3 files changed, 138 insertions(+), 17 deletions(-) diff --git a/doc/api/assert.md b/doc/api/assert.md index b1c7168e94c7d7..dbd32096b5c096 100644 --- a/doc/api/assert.md +++ b/doc/api/assert.md @@ -635,18 +635,21 @@ If the values are not strictly equal, an `AssertionError` is thrown with a * `block` {Function} -* `error` {RegExp|Function} +* `error` {RegExp|Function|object} * `message` {any} Expects the function `block` to throw an error. -If specified, `error` can be a constructor, [`RegExp`][], or validation -function. +If specified, `error` can be a constructor, [`RegExp`][], a validation +function, or an object where each property will be tested for. If specified, `message` will be the message provided by the `AssertionError` if the block fails to throw. @@ -689,6 +692,23 @@ assert.throws( ); ``` +Custom error object / error instance: + +```js +assert.throws( + () => { + const err = new TypeError('Wrong value'); + err.code = 404; + throw err; + }, + { + name: 'TypeError', + message: 'Wrong value' + // Note that only properties on the error object will be tested! + } +); +``` + Note that `error` can not be a string. If a string is provided as the second argument, then `error` is assumed to be omitted and the string will be used for `message` instead. This can lead to easy-to-miss mistakes. Please read the diff --git a/lib/assert.js b/lib/assert.js index b0513c9795bed5..4fdb48477053b4 100644 --- a/lib/assert.js +++ b/lib/assert.js @@ -25,6 +25,7 @@ const { isSet, isMap, isDate, isRegExp } = process.binding('util'); const { objectToString } = require('internal/util'); const { isArrayBufferView } = require('internal/util/types'); const errors = require('internal/errors'); +const { inspect } = require('util'); // The assert module provides functions that throw // AssertionError's when particular conditions are not met. The @@ -660,10 +661,44 @@ assert.notStrictEqual = function notStrictEqual(actual, expected, message) { } }; -function expectedException(actual, expected) { +function createMsg(msg, key, actual, expected) { + if (msg) + return msg; + return `${key}: expected ${inspect(expected[key])}, ` + + `not ${inspect(actual[key])}`; +} + +function expectedException(actual, expected, msg) { if (typeof expected !== 'function') { - // Should be a RegExp, if not fail hard - return expected.test(actual); + if (expected instanceof RegExp) + return expected.test(actual); + // assert.doesNotThrow does not accept objects. + if (arguments.length === 2) { + throw new errors.TypeError('ERR_INVALID_ARG_TYPE', 'expected', + ['Function', 'RegExp'], expected); + } + // The name and message could be non enumerable. Therefore test them + // explicitly. + if ('name' in expected) { + assert.strictEqual( + actual.name, + expected.name, + createMsg(msg, 'name', actual, expected)); + } + if ('message' in expected) { + assert.strictEqual( + actual.message, + expected.message, + createMsg(msg, 'message', actual, expected)); + } + const keys = Object.keys(expected); + for (const key of keys) { + assert.deepStrictEqual( + actual[key], + expected[key], + createMsg(msg, key, actual, expected)); + } + return true; } // Guard instanceof against arrow functions as they don't have a prototype. if (expected.prototype !== undefined && actual instanceof expected) { @@ -716,7 +751,7 @@ assert.throws = function throws(block, error, message) { stackStartFn: throws }); } - if (error && expectedException(actual, error) === false) { + if (error && expectedException(actual, error, message) === false) { throw actual; } }; diff --git a/test/parallel/test-assert.js b/test/parallel/test-assert.js index 72606a911e5a15..ca3cca1fa0d479 100644 --- a/test/parallel/test-assert.js +++ b/test/parallel/test-assert.js @@ -36,16 +36,6 @@ assert.ok(a.AssertionError.prototype instanceof Error, assert.throws(makeBlock(a, false), a.AssertionError, 'ok(false)'); -// Using a object as second arg results in a failure -assert.throws( - () => { assert.throws(() => { throw new Error(); }, { foo: 'bar' }); }, - common.expectsError({ - type: TypeError, - message: 'expected.test is not a function' - }) -); - - assert.doesNotThrow(makeBlock(a, true), a.AssertionError, 'ok(true)'); assert.doesNotThrow(makeBlock(a, 'test', 'ok(\'test\')')); @@ -742,3 +732,79 @@ common.expectsError( 'Received type string' } ); + +{ + const errFn = () => { + const err = new TypeError('Wrong value'); + err.code = 404; + throw err; + }; + const errObj = { + name: 'TypeError', + message: 'Wrong value' + }; + assert.throws(errFn, errObj); + + errObj.code = 404; + assert.throws(errFn, errObj); + + errObj.code = '404'; + common.expectsError( + // eslint-disable-next-line no-restricted-syntax + () => assert.throws(errFn, errObj), + { + code: 'ERR_ASSERTION', + type: assert.AssertionError, + message: 'code: expected \'404\', not 404' + } + ); + + errObj.code = 404; + errObj.foo = 'bar'; + common.expectsError( + // eslint-disable-next-line no-restricted-syntax + () => assert.throws(errFn, errObj), + { + code: 'ERR_ASSERTION', + type: assert.AssertionError, + message: 'foo: expected \'bar\', not undefined' + } + ); + + common.expectsError( + () => assert.throws(() => { throw new Error(); }, { foo: 'bar' }, 'foobar'), + { + type: assert.AssertionError, + code: 'ERR_ASSERTION', + message: 'foobar' + } + ); + + common.expectsError( + () => assert.doesNotThrow(() => { throw new Error(); }, { foo: 'bar' }), + { + type: TypeError, + code: 'ERR_INVALID_ARG_TYPE', + message: 'The "expected" argument must be one of type Function or ' + + 'RegExp. Received type object' + } + ); + + assert.throws(() => { throw new Error('e'); }, new Error('e')); + common.expectsError( + () => assert.throws(() => { throw new TypeError('e'); }, new Error('e')), + { + type: assert.AssertionError, + code: 'ERR_ASSERTION', + message: "name: expected 'Error', not 'TypeError'" + } + ); + common.expectsError( + () => assert.throws(() => { throw new Error('foo'); }, new Error('')), + { + type: assert.AssertionError, + code: 'ERR_ASSERTION', + message: "message: expected '', not 'foo'" + } + ); +} From 562787efb2aa880a628bfb486a9784535347ccc2 Mon Sep 17 00:00:00 2001 From: Ruben Bridgewater Date: Thu, 28 Dec 2017 18:58:08 +0100 Subject: [PATCH 110/129] assert: fix strict regression MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Using `assert()` or `assert.ok()` resulted in a error since a refactoring. Refs: https://github.com/nodejs/node/pull/17582 Backport-PR-URL: https://github.com/nodejs/node/pull/23223 PR-URL: https://github.com/nodejs/node/pull/17903 Refs: https://github.com/nodejs/node/pull/17582 Reviewed-By: Anatoli Papirovski Reviewed-By: Tobias Nießen Reviewed-By: Colin Ihrig Reviewed-By: James M Snell --- lib/assert.js | 10 +++++++++- test/parallel/test-assert.js | 8 ++++++++ 2 files changed, 17 insertions(+), 1 deletion(-) diff --git a/lib/assert.js b/lib/assert.js index 4fdb48477053b4..75fe173094aedb 100644 --- a/lib/assert.js +++ b/lib/assert.js @@ -783,7 +783,15 @@ assert.ifError = function ifError(err) { if (err) throw err; }; // Expose a strict only variant of assert function strict(value, message) { - if (!value) innerFail(value, true, message, '==', strict); + if (!value) { + innerFail({ + actual: value, + expected: true, + message, + operator: '==', + stackStartFn: strict + }); + } } assert.strict = Object.assign(strict, assert, { equal: assert.strictEqual, diff --git a/test/parallel/test-assert.js b/test/parallel/test-assert.js index ca3cca1fa0d479..08b81b7c37f7fa 100644 --- a/test/parallel/test-assert.js +++ b/test/parallel/test-assert.js @@ -711,6 +711,14 @@ common.expectsError( assert.equal(Object.keys(assert).length, Object.keys(a).length); /* eslint-enable no-restricted-properties */ assert(7); + common.expectsError( + () => assert(), + { + code: 'ERR_ASSERTION', + type: assert.AssertionError, + message: 'undefined == true' + } + ); } common.expectsError( From 18071db274cffc4364440ef8dfc6cc2e5bfba1bf Mon Sep 17 00:00:00 2001 From: Ruben Bridgewater Date: Tue, 6 Feb 2018 11:08:23 +0100 Subject: [PATCH 111/129] assert: fix throws trace The current stack trace thrown in case `assert.throws(fn, object)` is used did not filter the stack trace. This fixes it. Backport-PR-URL: https://github.com/nodejs/node/pull/23223 PR-URL: https://github.com/nodejs/node/pull/18595 Reviewed-By: James M Snell Reviewed-By: Anatoli Papirovski --- lib/assert.js | 34 ++++++++++++---------------- test/message/assert_throws_stack.js | 6 +++++ test/message/assert_throws_stack.out | 14 ++++++++++++ 3 files changed, 35 insertions(+), 19 deletions(-) create mode 100644 test/message/assert_throws_stack.js create mode 100644 test/message/assert_throws_stack.out diff --git a/lib/assert.js b/lib/assert.js index 75fe173094aedb..ad568547b45c6a 100644 --- a/lib/assert.js +++ b/lib/assert.js @@ -661,11 +661,17 @@ assert.notStrictEqual = function notStrictEqual(actual, expected, message) { } }; -function createMsg(msg, key, actual, expected) { - if (msg) - return msg; - return `${key}: expected ${inspect(expected[key])}, ` + - `not ${inspect(actual[key])}`; +function compareExceptionKey(actual, expected, key, msg) { + if (!innerDeepEqual(actual[key], expected[key], true)) { + innerFail({ + actual: actual[key], + expected: expected[key], + message: msg || `${key}: expected ${inspect(expected[key])}, ` + + `not ${inspect(actual[key])}`, + operator: 'throws', + stackStartFn: assert.throws + }); + } } function expectedException(actual, expected, msg) { @@ -680,23 +686,13 @@ function expectedException(actual, expected, msg) { // The name and message could be non enumerable. Therefore test them // explicitly. if ('name' in expected) { - assert.strictEqual( - actual.name, - expected.name, - createMsg(msg, 'name', actual, expected)); + compareExceptionKey(actual, expected, 'name', msg); } if ('message' in expected) { - assert.strictEqual( - actual.message, - expected.message, - createMsg(msg, 'message', actual, expected)); + compareExceptionKey(actual, expected, 'message', msg); } - const keys = Object.keys(expected); - for (const key of keys) { - assert.deepStrictEqual( - actual[key], - expected[key], - createMsg(msg, key, actual, expected)); + for (const key of Object.keys(expected)) { + compareExceptionKey(actual, expected, key, msg); } return true; } diff --git a/test/message/assert_throws_stack.js b/test/message/assert_throws_stack.js new file mode 100644 index 00000000000000..36bc5734cae37f --- /dev/null +++ b/test/message/assert_throws_stack.js @@ -0,0 +1,6 @@ +'use strict'; + +require('../common'); +const assert = require('assert').strict; + +assert.throws(() => { throw new Error('foo'); }, { bar: true }); diff --git a/test/message/assert_throws_stack.out b/test/message/assert_throws_stack.out new file mode 100644 index 00000000000000..04e62b98139eed --- /dev/null +++ b/test/message/assert_throws_stack.out @@ -0,0 +1,14 @@ +assert.js:* + throw new errors.AssertionError(obj); + ^ + +AssertionError [ERR_ASSERTION]: bar: expected true, not undefined + at Object. (*assert_throws_stack.js:*:*) + at * + at * + at * + at * + at * + at * + at * + at * From 7f34c277accdfffe0b87b24f5e2fda889b1ac924 Mon Sep 17 00:00:00 2001 From: Anna Henningsen Date: Thu, 22 Feb 2018 13:48:06 +0100 Subject: [PATCH 112/129] http2: simplify timeout tracking MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit There’s no need to reset the chunk counter for every write. Backport-PR-URL: https://github.com/nodejs/node/pull/23774 PR-URL: https://github.com/nodejs/node/pull/19206 Reviewed-By: James M Snell Reviewed-By: Daniel Bevenius Reviewed-By: Anatoli Papirovski Reviewed-By: Colin Ihrig --- src/node_http2.cc | 7 ------- src/node_http2.h | 2 -- 2 files changed, 9 deletions(-) diff --git a/src/node_http2.cc b/src/node_http2.cc index 49238dcae2d966..f56489bc5a034e 100644 --- a/src/node_http2.cc +++ b/src/node_http2.cc @@ -1798,10 +1798,6 @@ inline Http2Stream* Http2Session::SubmitRequest( return stream; } -inline void Http2Session::SetChunksSinceLastWrite(size_t n) { - chunks_sent_since_last_write_ = n; -} - // Allocates the data buffer used to pass outbound data to the i/o stream. WriteWrap* Http2Session::AllocateSend() { HandleScope scope(env()->isolate()); @@ -2257,7 +2253,6 @@ inline int Http2Stream::DoWrite(WriteWrap* req_wrap, CHECK(!this->IsDestroyed()); CHECK_EQ(send_handle, nullptr); Http2Scope h2scope(this); - session_->SetChunksSinceLastWrite(); req_wrap->Dispatched(); if (!IsWritable()) { req_wrap->Done(UV_EOF); @@ -2751,8 +2746,6 @@ void Http2Stream::RespondFD(const FunctionCallbackInfo& args) { int64_t length = args[3]->IntegerValue(context).ToChecked(); int options = args[4]->IntegerValue(context).ToChecked(); - stream->session()->SetChunksSinceLastWrite(); - Headers list(isolate, context, headers); args.GetReturnValue().Set(stream->SubmitFile(fd, *list, list.length(), offset, length, options)); diff --git a/src/node_http2.h b/src/node_http2.h index ca0dd55f97f674..a4e37562f3f313 100644 --- a/src/node_http2.h +++ b/src/node_http2.h @@ -815,8 +815,6 @@ class Http2Session : public AsyncWrap { // Write data to the session inline ssize_t Write(const uv_buf_t* bufs, size_t nbufs); - inline void SetChunksSinceLastWrite(size_t n = 0); - size_t self_size() const override { return sizeof(*this); } char* stream_alloc() { From 3babc5bb533d94e6355d062233fbe05744603115 Mon Sep 17 00:00:00 2001 From: feugy Date: Fri, 12 Jan 2018 00:16:41 +0100 Subject: [PATCH 113/129] assert: add rejects() and doesNotReject() Implement asynchronous equivalent for assert.throws() and assert.doesNotThrow(). Backport-PR-URL: https://github.com/nodejs/node/pull/24019 PR-URL: https://github.com/nodejs/node/pull/18023 Reviewed-By: James M Snell Reviewed-By: Shingo Inoue Reviewed-By: Joyee Cheung Reviewed-By: Ruben Bridgewater --- doc/api/assert.md | 80 ++++++++++++++++++++++++++++++ lib/assert.js | 65 ++++++++++++++++++------ test/parallel/test-assert-async.js | 66 ++++++++++++++++++++++++ test/parallel/test-assert.js | 11 ++++ 4 files changed, 207 insertions(+), 15 deletions(-) create mode 100644 test/parallel/test-assert-async.js diff --git a/doc/api/assert.md b/doc/api/assert.md index dbd32096b5c096..6cc3cd561eb8ef 100644 --- a/doc/api/assert.md +++ b/doc/api/assert.md @@ -242,6 +242,43 @@ If the values are not equal, an `AssertionError` is thrown with a `message` property set equal to the value of the `message` parameter. If the `message` parameter is undefined, a default error message is assigned. +## assert.doesNotReject(block[, error][, message]) + +* `block` {Function} +* `error` {RegExp|Function} +* `message` {any} + +Awaits for the promise returned by function `block` to complete and not be +rejected. See [`assert.rejects()`][] for more details. + +When `assert.doesNotReject()` is called, it will immediately call the `block` +function, and awaits for completion. + +Besides the async nature to await the completion behaves identical to +[`assert.doesNotThrow()`][]. + +```js +(async () => { + await assert.doesNotReject( + async () => { + throw new TypeError('Wrong value'); + }, + SyntaxError + ); +})(); +``` + +```js +assert.doesNotReject( + () => Promise.reject(new TypeError('Wrong value')), + SyntaxError +).then(() => { + // ... +}); +``` + ## assert.doesNotThrow(block[, error][, message]) +* `block` {Function} +* `error` {RegExp|Function|Object} +* `message` {any} + +Awaits for promise returned by function `block` to be rejected. + +When `assert.rejects()` is called, it will immediately call the `block` +function, and awaits for completion. + +Besides the async nature to await the completion behaves identical to +[`assert.throws()`][]. + +If specified, `error` can be a constructor, [`RegExp`][], a validation +function, or an object where each property will be tested for. + +If specified, `message` will be the message provided by the `AssertionError` if +the block fails to reject. + +```js +(async () => { + await assert.rejects( + async () => { + throw new Error('Wrong value'); + }, + Error + ); +})(); +``` + +```js +assert.rejects( + () => Promise.reject(new Error('Wrong value')), + Error +).then(() => { + // ... +}); +``` + ## assert.throws(block[, error][, message]) @@ -244,7 +244,7 @@ parameter is undefined, a default error message is assigned. ## assert.doesNotReject(block[, error][, message]) * `block` {Function} * `error` {RegExp|Function} @@ -670,7 +670,7 @@ If the values are not strictly equal, an `AssertionError` is thrown with a ## assert.rejects(block[, error][, message]) * `block` {Function} * `error` {RegExp|Function|Object} @@ -714,8 +714,8 @@ assert.rejects( * {boolean} diff --git a/doc/api/http2.md b/doc/api/http2.md index 464acfdbc650cf..dfa3e78b9fbf8e 100644 --- a/doc/api/http2.md +++ b/doc/api/http2.md @@ -2,7 +2,7 @@ @@ -225,7 +225,7 @@ session.on('localSettings', (settings) => { #### Event: 'ping' * `payload` {Buffer} The `PING` frame 8-byte payload @@ -682,7 +682,7 @@ are passed through as provided by the user or received from the peer. #### serverhttp2session.origin(...origins) * `origins` { string | URL | Object } One or more URL Strings passed as @@ -762,7 +762,7 @@ client.on('altsvc', (alt, origin, streamId) => { #### Event: 'origin' * `origins` {string[]} @@ -993,7 +993,7 @@ stream.on('trailers', (headers, flags) => { #### Event: 'wantTrailers' The `'wantTrailers'` event is emitted when the `Http2Stream` has queued the @@ -1046,7 +1046,7 @@ usable. #### http2stream.endAfterHeaders * {boolean} @@ -1179,7 +1179,7 @@ A current state of this `Http2Stream`. #### http2stream.sendTrailers(headers) * `headers` {HTTP/2 Headers Object} @@ -1995,7 +1995,7 @@ server.listen(80); * {boolean} diff --git a/doc/changelogs/CHANGELOG_V8.md b/doc/changelogs/CHANGELOG_V8.md index fdd5d4271bc057..ec05099604924f 100644 --- a/doc/changelogs/CHANGELOG_V8.md +++ b/doc/changelogs/CHANGELOG_V8.md @@ -10,6 +10,7 @@ +8.13.0
8.12.0
8.11.4
8.11.3
@@ -57,6 +58,152 @@ [Node.js Long Term Support Plan](https://github.com/nodejs/LTS) and will be supported actively until April 2019 and maintained until December 2019. + +## 2018-11-20, Version 8.13.0 'Carbon' (LTS), @MylesBorins prepared by @BethGriggs + +### Notable changes + +* **assert**: + - backport some assert commits (Ruben Bridgewater) [#23223](https://github.com/nodejs/node/pull/23223) +* **deps**: + - upgrade to libuv 1.23.2 (cjihrig) [#23336](https://github.com/nodejs/node/pull/23336) + - V8: cherry-pick 64-bit hash seed commits (Yang Guo) [#23274](https://github.com/nodejs/node/pull/23274) +* **http**: + - added aborted property to request (Robert Nagy) [#20094](https://github.com/nodejs/node/pull/20094) +* **http2**: + - graduate from experimental (James M Snell) [#22466](https://github.com/nodejs/node/pull/22466) + +### Commits + +* [[`0d241ba385`](https://github.com/nodejs/node/commit/0d241ba385)] - **assert**: ensure .rejects() disallows sync throws (Teddy Katz) [#19650](https://github.com/nodejs/node/pull/19650) +* [[`3babc5bb53`](https://github.com/nodejs/node/commit/3babc5bb53)] - **(SEMVER-MINOR)** **assert**: add rejects() and doesNotReject() (feugy) [#18023](https://github.com/nodejs/node/pull/18023) +* [[`18071db274`](https://github.com/nodejs/node/commit/18071db274)] - **assert**: fix throws trace (Ruben Bridgewater) [#18595](https://github.com/nodejs/node/pull/18595) +* [[`562787efb2`](https://github.com/nodejs/node/commit/562787efb2)] - **assert**: fix strict regression (Ruben Bridgewater) [#17903](https://github.com/nodejs/node/pull/17903) +* [[`f2af930ebb`](https://github.com/nodejs/node/commit/f2af930ebb)] - **(SEMVER-MINOR)** **assert**: .throws accept objects (Ruben Bridgewater) [#17584](https://github.com/nodejs/node/pull/17584) +* [[`147aeedc8d`](https://github.com/nodejs/node/commit/147aeedc8d)] - **(SEMVER-MINOR)** **assert**: improve assert.throws (Ruben Bridgewater) [#17585](https://github.com/nodejs/node/pull/17585) +* [[`c9d84b6d4f`](https://github.com/nodejs/node/commit/c9d84b6d4f)] - **assert**: fix throws and doesNotThrow stack frames (Ruben Bridgewater) [#17703](https://github.com/nodejs/node/pull/17703) +* [[`a42d0726ac`](https://github.com/nodejs/node/commit/a42d0726ac)] - **assert**: use object argument in innerFail (Ruben Bridgewater) [#17582](https://github.com/nodejs/node/pull/17582) +* [[`84948cf14f`](https://github.com/nodejs/node/commit/84948cf14f)] - **assert**: fix .throws operator (Ruben Bridgewater) [#17575](https://github.com/nodejs/node/pull/17575) +* [[`c6d94f8fa5`](https://github.com/nodejs/node/commit/c6d94f8fa5)] - **(SEMVER-MINOR)** **assert**: add strict functionality export (Ruben Bridgewater) [#17002](https://github.com/nodejs/node/pull/17002) +* [[`26d145a77f`](https://github.com/nodejs/node/commit/26d145a77f)] - **async_hooks**: add missing async\_hooks destroys in AsyncReset (Bastian Krol) [#23272](https://github.com/nodejs/node/pull/23272) +* [[`104fbc64ed`](https://github.com/nodejs/node/commit/104fbc64ed)] - **build**: update arm64 minimum supported platform (Gibson Fahnestock) [#19164](https://github.com/nodejs/node/pull/19164) +* [[`afcf059898`](https://github.com/nodejs/node/commit/afcf059898)] - **build**: do not cd on vcbuild help (Vse Mozhet Byt) [#19291](https://github.com/nodejs/node/pull/19291) +* [[`ca8d4e3450`](https://github.com/nodejs/node/commit/ca8d4e3450)] - **build**: define NOMINMAX on windows (Ben Noordhuis) [#22731](https://github.com/nodejs/node/pull/22731) +* [[`5245d6ac97`](https://github.com/nodejs/node/commit/5245d6ac97)] - **deps**: V8: partially revert d868eb7 (Ali Ijaz Sheikh) [#24499](https://github.com/nodejs/node/pull/24499) +* [[`62dd1d7bd4`](https://github.com/nodejs/node/commit/62dd1d7bd4)] - **deps**: upgrade to libuv 1.23.2 (cjihrig) [#23336](https://github.com/nodejs/node/pull/23336) +* [[`b38190ebb0`](https://github.com/nodejs/node/commit/b38190ebb0)] - **deps**: upgrade to libuv 1.23.1 (cjihrig) [#22997](https://github.com/nodejs/node/pull/22997) +* [[`d9d541c415`](https://github.com/nodejs/node/commit/d9d541c415)] - **deps**: upgrade to libuv 1.23.0 (cjihrig) [#22365](https://github.com/nodejs/node/pull/22365) +* [[`e3d08af7c1`](https://github.com/nodejs/node/commit/e3d08af7c1)] - **deps**: upgrade to libuv 1.22.0 (cjihrig) [#21731](https://github.com/nodejs/node/pull/21731) +* [[`11cb09b25a`](https://github.com/nodejs/node/commit/11cb09b25a)] - **deps**: upgrade to libuv 1.21.0 (cjihrig) [#21466](https://github.com/nodejs/node/pull/21466) +* [[`c54f4bc8e8`](https://github.com/nodejs/node/commit/c54f4bc8e8)] - **deps**: upgrade to libuv 1.20.3 (cjihrig) [#20585](https://github.com/nodejs/node/pull/20585) +* [[`2307653abf`](https://github.com/nodejs/node/commit/2307653abf)] - **deps**: upgrade to libuv 1.20.2 (cjihrig) [#20129](https://github.com/nodejs/node/pull/20129) +* [[`a1b94d35e7`](https://github.com/nodejs/node/commit/a1b94d35e7)] - **deps**: upgrade libuv to 1.20.0 (cjihrig) [#19758](https://github.com/nodejs/node/pull/19758) +* [[`ce65d84537`](https://github.com/nodejs/node/commit/ce65d84537)] - **deps**: backport a8f6869 from upstream V8 (Ben Newman) [#22714](https://github.com/nodejs/node/pull/22714) +* [[`7ab253f62e`](https://github.com/nodejs/node/commit/7ab253f62e)] - **deps**: V8: cherry-pick 64-bit hash seed commits (Yang Guo) [#23274](https://github.com/nodejs/node/pull/23274) +* [[`60f7bfa4d7`](https://github.com/nodejs/node/commit/60f7bfa4d7)] - **deps**: update to nghttp2 1.33.0 (Anna Henningsen) [#22649](https://github.com/nodejs/node/pull/22649) +* [[`48f31bdf20`](https://github.com/nodejs/node/commit/48f31bdf20)] - **deps**: V8: backport 20 CPU profiler commits from upstream (Peter Marshall) [#21558](https://github.com/nodejs/node/pull/21558) +* [[`9e2077afee`](https://github.com/nodejs/node/commit/9e2077afee)] - **deps**: backport 9a23bdd from upstream V8 (Daniel Beckert) [#22418](https://github.com/nodejs/node/pull/22418) +* [[`610297e2ab`](https://github.com/nodejs/node/commit/610297e2ab)] - **doc**: improve best practices in onboarding-extras (Rich Trott) [#19315](https://github.com/nodejs/node/pull/19315) +* [[`9446bb68ea`](https://github.com/nodejs/node/commit/9446bb68ea)] - **doc**: fix minor issues in async\_hooks.md (Rich Trott) [#19313](https://github.com/nodejs/node/pull/19313) +* [[`5b9af6ea73`](https://github.com/nodejs/node/commit/5b9af6ea73)] - **doc**: update username and email (Yuta Hiroto) [#19338](https://github.com/nodejs/node/pull/19338) +* [[`bae7c608e2`](https://github.com/nodejs/node/commit/bae7c608e2)] - **doc**: document http2 timeouts (Sagi Tsofan) [#22798](https://github.com/nodejs/node/pull/22798) +* [[`d0be932375`](https://github.com/nodejs/node/commit/d0be932375)] - **doc**: simplify http2 wording and formatting (Rich Trott) [#22541](https://github.com/nodejs/node/pull/22541) +* [[`3fe9293efc`](https://github.com/nodejs/node/commit/3fe9293efc)] - **doc**: make createPushResponse() more detailled (MaleDong) [#22366](https://github.com/nodejs/node/pull/22366) +* [[`3980ca1840`](https://github.com/nodejs/node/commit/3980ca1840)] - **doc**: clarify http2 docs around class exports (James M Snell) [#22247](https://github.com/nodejs/node/pull/22247) +* [[`32bfd7ebfb`](https://github.com/nodejs/node/commit/32bfd7ebfb)] - **doc**: add missing `require` to example in http2.md (Kevin Simper) [#21858](https://github.com/nodejs/node/pull/21858) +* [[`2116ace0ad`](https://github.com/nodejs/node/commit/2116ace0ad)] - **doc**: fix http2stream.pushStream error doc (Сковорода Никита Андреевич) [#21487](https://github.com/nodejs/node/pull/21487) +* [[`4228141012`](https://github.com/nodejs/node/commit/4228141012)] - **doc**: Improve doc for Http2 headers object (Gerhard Stoebich) [#21296](https://github.com/nodejs/node/pull/21296) +* [[`11a63ddf48`](https://github.com/nodejs/node/commit/11a63ddf48)] - **doc**: fix typo in http2.md (Keita Akutsu) [#20843](https://github.com/nodejs/node/pull/20843) +* [[`4f0035485f`](https://github.com/nodejs/node/commit/4f0035485f)] - **doc**: add parameters for Http2Stream:error event (Ujjwal Sharma) [#20610](https://github.com/nodejs/node/pull/20610) +* [[`77acef4af2`](https://github.com/nodejs/node/commit/77acef4af2)] - **doc**: add params for ClientHttp2Session:altsvc (Ujjwal Sharma) [#20598](https://github.com/nodejs/node/pull/20598) +* [[`448922d0de`](https://github.com/nodejs/node/commit/448922d0de)] - **doc**: add parameters for Http2Session:stream event (Ujjwal Sharma) [#20547](https://github.com/nodejs/node/pull/20547) +* [[`41e89316e6`](https://github.com/nodejs/node/commit/41e89316e6)] - **doc**: add parameters for settings events (Ujjwal Sharma) [#20371](https://github.com/nodejs/node/pull/20371) +* [[`1a6a054899`](https://github.com/nodejs/node/commit/1a6a054899)] - **doc**: improve parameters for Http2Session:goaway event (Ujjwal Sharma) +* [[`98ed30f3f5`](https://github.com/nodejs/node/commit/98ed30f3f5)] - **doc**: improve docs for Http2Session:frameError (Ujjwal Sharma) [#20236](https://github.com/nodejs/node/pull/20236) +* [[`b32cf8fa40`](https://github.com/nodejs/node/commit/b32cf8fa40)] - **doc**: add parameters for Http2Session:error event (Ujjwal Sharma) [#20206](https://github.com/nodejs/node/pull/20206) +* [[`c0d1423bd3`](https://github.com/nodejs/node/commit/c0d1423bd3)] - **doc**: close event does not take arguments (Indranil Dasgupta) [#20031](https://github.com/nodejs/node/pull/20031) +* [[`459690aca4`](https://github.com/nodejs/node/commit/459690aca4)] - **doc**: improve style guide text (Rich Trott) [#19269](https://github.com/nodejs/node/pull/19269) +* [[`eaabbf4ff0`](https://github.com/nodejs/node/commit/eaabbf4ff0)] - **doc**: make caveat in stream.md more concise (Rich Trott) [#19251](https://github.com/nodejs/node/pull/19251) +* [[`0340dd8c8d`](https://github.com/nodejs/node/commit/0340dd8c8d)] - **doc**: add and unify return statements in crypto.md (Vse Mozhet Byt) [#19853](https://github.com/nodejs/node/pull/19853) +* [[`b0d6067d87`](https://github.com/nodejs/node/commit/b0d6067d87)] - **doc**: fix 8.12.0 changelog (Myles Borins) [#22803](https://github.com/nodejs/node/pull/22803) +* [[`af5cebb326`](https://github.com/nodejs/node/commit/af5cebb326)] - **doc,http2**: add parameters for Http2Session:connect event (Ujjwal Sharma) [#20193](https://github.com/nodejs/node/pull/20193) +* [[`57618aae0a`](https://github.com/nodejs/node/commit/57618aae0a)] - **errors**: fix undefined HTTP2 and tls errors (Shailesh Shekhawat) [#21564](https://github.com/nodejs/node/pull/21564) +* [[`e3bddeec18`](https://github.com/nodejs/node/commit/e3bddeec18)] - **http**: fix undefined error in parser event (Anatoli Papirovski) [#20029](https://github.com/nodejs/node/pull/20029) +* [[`1edd7f6393`](https://github.com/nodejs/node/commit/1edd7f6393)] - **(SEMVER-MINOR)** **http**: added aborted property to request (Robert Nagy) [#20094](https://github.com/nodejs/node/pull/20094) +* [[`7f34c277ac`](https://github.com/nodejs/node/commit/7f34c277ac)] - **http2**: simplify timeout tracking (Anna Henningsen) [#19206](https://github.com/nodejs/node/pull/19206) +* [[`18a2b3dc8e`](https://github.com/nodejs/node/commit/18a2b3dc8e)] - **(SEMVER-MINOR)** **http2**: graduate from experimental (James M Snell) [#22466](https://github.com/nodejs/node/pull/22466) +* [[`10576d6e77`](https://github.com/nodejs/node/commit/10576d6e77)] - **(SEMVER-MINOR)** **http2**: add ping event (James M Snell) [#23009](https://github.com/nodejs/node/pull/23009) +* [[`ca933ce577`](https://github.com/nodejs/node/commit/ca933ce577)] - **http2**: do not falsely emit 'aborted' on push (Anatoli Papirovski) [#22878](https://github.com/nodejs/node/pull/22878) +* [[`49f44f3b44`](https://github.com/nodejs/node/commit/49f44f3b44)] - **(SEMVER-MINOR)** **http2**: add origin frame support (James M Snell) [#22956](https://github.com/nodejs/node/pull/22956) +* [[`9f7934159e`](https://github.com/nodejs/node/commit/9f7934159e)] - **http2**: check if stream is not destroyed before sending trailers (Matteo Collina) [#22896](https://github.com/nodejs/node/pull/22896) +* [[`2de17ead89`](https://github.com/nodejs/node/commit/2de17ead89)] - **(SEMVER-MINOR)** **http2**: add http2stream.endAfterHeaders property (James M Snell) [#22843](https://github.com/nodejs/node/pull/22843) +* [[`805bf40bfd`](https://github.com/nodejs/node/commit/805bf40bfd)] - **http2**: don't expose the original socket through the socket proxy (Szymon Marczak) [#22650](https://github.com/nodejs/node/pull/22650) +* [[`6a396ff911`](https://github.com/nodejs/node/commit/6a396ff911)] - **http2**: throw better error when accessing unbound socket proxy (James M Snell) [#22486](https://github.com/nodejs/node/pull/22486) +* [[`348cde07fd`](https://github.com/nodejs/node/commit/348cde07fd)] - **http2**: emit timeout on compat request and response (James M Snell) [#22252](https://github.com/nodejs/node/pull/22252) +* [[`cc561cc5a7`](https://github.com/nodejs/node/commit/cc561cc5a7)] - **http2**: explicitly disallow nested push streams (James M Snell) [#22245](https://github.com/nodejs/node/pull/22245) +* [[`5c3edd3479`](https://github.com/nodejs/node/commit/5c3edd3479)] - **http2**: avoid race condition in OnHeaderCallback (James M Snell) [#22256](https://github.com/nodejs/node/pull/22256) +* [[`f2f66b4cfb`](https://github.com/nodejs/node/commit/f2f66b4cfb)] - **http2**: remove `streamError` from docs (James M Snell) [#22246](https://github.com/nodejs/node/pull/22246) +* [[`d602c7a2ed`](https://github.com/nodejs/node/commit/d602c7a2ed)] - **http2**: release request()'s "connect" event listener after it runs (James Ide) [#21916](https://github.com/nodejs/node/pull/21916) +* [[`745e1e6192`](https://github.com/nodejs/node/commit/745e1e6192)] - **http2**: remove unused nghttp2 error list (Anna Henningsen) [#21827](https://github.com/nodejs/node/pull/21827) +* [[`e5175e6596`](https://github.com/nodejs/node/commit/e5175e6596)] - **http2**: remove `waitTrailers` listener after closing a stream (RidgeA) [#21764](https://github.com/nodejs/node/pull/21764) +* [[`071a022dbc`](https://github.com/nodejs/node/commit/071a022dbc)] - **http2**: order declarations in core.js (Rich Trott) [#21689](https://github.com/nodejs/node/pull/21689) +* [[`1cdf93ecdc`](https://github.com/nodejs/node/commit/1cdf93ecdc)] - **http2**: pass incoming set-cookie header as array (Gerhard Stoebich) [#21360](https://github.com/nodejs/node/pull/21360) +* [[`20b72fc94d`](https://github.com/nodejs/node/commit/20b72fc94d)] - **http2**: track memory allocated by nghttp2 (Anna Henningsen) [#21374](https://github.com/nodejs/node/pull/21374) +* [[`e9e4f434b3`](https://github.com/nodejs/node/commit/e9e4f434b3)] - **http2**: fix memory leak when headers are not emitted (Anna Henningsen) [#21373](https://github.com/nodejs/node/pull/21373) +* [[`0f3e65099d`](https://github.com/nodejs/node/commit/0f3e65099d)] - **http2**: fix memory leak for uncommon headers (Anna Henningsen) [#21336](https://github.com/nodejs/node/pull/21336) +* [[`0a8d0861f2`](https://github.com/nodejs/node/commit/0a8d0861f2)] - **http2**: safer Http2Session destructor (Anatoli Papirovski) [#21194](https://github.com/nodejs/node/pull/21194) +* [[`3c8c53f4f4`](https://github.com/nodejs/node/commit/3c8c53f4f4)] - **http2**: fix premature destroy (Anatoli Papirovski) [#21051](https://github.com/nodejs/node/pull/21051) +* [[`b22266cc97`](https://github.com/nodejs/node/commit/b22266cc97)] - **http2**: force through RST\_STREAM in destroy (Anatoli Papirovski) [#21016](https://github.com/nodejs/node/pull/21016) +* [[`91be1dc2a5`](https://github.com/nodejs/node/commit/91be1dc2a5)] - **http2**: delay closing stream (Anatoli Papirovski) [#20997](https://github.com/nodejs/node/pull/20997) +* [[`0a6672fbcf`](https://github.com/nodejs/node/commit/0a6672fbcf)] - **http2**: fix several serious bugs (Anatoli Papirovski) [#20772](https://github.com/nodejs/node/pull/20772) +* [[`b0c92cadfa`](https://github.com/nodejs/node/commit/b0c92cadfa)] - **http2**: fix end without read (Anatoli Papirovski) [#20621](https://github.com/nodejs/node/pull/20621) +* [[`d1b78252b1`](https://github.com/nodejs/node/commit/d1b78252b1)] - **http2**: avoid bind and properly clean up in compat (Robert Nagy) [#20374](https://github.com/nodejs/node/pull/20374) +* [[`395ce845da`](https://github.com/nodejs/node/commit/395ce845da)] - **http2**: rename http2\_state class to Http2State (Daniel Bevenius) [#20423](https://github.com/nodejs/node/pull/20423) +* [[`74192ddb66`](https://github.com/nodejs/node/commit/74192ddb66)] - **http2**: reduce require calls in http2/core (Daniel Bevenius) [#20422](https://github.com/nodejs/node/pull/20422) +* [[`28a6e59bd3`](https://github.com/nodejs/node/commit/28a6e59bd3)] - **http2**: fix ping callback (Ruben Bridgewater) [#20311](https://github.com/nodejs/node/pull/20311) +* [[`41dca9e851`](https://github.com/nodejs/node/commit/41dca9e851)] - **http2**: fix responses to long payload reqs (Anatoli Papirovski) [#20084](https://github.com/nodejs/node/pull/20084) +* [[`fa5a3809a3`](https://github.com/nodejs/node/commit/fa5a3809a3)] - **http2**: refactor how trailers are done (James M Snell) [#19959](https://github.com/nodejs/node/pull/19959) +* [[`5862d0372c`](https://github.com/nodejs/node/commit/5862d0372c)] - **http2**: fix ping duration calculation (James M Snell) [#19956](https://github.com/nodejs/node/pull/19956) +* [[`2ae98ce7cb`](https://github.com/nodejs/node/commit/2ae98ce7cb)] - **lib**: define printErr() in script string (cjihrig) [#19285](https://github.com/nodejs/node/pull/19285) +* [[`b0e3ce9c4b`](https://github.com/nodejs/node/commit/b0e3ce9c4b)] - **net,http2**: refactor \_write and \_writev (Ujjwal Sharma) [#20643](https://github.com/nodejs/node/pull/20643) +* [[`0187e3bef8`](https://github.com/nodejs/node/commit/0187e3bef8)] - **process**: avoid using the same fd for ipc and stdio (cjihrig) [#21466](https://github.com/nodejs/node/pull/21466) +* [[`5b2f6508f9`](https://github.com/nodejs/node/commit/5b2f6508f9)] - **src**: make AsyncWrap constructors delegate (Daniel Bevenius) [#19366](https://github.com/nodejs/node/pull/19366) +* [[`9e8f4e5047`](https://github.com/nodejs/node/commit/9e8f4e5047)] - **src**: remove unused uv.h include from async\_wrap.cc (Daniel Bevenius) [#19342](https://github.com/nodejs/node/pull/19342) +* [[`042434f9af`](https://github.com/nodejs/node/commit/042434f9af)] - **src**: fix indenting of wrap-\>EmitTraceEventBefore (Daniel Bevenius) [#19340](https://github.com/nodejs/node/pull/19340) +* [[`3ad10e5789`](https://github.com/nodejs/node/commit/3ad10e5789)] - **src**: add extractPromiseWrap function (Daniel Bevenius) [#19340](https://github.com/nodejs/node/pull/19340) +* [[`b67bf38f31`](https://github.com/nodejs/node/commit/b67bf38f31)] - **src**: fix fs.write() externalized string handling (Ben Noordhuis) [#18216](https://github.com/nodejs/node/pull/18216) +* [[`0157e3ebca`](https://github.com/nodejs/node/commit/0157e3ebca)] - **src,deps**: add ABI safe use of CheckMemoryPressure (Ali Ijaz Sheikh) [#24499](https://github.com/nodejs/node/pull/24499) +* [[`dbc7d9baae`](https://github.com/nodejs/node/commit/dbc7d9baae)] - **test**: read() on dir on AIX does not return EISDIR (Ben Noordhuis) [#23330](https://github.com/nodejs/node/pull/23330) +* [[`3cd4462370`](https://github.com/nodejs/node/commit/3cd4462370)] - **test**: ensure failed assertions cause build to fail (Teddy Katz) [#19650](https://github.com/nodejs/node/pull/19650) +* [[`9f15bc40b8`](https://github.com/nodejs/node/commit/9f15bc40b8)] - **test**: skip failing tests for osx mojave (jn99) [#23550](https://github.com/nodejs/node/pull/23550) +* [[`aba1ff202c`](https://github.com/nodejs/node/commit/aba1ff202c)] - **test**: refactor test-fs-readfile-tostring-fail (Rich Trott) [#19404](https://github.com/nodejs/node/pull/19404) +* [[`38ed6c2b25`](https://github.com/nodejs/node/commit/38ed6c2b25)] - **test**: fix flaky test-http2-ping-flood (Rich Trott) [#19395](https://github.com/nodejs/node/pull/19395) +* [[`b407060556`](https://github.com/nodejs/node/commit/b407060556)] - **test**: fix flaky test-http2-settings-flood (Rich Trott) [#19349](https://github.com/nodejs/node/pull/19349) +* [[`069fd79424`](https://github.com/nodejs/node/commit/069fd79424)] - **test**: improve debugging information for http2 test (Rich Trott) [#23058](https://github.com/nodejs/node/pull/23058) +* [[`c0f8e49c32`](https://github.com/nodejs/node/commit/c0f8e49c32)] - **test**: remove setImmediate from timeout test (Rich Trott) [#23058](https://github.com/nodejs/node/pull/23058) +* [[`b66cba0766`](https://github.com/nodejs/node/commit/b66cba0766)] - **test**: add test-http2-large-file sequential test (James M Snell) [#22254](https://github.com/nodejs/node/pull/22254) +* [[`7ea08eedac`](https://github.com/nodejs/node/commit/7ea08eedac)] - **test**: improve reliability in http2-session-timeout (Rich Trott) [#22026](https://github.com/nodejs/node/pull/22026) +* [[`dcf04dc7df`](https://github.com/nodejs/node/commit/dcf04dc7df)] - **test**: refactor test-http2-compat-serverresponse-finished.js (Anto Aravinth) [#21929](https://github.com/nodejs/node/pull/21929) +* [[`322f39d490`](https://github.com/nodejs/node/commit/322f39d490)] - **test**: minor adjustments to test-http2-respond-file (Anna Henningsen) [#21098](https://github.com/nodejs/node/pull/21098) +* [[`5d29e2c631`](https://github.com/nodejs/node/commit/5d29e2c631)] - **test**: fix flaky http2-session-unref (Anatoli Papirovski) [#20772](https://github.com/nodejs/node/pull/20772) +* [[`e5f8b08305`](https://github.com/nodejs/node/commit/e5f8b08305)] - **test**: improve reliability of http2-session-timeout (Rich Trott) [#20692](https://github.com/nodejs/node/pull/20692) +* [[`c30a8f468d`](https://github.com/nodejs/node/commit/c30a8f468d)] - **test**: fix flaky http2-flow-control test (Anatoli Papirovski) [#20556](https://github.com/nodejs/node/pull/20556) +* [[`aa341d1d3d`](https://github.com/nodejs/node/commit/aa341d1d3d)] - **test**: verify arguments length in common.expectsError (Ruben Bridgewater) [#20311](https://github.com/nodejs/node/pull/20311) +* [[`c7ba556264`](https://github.com/nodejs/node/commit/c7ba556264)] - **test**: removed assert.strictEqual message (kailash k yogeshwar) [#20223](https://github.com/nodejs/node/pull/20223) +* [[`5abe246a44`](https://github.com/nodejs/node/commit/5abe246a44)] - **test**: add strictEqual method to assert (Christine E. Taylor) [#20189](https://github.com/nodejs/node/pull/20189) +* [[`887417eb37`](https://github.com/nodejs/node/commit/887417eb37)] - **test**: remove message from strictEqual assertions (Bryan Azofeifa) [#20174](https://github.com/nodejs/node/pull/20174) +* [[`fe3836a871`](https://github.com/nodejs/node/commit/fe3836a871)] - **test**: delete test/parallel/test-regress-GH-4948 (Ujjwal Sharma) +* [[`4bcdc1b83c`](https://github.com/nodejs/node/commit/4bcdc1b83c)] - **test**: fix assertion argument order (Rich Trott) [#19264](https://github.com/nodejs/node/pull/19264) +* [[`534bc82578`](https://github.com/nodejs/node/commit/534bc82578)] - **test**: name test files appropriately (Ujjwal Sharma) [#19212](https://github.com/nodejs/node/pull/19212) +* [[`d58867a6a7`](https://github.com/nodejs/node/commit/d58867a6a7)] - **test**: call gc() explicitly to avoid OOM (Refael Ackermann) [#22301](https://github.com/nodejs/node/pull/22301) +* [[`8209ccb313`](https://github.com/nodejs/node/commit/8209ccb313)] - **test**: prepare test-assert for strictEqual linting (Rich Trott) [#22849](https://github.com/nodejs/node/pull/22849) +* [[`52b21caff2`](https://github.com/nodejs/node/commit/52b21caff2)] - **test**: remove string literal from assertion (Rich Trott) [#22849](https://github.com/nodejs/node/pull/22849) +* [[`976d55f9e3`](https://github.com/nodejs/node/commit/976d55f9e3)] - **test**: remove string literal from assertion (Rich Trott) [#22849](https://github.com/nodejs/node/pull/22849) +* [[`702d67f4c4`](https://github.com/nodejs/node/commit/702d67f4c4)] - **test**: refactor flag check (Rich Trott) [#22849](https://github.com/nodejs/node/pull/22849) +* [[`e9416d4f67`](https://github.com/nodejs/node/commit/e9416d4f67)] - **test**: simplify assertion in http2 tests (Rich Trott) [#22849](https://github.com/nodejs/node/pull/22849) +* [[`f2158f30fb`](https://github.com/nodejs/node/commit/f2158f30fb)] - **test**: improve assertion in test-inspector.js (Rich Trott) [#22849](https://github.com/nodejs/node/pull/22849) +* [[`f5985c734c`](https://github.com/nodejs/node/commit/f5985c734c)] - **tls,http2**: handle writes after SSL destroy more gracefully (Anna Henningsen) [#18987](https://github.com/nodejs/node/pull/18987) + ## 2018-09-11, Version 8.12.0 'Carbon' (LTS), @MylesBorins @@ -79,7 +226,7 @@ will be supported actively until April 2019 and maintained until December 2019. - support as and as+ flags in stringToFlags() (Sarat Addepalli) [#18801](https://github.com/nodejs/node/pull/18801) - emit 'ready' for fs streams and sockets (Sameer Srivastava) [#19408](https://github.com/nodejs/node/pull/19408) * **http, http2**: - - add options to http.createServer() (Peter Marton) [#15752](https://github.com/nodejs/node/pull/15752)- + - add options to http.createServer() (Peter Marton) [#15752](https://github.com/nodejs/node/pull/15752)- - add 103 Early Hints status code (Yosuke Furukawa) [#16644](https://github.com/nodejs/node/pull/16644) - add http fallback options to .createServer (Peter Marton) [#15752](https://github.com/nodejs/node/pull/15752) * **n-api**: @@ -246,7 +393,7 @@ will be supported actively until April 2019 and maintained until December 2019. * [[`e17f05a817`](https://github.com/nodejs/node/commit/e17f05a817)] - **src**: create per-isolate strings after platform setup (Ulan Degenbaev) [#20175](https://github.com/nodejs/node/pull/20175) * [[`d38ccbb07f`](https://github.com/nodejs/node/commit/d38ccbb07f)] - **src**: use `unordered\_map` for perf marks (Anna Henningsen) [#19558](https://github.com/nodejs/node/pull/19558) * [[`553e34ef9c`](https://github.com/nodejs/node/commit/553e34ef9c)] - **src**: simplify http2 perf tracking code (Anna Henningsen) [#19470](https://github.com/nodejs/node/pull/19470) -* [[`67182912d7`](https://github.com/nodejs/node/commit/67182912d7)] - **src**: add "icu::" prefix before ICU symbols (Steven R. Loomis) +* [[`67182912d7`](https://github.com/nodejs/node/commit/67182912d7)] - **src**: add "icu::" prefix before ICU symbols (Steven R. Loomis) * [[`2cf263519a`](https://github.com/nodejs/node/commit/2cf263519a)] - **src**: use unique\_ptr for scheduled delayed tasks (Franziska Hinkelmann) [#17083](https://github.com/nodejs/node/pull/17083) * [[`2148b1921e`](https://github.com/nodejs/node/commit/2148b1921e)] - **src**: use unique\_ptr in platform implementation (Franziska Hinkelmann) [#16970](https://github.com/nodejs/node/pull/16970) * [[`e9327541e1`](https://github.com/nodejs/node/commit/e9327541e1)] - **src**: cancel pending delayed platform tasks on exit (Anna Henningsen) [#16700](https://github.com/nodejs/node/pull/16700) diff --git a/src/node_version.h b/src/node_version.h index 09d6679202781f..f82781dcdf186d 100644 --- a/src/node_version.h +++ b/src/node_version.h @@ -23,13 +23,13 @@ #define SRC_NODE_VERSION_H_ #define NODE_MAJOR_VERSION 8 -#define NODE_MINOR_VERSION 12 -#define NODE_PATCH_VERSION 1 +#define NODE_MINOR_VERSION 13 +#define NODE_PATCH_VERSION 0 #define NODE_VERSION_IS_LTS 1 #define NODE_VERSION_LTS_CODENAME "Carbon" -#define NODE_VERSION_IS_RELEASE 0 +#define NODE_VERSION_IS_RELEASE 1 #ifndef NODE_STRINGIFY #define NODE_STRINGIFY(n) NODE_STRINGIFY_HELPER(n)