-
Notifications
You must be signed in to change notification settings - Fork 29.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
async_hooks: don't reuse resource in HttpAgent #27581
Closed
Flarna
wants to merge
2
commits into
nodejs:master
from
dynatrace-oss-contrib:async-hooks-reuse-http-agent
Closed
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,110 @@ | ||
'use strict'; | ||
// Flags: --expose-internals | ||
const common = require('../common'); | ||
const initHooks = require('./init-hooks'); | ||
const { checkInvocations } = require('./hook-checks'); | ||
const assert = require('assert'); | ||
const { async_id_symbol } = require('internal/async_hooks').symbols; | ||
const http = require('http'); | ||
|
||
// Checks that the async resource used in init in case of a resused handle | ||
// is not reused. Test is based on parallel\test-async-hooks-http-agent.js. | ||
|
||
const hooks = initHooks(); | ||
hooks.enable(); | ||
|
||
let asyncIdAtFirstReq; | ||
let asyncIdAtSecondReq; | ||
|
||
// Make sure a single socket is transparently reused for 2 requests. | ||
const agent = new http.Agent({ | ||
keepAlive: true, | ||
keepAliveMsecs: Infinity, | ||
maxSockets: 1 | ||
}); | ||
|
||
const server = http.createServer(common.mustCall((req, res) => { | ||
req.once('data', common.mustCallAtLeast(() => { | ||
res.writeHead(200, { 'Content-Type': 'text/plain' }); | ||
res.write('foo'); | ||
})); | ||
req.on('end', common.mustCall(() => { | ||
res.end('bar'); | ||
})); | ||
}, 2)).listen(0, common.mustCall(() => { | ||
const port = server.address().port; | ||
const payload = 'hello world'; | ||
|
||
// First request. This is useless except for adding a socket to the | ||
// agent’s pool for reuse. | ||
const r1 = http.request({ | ||
agent, port, method: 'POST' | ||
}, common.mustCall((res) => { | ||
// Remember which socket we used. | ||
const socket = res.socket; | ||
asyncIdAtFirstReq = socket[async_id_symbol]; | ||
assert.ok(asyncIdAtFirstReq > 0, `${asyncIdAtFirstReq} > 0`); | ||
// Check that request and response share their socket. | ||
assert.strictEqual(r1.socket, socket); | ||
|
||
res.on('data', common.mustCallAtLeast(() => {})); | ||
res.on('end', common.mustCall(() => { | ||
// setImmediate() to give the agent time to register the freed socket. | ||
setImmediate(common.mustCall(() => { | ||
// The socket is free for reuse now. | ||
assert.strictEqual(socket[async_id_symbol], -1); | ||
|
||
// Second request. To re-create the exact conditions from the | ||
// referenced issue, we use a POST request without chunked encoding | ||
// (hence the Content-Length header) and call .end() after the | ||
// response header has already been received. | ||
const r2 = http.request({ | ||
agent, port, method: 'POST', headers: { | ||
'Content-Length': payload.length | ||
} | ||
}, common.mustCall((res) => { | ||
asyncIdAtSecondReq = res.socket[async_id_symbol]; | ||
assert.ok(asyncIdAtSecondReq > 0, `${asyncIdAtSecondReq} > 0`); | ||
assert.strictEqual(r2.socket, socket); | ||
|
||
// Empty payload, to hit the “right” code path. | ||
r2.end(''); | ||
|
||
res.on('data', common.mustCallAtLeast(() => {})); | ||
res.on('end', common.mustCall(() => { | ||
// Clean up to let the event loop stop. | ||
server.close(); | ||
agent.destroy(); | ||
})); | ||
})); | ||
|
||
// Schedule a payload to be written immediately, but do not end the | ||
// request just yet. | ||
r2.write(payload); | ||
})); | ||
})); | ||
})); | ||
r1.end(payload); | ||
})); | ||
|
||
|
||
process.on('exit', onExit); | ||
|
||
function onExit() { | ||
hooks.disable(); | ||
hooks.sanityCheck(); | ||
const activities = hooks.activities; | ||
|
||
// Verify both invocations | ||
const first = activities.filter((x) => x.uid === asyncIdAtFirstReq)[0]; | ||
checkInvocations(first, { init: 1, destroy: 1 }, 'when process exits'); | ||
|
||
const second = activities.filter((x) => x.uid === asyncIdAtSecondReq)[0]; | ||
checkInvocations(second, { init: 1, destroy: 1 }, 'when process exits'); | ||
|
||
// Verify reuse handle has been wrapped | ||
assert.strictEqual(first.type, second.type); | ||
assert.ok(first.handle !== second.handle, 'Resource reused'); | ||
Flarna marked this conversation as resolved.
Show resolved
Hide resolved
|
||
assert.ok(first.handle === second.handle.handle, | ||
Flarna marked this conversation as resolved.
Show resolved
Hide resolved
|
||
'Resource not wrapped correctly'); | ||
} |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This doesn’t have to stop this from landing, but thinking out loud a bit … I think we don’t have to deviate from the original async_hooks design this strongly, at least as far as the 1:1 correspondence of native objects and JS objects is concerned (which also has the advantage of having the resource objects provide information about the resource itself, which this PR undoes).
I think ideally what we would want for these cases (and maybe others) is to make the resource object passed to async_hooks refer to the native object, and give the native object a property over which we can figure out the current async_hooks resource object assoicated with it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As far as I know for promises the resource passed via AsyncHooks is not linking to the
Promise
object itself at this time. So at least there we deviate already.To my understanding passing the native resource would avoid to implement
currentResource
(#21313) as long as they are reused as setting a property on them or use them in aWeakMap
would result in leak (except if the property gets removed via delete hook but this is the hook we want to get rid of).I fully agree that it is not optimal to use different object types as resources depending on reuse or not.
My idea in this is different but not sure if it can implemented efficiently: Instead of passing the native resource object itself (which is often internal and not well documented, sometimes partly initialized,...) we could pass a well defined object for all resources. This object offers an APIs related to the current executing resource and even more related to scheduling (a name I have seen in other languages: ExecutionContext). With such an object in placed it's easier to extend the API as we are no longer bound to already existing low level objects.
If we consider the currently passed resource objects and types as part of the AsyncHooks API it's in my opinion more or less impossible to get it out of experimental as to much low level APIs are then part of the public API which would hinder refactorings,...
I could imagine that it would be also more efficient to think/signal more like a scheduler and issue only one
executionChanged
event with old and new context instead of aafter
event directly followed by abefore
event.But again, this is just some idea I have in mind, not yet tried to implement it and not even sure if we get the needed information from v8/libuv. For sure this would be a breaking change.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Flarna Yeah, I think we are in agreement here :)