Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

async_hooks: don't reuse resource in HttpAgent #27581

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 11 additions & 3 deletions lib/_http_agent.js
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,13 @@ const { async_id_symbol } = require('internal/async_hooks').symbols;
// ClientRequest.onSocket(). The Agent is now *strictly*
// concerned with managing a connection pool.

class ReusedHandle {
constructor(type, handle) {
this.type = type;
this.handle = handle;
}
}

function Agent(options) {
if (!(this instanceof Agent))
return new Agent(options);
Expand Down Expand Up @@ -166,10 +173,11 @@ Agent.prototype.addRequest = function addRequest(req, options, port/* legacy */,
// We have a free socket, so use that.
var socket = this.freeSockets[name].shift();
// Guard against an uninitialized or user supplied Socket.
if (socket._handle && typeof socket._handle.asyncReset === 'function') {
const handle = socket._handle;
if (handle && typeof handle.asyncReset === 'function') {
// Assign the handle a new asyncId and run any destroy()/init() hooks.
socket._handle.asyncReset();
socket[async_id_symbol] = socket._handle.getAsyncId();
handle.asyncReset(new ReusedHandle(handle.getProviderType(), handle));
socket[async_id_symbol] = handle.getAsyncId();
}

// don't leak
Expand Down
18 changes: 16 additions & 2 deletions src/async_wrap.cc
Original file line number Diff line number Diff line change
Expand Up @@ -412,13 +412,26 @@ void AsyncWrap::PopAsyncIds(const FunctionCallbackInfo<Value>& args) {


void AsyncWrap::AsyncReset(const FunctionCallbackInfo<Value>& args) {
CHECK(args[0]->IsObject());

AsyncWrap* wrap;
ASSIGN_OR_RETURN_UNWRAP(&wrap, args.Holder());

Local<Object> resource = args[0].As<Object>();
double execution_async_id =
args[0]->IsNumber() ? args[0].As<Number>()->Value() : kInvalidAsyncId;
wrap->AsyncReset(execution_async_id);
args[1]->IsNumber() ? args[1].As<Number>()->Value() : kInvalidAsyncId;
wrap->AsyncReset(resource, execution_async_id);
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn’t have to stop this from landing, but thinking out loud a bit … I think we don’t have to deviate from the original async_hooks design this strongly, at least as far as the 1:1 correspondence of native objects and JS objects is concerned (which also has the advantage of having the resource objects provide information about the resource itself, which this PR undoes).

I think ideally what we would want for these cases (and maybe others) is to make the resource object passed to async_hooks refer to the native object, and give the native object a property over which we can figure out the current async_hooks resource object assoicated with it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I know for promises the resource passed via AsyncHooks is not linking to the Promise object itself at this time. So at least there we deviate already.

To my understanding passing the native resource would avoid to implement currentResource (#21313) as long as they are reused as setting a property on them or use them in a WeakMap would result in leak (except if the property gets removed via delete hook but this is the hook we want to get rid of).

I fully agree that it is not optimal to use different object types as resources depending on reuse or not.

My idea in this is different but not sure if it can implemented efficiently: Instead of passing the native resource object itself (which is often internal and not well documented, sometimes partly initialized,...) we could pass a well defined object for all resources. This object offers an APIs related to the current executing resource and even more related to scheduling (a name I have seen in other languages: ExecutionContext). With such an object in placed it's easier to extend the API as we are no longer bound to already existing low level objects.
If we consider the currently passed resource objects and types as part of the AsyncHooks API it's in my opinion more or less impossible to get it out of experimental as to much low level APIs are then part of the public API which would hinder refactorings,...

I could imagine that it would be also more efficient to think/signal more like a scheduler and issue only one executionChanged event with old and new context instead of a after event directly followed by a before event.

But again, this is just some idea I have in mind, not yet tried to implement it and not even sure if we get the needed information from v8/libuv. For sure this would be a breaking change.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My idea in this is different but not sure if it can implemented efficiently: Instead of passing the native resource object itself (which is often internal and not well documented, sometimes partly initialized,...) we could pass a well defined object for all resources. This object offers an APIs related to the current executing resource and even more related to scheduling (a name I have seen in other languages: ExecutionContext). With such an object in placed it's easier to extend the API as we are no longer bound to already existing low level objects.

@Flarna Yeah, I think we are in agreement here :)



void AsyncWrap::GetProviderType(const FunctionCallbackInfo<Value>& args) {
AsyncWrap* wrap;
args.GetReturnValue().Set(AsyncWrap::PROVIDER_NONE);
ASSIGN_OR_RETURN_UNWRAP(&wrap, args.Holder());
args.GetReturnValue().Set(wrap->provider_type());
}


void AsyncWrap::EmitDestroy() {
AsyncWrap::EmitDestroy(env(), async_id_);
// Ensure no double destroy is emitted via AsyncReset().
Expand All @@ -439,6 +452,7 @@ Local<FunctionTemplate> AsyncWrap::GetConstructorTemplate(Environment* env) {
tmpl->SetClassName(FIXED_ONE_BYTE_STRING(env->isolate(), "AsyncWrap"));
env->SetProtoMethod(tmpl, "getAsyncId", AsyncWrap::GetAsyncId);
env->SetProtoMethod(tmpl, "asyncReset", AsyncWrap::AsyncReset);
env->SetProtoMethod(tmpl, "getProviderType", AsyncWrap::GetProviderType);
env->set_async_wrap_ctor_template(tmpl);
}
return tmpl;
Expand Down
1 change: 1 addition & 0 deletions src/async_wrap.h
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,7 @@ class AsyncWrap : public BaseObject {
static void PushAsyncIds(const v8::FunctionCallbackInfo<v8::Value>& args);
static void PopAsyncIds(const v8::FunctionCallbackInfo<v8::Value>& args);
static void AsyncReset(const v8::FunctionCallbackInfo<v8::Value>& args);
static void GetProviderType(const v8::FunctionCallbackInfo<v8::Value>& args);
static void QueueDestroyAsyncId(
const v8::FunctionCallbackInfo<v8::Value>& args);

Expand Down
5 changes: 3 additions & 2 deletions test/async-hooks/init-hooks.js
Original file line number Diff line number Diff line change
Expand Up @@ -158,7 +158,7 @@ class ActivityCollector {
// events this makes sense for a few tests in which we enable some hooks
// later
if (this._allowNoInit) {
const stub = { uid, type: 'Unknown', handleIsObject: true };
const stub = { uid, type: 'Unknown', handleIsObject: true, handle: {} };
this._activities.set(uid, stub);
return stub;
} else if (!common.isMainThread) {
Expand All @@ -184,7 +184,8 @@ class ActivityCollector {
triggerAsyncId,
// In some cases (e.g. Timeout) the handle is a function, thus the usual
// `typeof handle === 'object' && handle !== null` check can't be used.
handleIsObject: handle instanceof Object
handleIsObject: handle instanceof Object,
handle
};
this._stamp(activity, 'init');
this._activities.set(uid, activity);
Expand Down
110 changes: 110 additions & 0 deletions test/async-hooks/test-http-agent-handle-reuse.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
'use strict';
// Flags: --expose-internals
const common = require('../common');
const initHooks = require('./init-hooks');
const { checkInvocations } = require('./hook-checks');
const assert = require('assert');
const { async_id_symbol } = require('internal/async_hooks').symbols;
const http = require('http');

// Checks that the async resource used in init in case of a resused handle
// is not reused. Test is based on parallel\test-async-hooks-http-agent.js.

const hooks = initHooks();
hooks.enable();

let asyncIdAtFirstReq;
let asyncIdAtSecondReq;

// Make sure a single socket is transparently reused for 2 requests.
const agent = new http.Agent({
keepAlive: true,
keepAliveMsecs: Infinity,
maxSockets: 1
});

const server = http.createServer(common.mustCall((req, res) => {
req.once('data', common.mustCallAtLeast(() => {
res.writeHead(200, { 'Content-Type': 'text/plain' });
res.write('foo');
}));
req.on('end', common.mustCall(() => {
res.end('bar');
}));
}, 2)).listen(0, common.mustCall(() => {
const port = server.address().port;
const payload = 'hello world';

// First request. This is useless except for adding a socket to the
// agent’s pool for reuse.
const r1 = http.request({
agent, port, method: 'POST'
}, common.mustCall((res) => {
// Remember which socket we used.
const socket = res.socket;
asyncIdAtFirstReq = socket[async_id_symbol];
assert.ok(asyncIdAtFirstReq > 0, `${asyncIdAtFirstReq} > 0`);
// Check that request and response share their socket.
assert.strictEqual(r1.socket, socket);

res.on('data', common.mustCallAtLeast(() => {}));
res.on('end', common.mustCall(() => {
// setImmediate() to give the agent time to register the freed socket.
setImmediate(common.mustCall(() => {
// The socket is free for reuse now.
assert.strictEqual(socket[async_id_symbol], -1);

// Second request. To re-create the exact conditions from the
// referenced issue, we use a POST request without chunked encoding
// (hence the Content-Length header) and call .end() after the
// response header has already been received.
const r2 = http.request({
agent, port, method: 'POST', headers: {
'Content-Length': payload.length
}
}, common.mustCall((res) => {
asyncIdAtSecondReq = res.socket[async_id_symbol];
assert.ok(asyncIdAtSecondReq > 0, `${asyncIdAtSecondReq} > 0`);
assert.strictEqual(r2.socket, socket);

// Empty payload, to hit the “right” code path.
r2.end('');

res.on('data', common.mustCallAtLeast(() => {}));
res.on('end', common.mustCall(() => {
// Clean up to let the event loop stop.
server.close();
agent.destroy();
}));
}));

// Schedule a payload to be written immediately, but do not end the
// request just yet.
r2.write(payload);
}));
}));
}));
r1.end(payload);
}));


process.on('exit', onExit);

function onExit() {
hooks.disable();
hooks.sanityCheck();
const activities = hooks.activities;

// Verify both invocations
const first = activities.filter((x) => x.uid === asyncIdAtFirstReq)[0];
checkInvocations(first, { init: 1, destroy: 1 }, 'when process exits');

const second = activities.filter((x) => x.uid === asyncIdAtSecondReq)[0];
checkInvocations(second, { init: 1, destroy: 1 }, 'when process exits');

// Verify reuse handle has been wrapped
assert.strictEqual(first.type, second.type);
assert.ok(first.handle !== second.handle, 'Resource reused');
Flarna marked this conversation as resolved.
Show resolved Hide resolved
assert.ok(first.handle === second.handle.handle,
Flarna marked this conversation as resolved.
Show resolved Hide resolved
'Resource not wrapped correctly');
}