Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(ws server): fix shutdown on connection closed #1103

Merged
merged 18 commits into from
Apr 27, 2023

Conversation

niklasad1
Copy link
Member

No description provided.

@niklasad1 niklasad1 requested a review from a team as a code owner April 26, 2023 11:43
server/src/transport/ws.rs Outdated Show resolved Hide resolved
server/src/transport/ws.rs Outdated Show resolved Hide resolved
});

let pending = pending_calls.for_each(|_| async {});
let disconnect = disconnect_stream.for_each(|_| async {});
Copy link
Member Author

@niklasad1 niklasad1 Apr 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's possible that the connection is terminated during the graceful shutdown and this ensures that it is aborted once the close message is sent.

however, this is slightly error prone if the no proper close is sent but yeah would neat if soketto had something to indicate when the connection is closed

server/src/tests/ws.rs Outdated Show resolved Hide resolved
@@ -272,11 +272,11 @@ pub(crate) async fn background_task<L: Logger>(
stopped = stop;
permit
}
None => break Ok(()),
None => break Ok(Shutdown::ConnectionClosed),
Copy link
Collaborator

@jsdw jsdw Apr 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Am I right in thinking that:

  • Stopped means the user has manually stopped the server, so we want to gracefully close the eonnction
  • ConnectionClosed means the connection was closed for some other reason (eg network issue or whatever)

Copy link
Member Author

@niklasad1 niklasad1 Apr 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, this is only fails if the send_task has been completed and the receiver of the bounded channel has been dropped.

this can only occur if the send_ping fails I think i.e, connection closed

Comment on lines 351 to 354
tokio::select! {
_ = pending => (),
_ = disconnect => (),
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So basically, whatever finishes first out of disconnect (which looks like any final things to be received?) and pending (whihc looks like anything to be sent back?) will lead to the thing ending?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find it a bit hard to follow the logic here!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, it's way to make it possible to detect whether connection has terminated while we try to wait for the pending futures to complete

server/src/transport/ws.rs Outdated Show resolved Hide resolved
server/src/transport/ws.rs Outdated Show resolved Hide resolved
server/src/transport/ws.rs Outdated Show resolved Hide resolved
server/src/transport/ws.rs Outdated Show resolved Hide resolved
server/src/tests/ws.rs Outdated Show resolved Hide resolved
client.send(req).with_default_timeout().await.unwrap().unwrap();
}
// Assumption: the calls would be ACK:ed by server after 10 seconds (no way knowing that except parsing CLI output)
tokio::time::sleep(std::time::Duration::from_secs(10)).await;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dq: I remember from the substrate CI that these constants could be exceeded on overcommited days. Should we increase the timeout a bit here, or this should be sufficient?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know what I was thinking, I could embed mpsc::Sender in the method callback and await for 10 messages

Nice to get rid of sleeps, those shouldn't be used if possible :D

@niklasad1
Copy link
Member Author

@jsdw can you have another look if it's easier to read now? Did some minor refactoring

@niklasad1 niklasad1 merged commit 457d2d2 into master Apr 27, 2023
@niklasad1 niklasad1 deleted the na-fix-flaky-ws-conn-closed-test branch April 27, 2023 12:51
niklasad1 added a commit that referenced this pull request Apr 27, 2023
niklasad1 added a commit that referenced this pull request Apr 27, 2023
* chore: release v0.18.1

* fix(ws server): fix flaky shutdown test

* adjust changelog for #1103

* Update CHANGELOG.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants