Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[QUIC] Stream write cancellation #53304

Merged
merged 8 commits into from
Jun 4, 2021

Conversation

CarnaViire
Copy link
Member

Created a test to verify write cancellation behavior. Fixed MockStream to also work as expected. Added throwing on msquic returning write canceled status.

Fixes #32077

@ghost
Copy link

ghost commented May 26, 2021

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

Created a test to verify write cancellation behavior. Fixed MockStream to also work as expected. Added throwing on msquic returning write canceled status.

Fixes #32077

Author: CarnaViire
Assignees: -
Labels:

area-System.Net.Quic

Milestone: -


CancellationTokenSource cts = new CancellationTokenSource();
var task = stream.WriteAsync(new byte[1024 * 1024], cts.Token);
cts.Cancel();
Copy link
Member Author

@CarnaViire CarnaViire May 27, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This way of testing cancellation is not reliable. Sometimes it seems that task finishes before cancellation occurs. Does anyone have an idea on how to make sure write is not over before cancellation? Pre-cancelling before calling WriteAsync is not appealing to me...

Copy link
Member

@ManickaP ManickaP May 27, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can only think of ways which touch MsQuicStream code so no help, sorry.
But I think that adding extra test for pre-cancellation is actually a good idea. I'm looking at the code and it looks like we will send the data even if the token has been cancelled beforehand. Is that expected behavior? Should we do some best-effort token check before we call into msquic? Or did I just overlook something 😊

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sending 64M instead of 1M helped, at least I didn't get any failures on my setup while running it ~20 times. If someone believes it is not reliable enough, the only option I see here is to remove this test completely. I've added a second one that checks pre-cancelled token.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rewrote to an infinite loop until canceled, as per @geoffkizer suggestion

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could that infinite loop be somehow finite, but still big enough for the test? I already see hanging tests in CI :)

@CarnaViire
Copy link
Member Author

Pre-cancellation indeed doesn't work as expected (for several reasons). Will change this PR to draft until I come up with a fix.

@CarnaViire CarnaViire marked this pull request as draft June 1, 2021 12:18
@CarnaViire CarnaViire marked this pull request as ready for review June 1, 2021 14:34
{
throw GetConnectionAbortedException(_state);
}
await _state.SendResettableCompletionSource.GetTypelessValueTask().ConfigureAwait(false);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I don't wait for it before creating CancellationTokenRegistration, in case of pre-cancelled token, SendResettableCompletionSource.CompleteException fails with InvalidOperationException: Operation is not valid due to the current state of the object. 🤔

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, we guard the ResettableCompletionSource completion with SendState, but it "breaks" here since we're (ab)using SendResettableCompletionSource for START_COMPLETE as well.
I don't have any better suggestions than what you did here.

}

// a write would eventually be canceled
await Assert.ThrowsAsync<OperationCanceledException>(() => WriteUntilCanceled());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we call WriteUntilCanceled().WaitAsync(DefaultTestTimeout) or something similar to ensure that if this does fail we get a proper exception instead of a hanging test? We'd have to be careful the ThrowsAsync doesn't swallow the test timeout.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will try that, or making a loop big enough but finite, as @ManickaP suggested, and see what will work best

Copy link
Member

@ManickaP ManickaP left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.
I'd still prefer if somebody else (@scalablecory @geoffkizer) give this look as well though.

{
throw GetConnectionAbortedException(_state);
}
await _state.SendResettableCompletionSource.GetTypelessValueTask().ConfigureAwait(false);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, we guard the ResettableCompletionSource completion with SendState, but it "breaks" here since we're (ab)using SendResettableCompletionSource for START_COMPLETE as well.
I don't have any better suggestions than what you did here.


CancellationTokenSource cts = new CancellationTokenSource();
var task = stream.WriteAsync(new byte[1024 * 1024], cts.Token);
cts.Cancel();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could that infinite loop be somehow finite, but still big enough for the test? I already see hanging tests in CI :)

@CarnaViire
Copy link
Member Author

@scalablecory @geoffkizer will you take a look or shall I merge?

@CarnaViire CarnaViire merged commit e0671e7 into dotnet:main Jun 4, 2021
@CarnaViire CarnaViire deleted the stream-cancellation branch June 4, 2021 14:32
@ghost ghost locked as resolved and limited conversation to collaborators Jul 4, 2021
@karelz karelz added this to the 6.0.0 milestone Jul 15, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[QUIC] Support cancellation of QUIC writes
4 participants