HTTP/2 stress test TaskCanceledException when client hasn't cancelled

[ManickaP]

When a server receives DATA frame after RST (known issue #1510, probably caused by the same problem as #42200) it makes the connection faulted and sends GO_AWAY. On the client, the connections is getting closed, all the streams getting reset, triggering Http2Stream._requestBodyCancellationSource. This CTS will cancel ongoing request body with TaskCancelledException eventually propagating out of SendAsync. This is magnified by #38774 that tried to propagate errors from faulty request content to the user, therefore is mostly visible in POST ExpectContinue stress test operation.

[ghost]

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

[geoffkizer]

Yeah, this is ugly. We should be reporting some sort of exception here that indicates the server sent us a GOAWAY that killed these requests. Otherwise it's super confusing.

[BrennanConroy]

I believe there is another case where you can get System.Threading.Tasks.TaskCanceledException : The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing. unexpectedly.

The client sends the headers first, after that it kicks off two tasks, one for sending the data frame and one for reading response headers. An RST_FRAME can be received by the client due to a stream limit being hit on the server which should cause a retry but ReadResponseHeadersAsync will call Cancel() which cancels _requestBodyCancellationSource which will cause the previously mentioned data frame task to fail. This will race with the reading response headers task and you'll get either a TaskCanceledException which will cause the retry to not happen, or you'll get a HttpRequestException which will allow a retry.

[karelz]

@alnikola is it similar to #42200 as mentioned above?

[alnikola]

The root cause of this issue is the same as #42200, but it seems this is more about a proper error reporting than fixing the root cause itself.