Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Quarantine GOAWAY_Received_SetsConnectionStateToClosingAndWaitForAllStreamsToComplete #39520

Closed
wtgodbe opened this issue Jan 14, 2022 · 3 comments
Closed

Quarantine GOAWAY_Received_SetsConnectionStateToClosingAndWaitForAllStreamsToComplete #39520

wtgodbe opened this issue Jan 14, 2022 · 3 comments
Labels
area-networking Includes servers, yarp, json patch, bedrock, websockets, http client factory, and http abstractions test-failure test-fixed
Milestone
Test failures

Comments

@wtgodbe
Copy link
Member

wtgodbe commented Jan 14, 2022

Failing Test(s)

Microsoft.AspNetCore.Server.Kestrel.Core.Tests.Http2ConnectionTests.GOAWAY_Received_SetsConnectionStateToClosingAndWaitForAllStreamsToComplete

Error Message

System.TimeoutException : The operation has timed out.

Stacktrace

   at Microsoft.AspNetCore.Testing.TaskExtensions.TimeoutAfter[T](Task`1 task, TimeSpan timeout, String filePath, Int32 lineNumber) in /_/src/Shared/TaskExtensions.cs:line 108
   at Microsoft.AspNetCore.Server.Kestrel.Core.Tests.Http2TestBase.ReceiveFrameAsync(UInt32 maxFrameSize) in /_/src/Servers/Kestrel/test/InMemory.FunctionalTests/Http2/Http2TestBase.cs:line 1139
   at Microsoft.AspNetCore.Server.Kestrel.Core.Tests.Http2TestBase.WaitForConnectionErrorAsyncDoNotCloseTransport[TException](Boolean ignoreNonGoAwayFrames, Int32 expectedLastStreamId, Http2ErrorCode expectedErrorCode, String[] expectedErrorMessage) in /_/src/Servers/Kestrel/test/InMemory.FunctionalTests/Http2/Http2TestBase.cs:line 1217
   at Microsoft.AspNetCore.Server.Kestrel.Core.Tests.Http2TestBase.WaitForConnectionErrorAsync[TException](Boolean ignoreNonGoAwayFrames, Int32 expectedLastStreamId, Http2ErrorCode expectedErrorCode, String[] expectedErrorMessage) in /_/src/Servers/Kestrel/test/InMemory.FunctionalTests/Http2/Http2TestBase.cs:line 1212
   at Microsoft.AspNetCore.Server.Kestrel.Core.Tests.Http2ConnectionTests.GOAWAY_Received_SetsConnectionStateToClosingAndWaitForAllStreamsToComplete() in /_/src/Servers/Kestrel/test/InMemory.FunctionalTests/Http2/Http2ConnectionTests.cs:line 3738
--- End of stack trace from previous location ---

Logs

[0.001s] [TestLifetime] [Information] Starting test GOAWAY_Received_SetsConnectionStateToClosingAndWaitForAllStreamsToComplete at 2022-01-14T05:40:40
[0.003s] [Microsoft.AspNetCore.Server.Kestrel.Http2] [Verbose] Connection id "TestConnectionId" sending SETTINGS frame for stream ID 0 with length 18 and flags NONE.
[0.004s] [Microsoft.AspNetCore.Server.Kestrel.Http2] [Verbose] Connection id "TestConnectionId" sending WINDOW_UPDATE frame for stream ID 0 with length 4 and flags 0x0.
[0.004s] [Microsoft.AspNetCore.Server.Kestrel.Http2] [Verbose] Connection id "TestConnectionId" received SETTINGS frame for stream ID 0 with length 0 and flags NONE.
[0.004s] [Microsoft.AspNetCore.Server.Kestrel.Http2] [Verbose] Connection id "TestConnectionId" sending SETTINGS frame for stream ID 0 with length 0 and flags ACK.
[0.004s] [Microsoft.AspNetCore.Server.Kestrel.Http2] [Verbose] Connection id "TestConnectionId" received HEADERS frame for stream ID 1 with length 318 and flags END_HEADERS.
[0.005s] [Microsoft.AspNetCore.Server.Kestrel] [Debug] Connection id "TestConnectionId", Request id "TestConnectionId:00000001": started reading request body.
[0.005s] [Microsoft.AspNetCore.Server.Kestrel.Http2] [Verbose] Connection id "TestConnectionId" received HEADERS frame for stream ID 3 with length 9 and flags END_HEADERS.
[0.005s] [Microsoft.AspNetCore.Server.Kestrel] [Debug] Connection id "TestConnectionId", Request id "TestConnectionId:00000003": started reading request body.
[0.005s] [Microsoft.AspNetCore.Server.Kestrel.Http2] [Verbose] Connection id "TestConnectionId" received GOAWAY frame for stream ID 0 with length 8 and flags 0x0.
[0.006s] [Microsoft.AspNetCore.Server.Kestrel.Http2] [Debug] Connection id "TestConnectionId" is closing.
[0.006s] [Microsoft.AspNetCore.Server.Kestrel.Http2] [Verbose] Connection id "TestConnectionId" received DATA frame for stream ID 1 with length 5 and flags END_STREAM.
[0.006s] [Microsoft.AspNetCore.Server.Kestrel] [Debug] Connection id "TestConnectionId", Request id "TestConnectionId:00000001": done reading request body.
[0.010s] [Microsoft.AspNetCore.Server.Kestrel.Http2] [Verbose] Connection id "TestConnectionId" sending HEADERS frame for stream ID 1 with length 32 and flags END_HEADERS.
[0.010s] [Microsoft.AspNetCore.Server.Kestrel.Http2] [Verbose] Connection id "TestConnectionId" sending DATA frame for stream ID 1 with length 5 and flags NONE.
[0.016s] [Microsoft.AspNetCore.Server.Kestrel.Http2] [Verbose] Connection id "TestConnectionId" sending DATA frame for stream ID 1 with length 0 and flags END_STREAM.
[0.019s] [Microsoft.AspNetCore.Server.Kestrel.Http2] [Verbose] Connection id "TestConnectionId" received DATA frame for stream ID 3 with length 5 and flags END_STREAM.
[0.019s] [Microsoft.AspNetCore.Server.Kestrel] [Debug] Connection id "TestConnectionId", Request id "TestConnectionId:00000003": done reading request body.
[0.020s] [Microsoft.AspNetCore.Server.Kestrel.Http2] [Verbose] Connection id "TestConnectionId" sending HEADERS frame for stream ID 3 with length 2 and flags END_HEADERS.
[0.020s] [Microsoft.AspNetCore.Server.Kestrel.Http2] [Verbose] Connection id "TestConnectionId" sending DATA frame for stream ID 3 with length 5 and flags NONE.
[0.021s] [Microsoft.AspNetCore.Server.Kestrel.Http2] [Verbose] Connection id "TestConnectionId" sending DATA frame for stream ID 3 with length 0 and flags END_STREAM.
[30.026s] [Microsoft.AspNetCore.Server.Kestrel.Core.Tests.Http2ConnectionTests] [Error] Test threw an exception.
System.TimeoutException: The operation has timed out.
   at Microsoft.AspNetCore.Testing.TaskExtensions.TimeoutAfter[T](Task`1 task, TimeSpan timeout, String filePath, Int32 lineNumber) in /_/src/Shared/TaskExtensions.cs:line 108
   at Microsoft.AspNetCore.Server.Kestrel.Core.Tests.Http2TestBase.ReceiveFrameAsync(UInt32 maxFrameSize) in /_/src/Servers/Kestrel/test/InMemory.FunctionalTests/Http2/Http2TestBase.cs:line 1139
   at Microsoft.AspNetCore.Server.Kestrel.Core.Tests.Http2TestBase.WaitForConnectionErrorAsyncDoNotCloseTransport[TException](Boolean ignoreNonGoAwayFrames, Int32 expectedLastStreamId, Http2ErrorCode expectedErrorCode, String[] expectedErrorMessage) in /_/src/Servers/Kestrel/test/InMemory.FunctionalTests/Http2/Http2TestBase.cs:line 1217
   at Microsoft.AspNetCore.Server.Kestrel.Core.Tests.Http2TestBase.WaitForConnectionErrorAsync[TException](Boolean ignoreNonGoAwayFrames, Int32 expectedLastStreamId, Http2ErrorCode expectedErrorCode, String[] expectedErrorMessage) in /_/src/Servers/Kestrel/test/InMemory.FunctionalTests/Http2/Http2TestBase.cs:line 1212
   at Microsoft.AspNetCore.Server.Kestrel.Core.Tests.Http2ConnectionTests.GOAWAY_Received_SetsConnectionStateToClosingAndWaitForAllStreamsToComplete() in /_/src/Servers/Kestrel/test/InMemory.FunctionalTests/Http2/Http2ConnectionTests.cs:line 3738
   at Xunit.Sdk.TestInvoker`1.<>c__DisplayClass48_1.<<InvokeTestMethodAsync>b__1>d.MoveNext() in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\Runners\TestInvoker.cs:line 273
--- End of stack trace from previous location ---
   at Xunit.Sdk.ExecutionTimer.AggregateAsync(Func`1 asyncAction) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\ExecutionTimer.cs:line 54
   at Xunit.Sdk.ExceptionAggregator.RunAsync(Func`1 code) in C:\Dev\xunit\xunit\src\xunit.core\Sdk\ExceptionAggregator.cs:line 96
[30.027s] [Microsoft.AspNetCore.Server.Kestrel.Http2] [Debug] Connection id "TestConnectionId" is closed. The last processed stream ID was 3.
[30.027s] [Microsoft.AspNetCore.Server.Kestrel.Http2] [Verbose] Connection id "TestConnectionId" sending GOAWAY frame for stream ID 0 with length 8 and flags 0x0.
[30.028s] [TestLifetime] [Information] Finished test GOAWAY_Received_SetsConnectionStateToClosingAndWaitForAllStreamsToComplete in 30.0267825s

Build

https://dev.azure.com/dnceng/public/_build/results?buildId=1552509&view=results
@wtgodbe wtgodbe changed the title Quarantine <FAILING_TEST_NAME> Quarantine GOAWAY_Received_SetsConnectionStateToClosingAndWaitForAllStreamsToComplete Jan 14, 2022
@wtgodbe wtgodbe mentioned this issue Jan 14, 2022
Merged
@TanayParikh
Copy link
Contributor

Likely related to #39492.

@adityamandaleeka adityamandaleeka added this to the Test failures milestone Jan 14, 2022
@adityamandaleeka
Copy link
Member

@JamesNK It appears this started happening on your PR #39358 and has been failing occasionally since then. Can you please take a look?

@JamesNK JamesNK mentioned this issue Jan 17, 2022
Merged
@JamesNK JamesNK closed this as completed Jan 19, 2022
@TanayParikh
Copy link
Contributor

I believe we'd like to keep these issues open to ensure the QuarantinedTest attribute is removed after 30 days.

@TanayParikh TanayParikh reopened this Jan 19, 2022
@HaoK HaoK assigned HaoK and unassigned JamesNK Feb 25, 2022
@HaoK HaoK closed this as completed Feb 25, 2022
@HaoK HaoK removed their assignment Feb 25, 2022
@ghost ghost locked as resolved and limited conversation to collaborators Mar 27, 2022
@amcasey amcasey added area-networking Includes servers, yarp, json patch, bedrock, websockets, http client factory, and http abstractions and removed area-runtime labels Aug 24, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
area-networking Includes servers, yarp, json patch, bedrock, websockets, http client factory, and http abstractions test-failure test-fixed
Projects
None yet
Milestone
Test failures
Development

No branches or pull requests
6 participants
@adityamandaleeka @JamesNK @HaoK @amcasey @wtgodbe @TanayParikh