Fixed outerloop tests for Sync HttpClient API #38453

ManickaP · 2020-06-26T16:38:02Z

Extended and added some timeouts to mitigate failing tests on slow infra.
Failures:
https://dev.azure.com/dnceng/public/_build/results?buildId=706219&view=results
https://dev.azure.com/dnceng/public/_build/results?buildId=706263&view=results

Fixes #38497

Dotnet-GitSync-Bot · 2020-06-26T16:38:06Z

I couldn't figure out the best area label to add to this PR. If you have write-permissions please help me learn by adding exactly one area label.

ghost · 2020-06-26T16:38:33Z

Tagging subscribers to this area: @dotnet/ncl
Notify danmosemsft if you want to be subscribed.

ManickaP · 2020-06-26T16:42:54Z

/azp run runtime-libraries outerloop

azure-pipelines · 2020-06-26T16:43:07Z

Azure Pipelines successfully started running 1 pipeline(s).

wfurt · 2020-06-26T17:57:03Z

src/libraries/System.Net.Http/tests/FunctionalTests/HttpClientTest.cs

@@ -833,6 +833,7 @@ public async Task Send_CancelledRequestContent_Throws()
                {
                    var sendTask = Task.Run(() => {
                        using HttpClient httpClient = CreateHttpClient();
+                        httpClient.Timeout = TimeSpan.FromMinutes(5);


Should this be constant somewhere? 5 minus seems really long. Is there evidence that the fails only on particular platform? If so, one-fits-all can be perhaps improved but if this is only upper boundary it may be ok.

We distinguish timeouts from cancellations through measuring how long the request took before it failed.
https://github.com/dotnet/runtime/blob/master/src/libraries/System.Net.Http/src/System/Net/Http/HttpClient.cs#L589

The error I was seeing was that the cancellation processing took so long that it was mistaken for a timeout, against which I'm asserting few lines bellow.

So this is just to guard against false positive timeouts.

It's most likely platform independent problem (seeing Windows and OSX in the referenced failures).

Lastly, I just searched our tests and I haven't found a single occurrence of HttpClient.Timeout being set with a defined constant 🤣 . All of it are direct values like it's here.

This feels a lot like we've got an actual bug somewhere -- be it in our code or in the CI system -- and we're covering up for it. It should not take 5 minutes for a cancelled token to trigger the socket to dispose.

Can timeout be lower? 1 minute seems like plenty.

If you look at the testResults.xml, the failed test took almost 2 minutes (method="Send_CancelledRequestContent_Throws" time="105.6715878").

Also if you look around the file on times of other tests, you'll see similar times elsewhere:

<test name="System.Net.Http.Functional.Tests.SyncHttpHandler_HttpClientHandler_Cancellation_Test.Expect100Continue_WaitsExpectedPeriodOfTimeBeforeSendingContent" type="System.Net.Http.Functional.Tests.SyncHttpHandler_HttpClientHandler_Cancellation_Test" method="Expect100Continue_WaitsExpectedPeriodOfTimeBeforeSendingContent" time="36.5480668" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_Proxy_Test.Proxy_SendSecureRequestThruProxy_ConnectTunnelUsed" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_Proxy_Test" method="Proxy_SendSecureRequestThruProxy_ConnectTunnelUsed" time="35.0460034" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_Proxy_Test.Proxy_BypassTrue_GetRequestDoesntGoesThroughCustomProxy(proxy: null)" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_Proxy_Test" method="Proxy_BypassTrue_GetRequestDoesntGoesThroughCustomProxy" time="10.8030974" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_ConnectionPooling_Test.ServerDisconnectsAfterInitialRequest_SubsequentRequestUsesDifferentConnection" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_ConnectionPooling_Test" method="ServerDisconnectsAfterInitialRequest_SubsequentRequestUsesDifferentConnection" time="38.0609817" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_HttpClientHandlerTest_Headers_Http2.SendAsync_GetWithValidHostHeader_Success(withPort: True)" type="System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_HttpClientHandlerTest_Headers_Http2" method="SendAsync_GetWithValidHostHeader_Success" time="15.9222679" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandlerTest.SendAsync_SendRequestUsingNoBodyMethodToEchoServerWithContent_NoBodySent(method: \"HEAD\", serverUri: http://corefx-net-http11.azurewebsites.net/Echo.ashx)" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandlerTest" method="SendAsync_SendRequestUsingNoBodyMethodToEchoServerWithContent_NoBodySent" time="35.262461" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandlerTest.SendAsync_SendRequestUsingNoBodyMethodToEchoServerWithContent_NoBodySent(method: \"HEAD\", serverUri: https://corefx-net-http11.azurewebsites.net/Echo.ashx)" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandlerTest" method="SendAsync_SendRequestUsingNoBodyMethodToEchoServerWithContent_NoBodySent" time="15.9431951" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandlerTest.SendAsync_SendRequestUsingNoBodyMethodToEchoServerWithContent_NoBodySent(method: \"HEAD\", serverUri: https://corefx-net-http2.azurewebsites.net/Echo.ashx)" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandlerTest" method="SendAsync_SendRequestUsingNoBodyMethodToEchoServerWithContent_NoBodySent" time="10.0093018" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_DefaultProxyCredentials_Test.ProxyNotExplicitlyProvided_DefaultCredentialsSet_DefaultWebProxySetToNull_Success" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_DefaultProxyCredentials_Test" method="ProxyNotExplicitlyProvided_DefaultCredentialsSet_DefaultWebProxySetToNull_Success" time="35.0168628" result="Pass"> <test name="System.Net.Http.Functional.Tests.NtAuthTests.PostAsync_NtAuthServer_UseExpect100Header_Success(ntlm: True, contentSize: 1023)" type="System.Net.Http.Functional.Tests.NtAuthTests" method="PostAsync_NtAuthServer_UseExpect100Header_Success" time="35.3466227" result="Pass"> <test name="System.Net.Http.Functional.Tests.NtAuthTests.PostAsync_NtAuthServer_UseExpect100Header_Success(ntlm: True, contentSize: 1024)" type="System.Net.Http.Functional.Tests.NtAuthTests" method="PostAsync_NtAuthServer_UseExpect100Header_Success" time="23.0162003" result="Pass"> <test name="System.Net.Http.Functional.Tests.NtAuthTests.PostAsync_NtAuthServer_UseExpect100Header_Success(ntlm: True, contentSize: 1025)" type="System.Net.Http.Functional.Tests.NtAuthTests" method="PostAsync_NtAuthServer_UseExpect100Header_Success" time="16.7852631" result="Pass"> <test name="System.Net.Http.Functional.Tests.SyncHttpHandlerTest_HttpClientHandlerTest_Headers.SendAsync_GetWithValidHostHeader_Success(withPort: False)" type="System.Net.Http.Functional.Tests.SyncHttpHandlerTest_HttpClientHandlerTest_Headers" method="SendAsync_GetWithValidHostHeader_Success" time="23.6571878" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_DangerousAcceptAllCertificatesValidator_Test.InvalidCertificateServers_CertificateValidationDisabled_Succeeds(url: \"https://expired.badssl.com/\")" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_DangerousAcceptAllCertificatesValidator_Test" method="InvalidCertificateServers_CertificateValidationDisabled_Succeeds" time="22.9605599" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_DangerousAcceptAllCertificatesValidator_Test.InvalidCertificateServers_CertificateValidationDisabled_Succeeds(url: \"https://self-signed.badssl.com/\")" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_DangerousAcceptAllCertificatesValidator_Test" method="InvalidCertificateServers_CertificateValidationDisabled_Succeeds" time="17.5470465" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_DangerousAcceptAllCertificatesValidator_Test.InvalidCertificateServers_CertificateValidationDisabled_Succeeds(url: \"https://wrong.host.badssl.com/\")" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_DangerousAcceptAllCertificatesValidator_Test" method="InvalidCertificateServers_CertificateValidationDisabled_Succeeds" time="10.0228789" result="Pass"> <test name="System.Net.Http.Functional.Tests.SyncHttpHandlerTest_AutoRedirect.GetAsync_CredentialIsNetworkCredentialUriRedirect_StatusCodeUnauthorized(remoteServer: (BaseUri: http://corefx-net-http11.azurewebsites.net/, HttpVersion: 1.1))" type="System.Net.Http.Functional.Tests.SyncHttpHandlerTest_AutoRedirect" method="GetAsync_CredentialIsNetworkCredentialUriRedirect_StatusCodeUnauthorized" time="14.1564323" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpProtocolTests_Dribble.GetAsync_Chunked_VaryingSizeChunks_ReceivedCorrectly(maxChunkSize: 1, lineEnding: \"\\n\", useCopyToAsync: False)" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpProtocolTests_Dribble" method="GetAsync_Chunked_VaryingSizeChunks_ReceivedCorrectly" time="36.3829722" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpProtocolTests_Dribble.GetAsync_Chunked_VaryingSizeChunks_ReceivedCorrectly(maxChunkSize: 1, lineEnding: \"\\n\", useCopyToAsync: True)" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpProtocolTests_Dribble" method="GetAsync_Chunked_VaryingSizeChunks_ReceivedCorrectly" time="10.6521162" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpProtocolTests_Dribble.GetAsync_Chunked_VaryingSizeChunks_ReceivedCorrectly(maxChunkSize: 1, lineEnding: \"\\r\\n\", useCopyToAsync: False)" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpProtocolTests_Dribble" method="GetAsync_Chunked_VaryingSizeChunks_ReceivedCorrectly" time="13.9719459" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpProtocolTests_Dribble.GetAsync_Chunked_VaryingSizeChunks_ReceivedCorrectly(maxChunkSize: 1, lineEnding: \"\\r\\n\", useCopyToAsync: True)" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpProtocolTests_Dribble" method="GetAsync_Chunked_VaryingSizeChunks_ReceivedCorrectly" time="15.7977029" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_MaxResponseHeadersLength_Test.ThresholdExceeded_ThrowsException(responseHeaders: \"HTTP/1.1 200 OK\\r\\nContent-Length: 0\\r\\nCustom-000\"..., maxResponseHeadersLength: null, shouldSucceed: True)" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_MaxResponseHeadersLength_Test" method="ThresholdExceeded_ThrowsException" time="35.374156" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_MaxResponseHeadersLength_Test.ThresholdExceeded_ThrowsException(responseHeaders: \"HTTP/1.1 200 OK\\r\\nContent-Length: 0\\r\\nCustom-000\"..., maxResponseHeadersLength: null, shouldSucceed: True)" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_MaxResponseHeadersLength_Test" method="ThresholdExceeded_ThrowsException" time="10.4789942" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_SslProtocols_Test.GetAsync_SupportedSSLVersion_Succeeds(sslProtocols: Tls, url: \"https://www.ssllabs.com:10301/\")" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_SslProtocols_Test" method="GetAsync_SupportedSSLVersion_Succeeds" time="16.0482474" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_SslProtocols_Test.GetAsync_SupportedSSLVersion_Succeeds(sslProtocols: Tls12, url: \"https://www.ssllabs.com:10303/\")" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_SslProtocols_Test" method="GetAsync_SupportedSSLVersion_Succeeds" time="10.4444674" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpProtocolTests.GetAsync_Chunked_VaryingSizeChunks_ReceivedCorrectly(maxChunkSize: 1, lineEnding: \"\\n\", useCopyToAsync: False)" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpProtocolTests" method="GetAsync_Chunked_VaryingSizeChunks_ReceivedCorrectly" time="35.1213315" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpProtocolTests.GetAsync_Chunked_VaryingSizeChunks_ReceivedCorrectly(maxChunkSize: 1, lineEnding: \"\\n\", useCopyToAsync: True)" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpProtocolTests" method="GetAsync_Chunked_VaryingSizeChunks_ReceivedCorrectly" time="10.7895331" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpProtocolTests.GetAsync_Chunked_VaryingSizeChunks_ReceivedCorrectly(maxChunkSize: 1, lineEnding: \"\\r\\n\", useCopyToAsync: False)" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpProtocolTests" method="GetAsync_Chunked_VaryingSizeChunks_ReceivedCorrectly" time="19.5267953" result="Pass"> <test name="System.Net.Http.Functional.Tests.SyncHttpHandler_HttpClientHandlerTest.GetAsync_ResponseHeadersRead_ReadFromEachIterativelyDoesntDeadlock(remoteServer: (BaseUri: http://corefx-net-http11.azurewebsites.net/, HttpVersion: 1.1))" type="System.Net.Http.Functional.Tests.SyncHttpHandler_HttpClientHandlerTest" method="GetAsync_ResponseHeadersRead_ReadFromEachIterativelyDoesntDeadlock" time="23.7963713" result="Pass"> <test name="System.Net.Http.Functional.Tests.SyncHttpHandler_HttpClientHandlerTest.GetAsync_ResponseHeadersRead_ReadFromEachIterativelyDoesntDeadlock(remoteServer: (BaseUri: https://corefx-net-http11.azurewebsites.net/, HttpVersion: 1.1))" type="System.Net.Http.Functional.Tests.SyncHttpHandler_HttpClientHandlerTest" method="GetAsync_ResponseHeadersRead_ReadFromEachIterativelyDoesntDeadlock" time="16.0797817" result="Pass"> <test name="System.Net.Http.Functional.Tests.SyncHttpHandler_HttpClientHandlerTest.GetAsync_ResponseHeadersRead_ReadFromEachIterativelyDoesntDeadlock(remoteServer: (BaseUri: https://corefx-net-http2.azurewebsites.net/, HttpVersion: 2.0))" type="System.Net.Http.Functional.Tests.SyncHttpHandler_HttpClientHandlerTest" method="GetAsync_ResponseHeadersRead_ReadFromEachIterativelyDoesntDeadlock" time="11.2258624" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_AutoRedirect.GetAsync_CredentialIsCredentialCacheUriRedirect_StatusCodeOK(remoteServer: (BaseUri: http://corefx-net-http11.azurewebsites.net/, HttpVersion: 1.1), statusCode: 300)" type="System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_AutoRedirect" method="GetAsync_CredentialIsCredentialCacheUriRedirect_StatusCodeOK" time="23.8443341" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_AutoRedirect.GetAsync_CredentialIsCredentialCacheUriRedirect_StatusCodeOK(remoteServer: (BaseUri: http://corefx-net-http11.azurewebsites.net/, HttpVersion: 1.1), statusCode: 301)" type="System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_AutoRedirect" method="GetAsync_CredentialIsCredentialCacheUriRedirect_StatusCodeOK" time="15.9792189" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_AutoRedirect.GetAsync_CredentialIsCredentialCacheUriRedirect_StatusCodeOK(remoteServer: (BaseUri: http://corefx-net-http11.azurewebsites.net/, HttpVersion: 1.1), statusCode: 303)" type="System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_AutoRedirect" method="GetAsync_CredentialIsCredentialCacheUriRedirect_StatusCodeOK" time="10.4584034" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_HttpClientHandlerTest_Http2.PostAsync_CallMethod_UnicodeStringContent(remoteServer: (BaseUri: http://corefx-net-http11.azurewebsites.net/, HttpVersion: 1.1))" type="System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_HttpClientHandlerTest_Http2" method="PostAsync_CallMethod_UnicodeStringContent" time="23.7853672" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_HttpClientHandlerTest_Http2.PostAsync_CallMethod_UnicodeStringContent(remoteServer: (BaseUri: https://corefx-net-http11.azurewebsites.net/, HttpVersion: 1.1))" type="System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_HttpClientHandlerTest_Http2" method="PostAsync_CallMethod_UnicodeStringContent" time="16.0752092" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_HttpClientHandlerTest_Http2.PostAsync_CallMethod_UnicodeStringContent(remoteServer: (BaseUri: https://corefx-net-http2.azurewebsites.net/, HttpVersion: 2.0))" type="System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_HttpClientHandlerTest_Http2" method="PostAsync_CallMethod_UnicodeStringContent" time="11.2122835" result="Pass"> <test name="System.Net.Http.Functional.Tests.SyncHttpHandler_HttpProtocolTests_Dribble.GetAsync_Chunked_VaryingSizeChunks_ReceivedCorrectly(maxChunkSize: 1, lineEnding: \"\\n\", useCopyToAsync: False)" type="System.Net.Http.Functional.Tests.SyncHttpHandler_HttpProtocolTests_Dribble" method="GetAsync_Chunked_VaryingSizeChunks_ReceivedCorrectly" time="35.3607151" result="Pass"> <test name="System.Net.Http.Functional.Tests.SyncHttpHandler_HttpProtocolTests_Dribble.GetAsync_Chunked_VaryingSizeChunks_ReceivedCorrectly(maxChunkSize: 1, lineEnding: \"\\n\", useCopyToAsync: True)" type="System.Net.Http.Functional.Tests.SyncHttpHandler_HttpProtocolTests_Dribble" method="GetAsync_Chunked_VaryingSizeChunks_ReceivedCorrectly" time="10.7854075" result="Pass"> <test name="System.Net.Http.Functional.Tests.SyncHttpHandler_HttpProtocolTests_Dribble.GetAsync_Chunked_VaryingSizeChunks_ReceivedCorrectly(maxChunkSize: 1, lineEnding: \"\\r\\n\", useCopyToAsync: False)" type="System.Net.Http.Functional.Tests.SyncHttpHandler_HttpProtocolTests_Dribble" method="GetAsync_Chunked_VaryingSizeChunks_ReceivedCorrectly" time="19.5215231" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_Decompression_Tests.GetAsync_SetAutomaticDecompression_HeadersRemoved(remoteServer: (BaseUri: http://corefx-net-http11.azurewebsites.net/, HttpVersion: 1.1), uri: http://corefx-net-http11.azurewebsites.net/GZip.ashx)" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_Decompression_Tests" method="GetAsync_SetAutomaticDecompression_HeadersRemoved" time="23.0756572" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_Decompression_Tests.GetAsync_SetAutomaticDecompression_HeadersRemoved(remoteServer: (BaseUri: http://corefx-net-http11.azurewebsites.net/, HttpVersion: 1.1), uri: http://corefx-net-http11.azurewebsites.net/Deflate.ashx)" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_Decompression_Tests" method="GetAsync_SetAutomaticDecompression_HeadersRemoved" time="10.8656218" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_Decompression_Tests.GetAsync_SetAutomaticDecompression_HeadersRemoved(remoteServer: (BaseUri: https://corefx-net-http11.azurewebsites.net/, HttpVersion: 1.1), uri: https://corefx-net-http11.azurewebsites.net/Deflate.ashx)" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_Decompression_Tests" method="GetAsync_SetAutomaticDecompression_HeadersRemoved" time="10.6907529" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_MaxConnectionsPerServer_Test.SmallConnectionLifetimeWithMaxConnections_PendingRequestUsesDifferentConnection(lifetimeMilliseconds: 0)" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_MaxConnectionsPerServer_Test" method="SmallConnectionLifetimeWithMaxConnections_PendingRequestUsesDifferentConnection" time="14.5614442" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_MaxConnectionsPerServer_Test.GetAsync_MaxLimited_ConcurrentCallsStillSucceed(maxConnections: 1, numRequests: 5, secure: False)" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_MaxConnectionsPerServer_Test" method="GetAsync_MaxLimited_ConcurrentCallsStillSucceed" time="16.4304093" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_ResponseStreamTest.GetStreamAsync_ReadZeroBytes_Success(remoteServer: (BaseUri: https://corefx-net-http11.azurewebsites.net/, HttpVersion: 1.1))" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_ResponseStreamTest" method="GetStreamAsync_ReadZeroBytes_Success" time="16.060719" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_ResponseStreamTest.GetStreamAsync_ReadZeroBytes_Success(remoteServer: (BaseUri: https://corefx-net-http2.azurewebsites.net/, HttpVersion: 2.0))" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_ResponseStreamTest" method="GetStreamAsync_ReadZeroBytes_Success" time="11.232438" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_Cancellation_Test_Http2.PostAsync_Cancel_CancellationTokenPassedToContent(content: [StreamContent { Headers = [...] }], cancellationTokenSource: CancellationTokenSource { IsCancellationRequested = False, Token = System.Threading.CancellationToken })" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_Cancellation_Test_Http2" method="PostAsync_Cancel_CancellationTokenPassedToContent" time="17.4982194" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_Cancellation_Test_Http2.PostAsync_Cancel_CancellationTokenPassedToContent(content: [StreamContent { Headers = [...] }], cancellationTokenSource: CancellationTokenSource { IsCancellationRequested = False, Token = System.Threading.CancellationToken })" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_Cancellation_Test_Http2" method="PostAsync_Cancel_CancellationTokenPassedToContent" time="12.792622" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_HttpClientHandlerTest_Headers_Http11.SendAsync_GetWithInvalidHostHeader_ThrowsException" type="System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_HttpClientHandlerTest_Headers_Http11" method="SendAsync_GetWithInvalidHostHeader_ThrowsException" time="35.9788529" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_HttpClientHandlerTest_Headers_Http11.SendAsync_GetWithValidHostHeader_Success(withPort: False)" type="System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_HttpClientHandlerTest_Headers_Http11" method="SendAsync_GetWithValidHostHeader_Success" time="16.0019838" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_ServerCertificates_Test.UseCallback_CallbackReturnsFailure_ThrowsException" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_ServerCertificates_Test" method="UseCallback_CallbackReturnsFailure_ThrowsException" time="35.1341052" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_ServerCertificates_Test.NoCallback_ValidCertificate_SuccessAndExpectedPropertyBehavior" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_ServerCertificates_Test" method="NoCallback_ValidCertificate_SuccessAndExpectedPropertyBehavior" time="16.0046528" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_ResponseDrain_Test.GetAsyncWithMaxConnections_DrainTakesLongerThanTimeout_KillsConnection(mode: ContentLength)" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_ResponseDrain_Test" method="GetAsyncWithMaxConnections_DrainTakesLongerThanTimeout_KillsConnection" time="36.0241138" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_ResponseDrain_Test.GetAsyncWithMaxConnections_DrainTakesLongerThanTimeout_KillsConnection(mode: SingleChunk)" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_ResponseDrain_Test" method="GetAsyncWithMaxConnections_DrainTakesLongerThanTimeout_KillsConnection" time="19.2937569" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_ResponseDrain_Test.GetAsyncWithMaxConnections_DrainTakesLongerThanTimeout_KillsConnection(mode: BytePerChunk)" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_ResponseDrain_Test" method="GetAsyncWithMaxConnections_DrainTakesLongerThanTimeout_KillsConnection" time="11.258772" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_ResponseDrain_Test.GetAsyncLargeRequestWithMaxConnections_DisposeBeforeReadingToEnd_DrainsRequestsAndReusesConnection(totalSize: 100000, readSize: 1, mode: BytePerChunk)" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_ResponseDrain_Test" method="GetAsyncLargeRequestWithMaxConnections_DisposeBeforeReadingToEnd_DrainsRequestsAndReusesConnection" time="13.181608" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_ResponseDrain_Test.GetAsyncLargeRequestWithMaxConnections_DisposeBeforeReadingToEnd_DrainsRequestsAndReusesConnection(totalSize: 800000, readSize: 1, mode: ContentLength)" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_ResponseDrain_Test" method="GetAsyncLargeRequestWithMaxConnections_DisposeBeforeReadingToEnd_DrainsRequestsAndReusesConnection" time="16.1632336" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_ResponseDrain_Test.GetAsyncLargeRequestWithMaxConnections_DisposeBeforeReadingToEnd_DrainsRequestsAndReusesConnection(totalSize: 800000, readSize: 1, mode: SingleChunk)" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_ResponseDrain_Test" method="GetAsyncLargeRequestWithMaxConnections_DisposeBeforeReadingToEnd_DrainsRequestsAndReusesConnection" time="26.9584792" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_ResponseDrain_Test.GetAsyncLargeRequestWithMaxConnections_DisposeBeforeReadingToEnd_DrainsRequestsAndReusesConnection(totalSize: 1048576, readSize: 1, mode: ContentLength)" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_ResponseDrain_Test" method="GetAsyncLargeRequestWithMaxConnections_DisposeBeforeReadingToEnd_DrainsRequestsAndReusesConnection" time="23.0721038" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_Asynchrony_Test.ExecutionContext_HttpConnectionLifetimeDoesntKeepContextAlive" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_Asynchrony_Test" method="ExecutionContext_HttpConnectionLifetimeDoesntKeepContextAlive" time="23.8963807" result="Pass"> <test name="System.Net.Http.Functional.Tests.HttpClientTest.Send_TimeoutResponseContent_Throws" type="System.Net.Http.Functional.Tests.HttpClientTest" method="Send_TimeoutResponseContent_Throws" time="36.4633647" result="Pass"> <test name="System.Net.Http.Functional.Tests.HttpClientTest.Send_CancelledRequestContent_Throws" type="System.Net.Http.Functional.Tests.HttpClientTest" method="Send_CancelledRequestContent_Throws" time="105.6715878" result="Fail"> <test name="System.Net.Http.Functional.Tests.HttpClientTest.Send_TimeoutRequestContent_Throws" type="System.Net.Http.Functional.Tests.HttpClientTest" method="Send_TimeoutRequestContent_Throws" time="73.8389487" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_Cancellation_Test.Expect100Continue_WaitsExpectedPeriodOfTimeBeforeSendingContent" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_Cancellation_Test" method="Expect100Continue_WaitsExpectedPeriodOfTimeBeforeSendingContent" time="13.8793637" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_PostScenarioTest.PostNonRewindableContentUsingAuth_PreAuthenticate_Success(remoteServer: (BaseUri: http://corefx-net-http11.azurewebsites.net/, HttpVersion: 1.1))" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_PostScenarioTest" method="PostNonRewindableContentUsingAuth_PreAuthenticate_Success" time="24.0552561" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_PostScenarioTest.PostNonRewindableContentUsingAuth_PreAuthenticate_Success(remoteServer: (BaseUri: https://corefx-net-http11.azurewebsites.net/, HttpVersion: 1.1))" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_PostScenarioTest" method="PostNonRewindableContentUsingAuth_PreAuthenticate_Success" time="20.4867086" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_PostScenarioTest.PostNonRewindableContentUsingAuth_PreAuthenticate_Success(remoteServer: (BaseUri: https://corefx-net-http2.azurewebsites.net/, HttpVersion: 2.0))" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_PostScenarioTest" method="PostNonRewindableContentUsingAuth_PreAuthenticate_Success" time="15.6915136" result="Pass"> <test name="System.Net.Http.Functional.Tests.SyncHttpHandler_PostScenarioTest.PostRewindableStreamContentMultipleTimes_StreamContentFullySent(remoteServer: (BaseUri: http://corefx-net-http11.azurewebsites.net/, HttpVersion: 1.1))" type="System.Net.Http.Functional.Tests.SyncHttpHandler_PostScenarioTest" method="PostRewindableStreamContentMultipleTimes_StreamContentFullySent" time="24.0119483" result="Pass"> <test name="System.Net.Http.Functional.Tests.SyncHttpHandler_PostScenarioTest.PostRewindableStreamContentMultipleTimes_StreamContentFullySent(remoteServer: (BaseUri: https://corefx-net-http11.azurewebsites.net/, HttpVersion: 1.1))" type="System.Net.Http.Functional.Tests.SyncHttpHandler_PostScenarioTest" method="PostRewindableStreamContentMultipleTimes_StreamContentFullySent" time="23.3359397" result="Pass"> <test name="System.Net.Http.Functional.Tests.SyncHttpHandler_PostScenarioTest.PostRewindableStreamContentMultipleTimes_StreamContentFullySent(remoteServer: (BaseUri: https://corefx-net-http2.azurewebsites.net/, HttpVersion: 2.0))" type="System.Net.Http.Functional.Tests.SyncHttpHandler_PostScenarioTest" method="PostRewindableStreamContentMultipleTimes_StreamContentFullySent" time="15.4472061" result="Pass"> <test name="System.Net.Http.Functional.Tests.SyncHttpHandler_HttpProtocolTests.GetAsync_Chunked_VaryingSizeChunks_ReceivedCorrectly(maxChunkSize: 1, lineEnding: \"\\n\", useCopyToAsync: False)" type="System.Net.Http.Functional.Tests.SyncHttpHandler_HttpProtocolTests" method="GetAsync_Chunked_VaryingSizeChunks_ReceivedCorrectly" time="12.2817692" result="Pass"> <test name="System.Net.Http.Functional.Tests.SyncHttpHandler_HttpProtocolTests.GetAsync_Chunked_VaryingSizeChunks_ReceivedCorrectly(maxChunkSize: 1, lineEnding: \"\\r\\n\", useCopyToAsync: False)" type="System.Net.Http.Functional.Tests.SyncHttpHandler_HttpProtocolTests" method="GetAsync_Chunked_VaryingSizeChunks_ReceivedCorrectly" time="12.9498969" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_Http2.PostAsyncDuplex_DisposeResponseBodyBeforeEnd_ResetsStreamAndThrowsOnRequestStreamWriteAndResponseStreamRead" type="System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_Http2" method="PostAsyncDuplex_DisposeResponseBodyBeforeEnd_ResetsStreamAndThrowsOnRequestStreamWriteAndResponseStreamRead" time="34.6259292" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_Http2.SendAsync_ConcurentSendReceive_Fail" type="System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_Http2" method="SendAsync_ConcurentSendReceive_Fail" time="20.0900152" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_Http2.PostAsyncDuplex_DisposeResponseBodyAfterEndReceivedButBeforeConsumed_ResetsStreamAndThrowsOnRequestStreamWriteAndResponseStreamRead" type="System.Net.Http.Functional.Tests.SocketsHttpHandlerTest_Http2" method="PostAsyncDuplex_DisposeResponseBodyAfterEndReceivedButBeforeConsumed_ResetsStreamAndThrowsOnRequestStreamWriteAndResponseStreamRead" time="10.1601642" result="Pass"> <test name="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_ClientCertificates_Test.Manual_CertificateSentMatchesCertificateReceived_Success(numberOfRequests: 6, reuseClient: False)" type="System.Net.Http.Functional.Tests.SocketsHttpHandler_HttpClientHandler_ClientCertificates_Test" method="Manual_CertificateSentMatchesCertificateReceived_Success" time="38.8830747" result="Pass">

I queried Kusto for slow tests (duration over 100s) and there are other tests as well, but I do concur that there's something fishy about the Send_..._Throws tests. They pop up in the query much much more that the others. I'll look into it some more.

btw there is option to run particular tests without parallelism. That may be good option for time sensitive tests.
Since this is upper bound, I think long timeouts are still acceptable. I just wish we do not sprinkle code with magic numbers.

Got the test times down to somewhat reasonable values: #38453 (comment)

antonfirsov · 2020-06-29T12:10:15Z

It would be really great if we could make all these timeouts configurable on per-platform basis. In my understanding, we have infrastructure for this. @wfurt any reason we are not using System.Net.Test.Common.Configuration for setting timeouts?

@ManickaP I would at least dedupe TimeSpan.FromMinutes(5) within HttpClientTest.cs making it a property or readonly field.

ManickaP · 2020-06-29T20:12:45Z

/azp run runtime-libraries outerloop

azure-pipelines · 2020-06-29T20:12:57Z

Azure Pipelines successfully started running 1 pipeline(s).

alnikola · 2020-07-01T13:06:24Z

/azp run runtime-libraries outerloop

azure-pipelines · 2020-07-01T13:06:39Z

Azure Pipelines successfully started running 1 pipeline(s).

ManickaP · 2020-07-03T14:43:49Z

/azp run runtime-libraries outerloop

azure-pipelines · 2020-07-03T14:44:03Z

Azure Pipelines successfully started running 1 pipeline(s).

ManickaP · 2020-07-04T15:52:00Z

So I got the tests to, IMHO, reasonable numbers:

Method	avg_Duration	max_Duration
Send_CancelledRequestContent_Throws	4.3647147999999989	12.6817939
Send_CancelledResponseContent_Throws	3.6841348045454536	12.1941902
Send_TimeoutRequestContent_Throws	1.8534523818181816	5.7133473
Send_TimeoutResponseContent_Throws	18.840777004545455	37.9700282

In comparison to 100s of seconds I was seeing at the beginning, I'm quite happy with the current result.
Note that those are outerloop tests.

ManickaP · 2020-07-04T15:54:53Z

This applies only to sync scenario.
While investigating this, I discovered that in case of cancel/timeout during response content reading, we trigger DrainOnAsync on the stream, depending on the response content type. This leads to ~3 seconds propagation (on my machine) of the cancellation exception to the caller. Unfortunately, there is now way to know we're in the middle of cancellation, because we're inside stream sync methods (without cancellationToken).

ManickaP · 2020-07-04T16:04:13Z

@antonfirsov @wfurt : Both of you are unhappy with the timeout constants in the code, but I'm hesitant to add fields/configuration for them at the moment. AFAIK, there's no such thing in the HTTP test code anywhere right now and I feel like it is an issue on its own. If we're to do it, than properly and per whole test library. I'm against doing it in this PR just for the 2 values used here.

I'm also a bit skeptical whether we wouldn't end up with 100s of timeout constants for each test/purpose. That would be a whole different mess to deal with.

wfurt · 2020-07-07T01:41:32Z

AFAIK, there's no such thing in the HTTP test code anywhere

I was thinking about following TestHelper.PassingTestTimeoutMilliseconds example. You can add TestHelper.ShortDelay and TestHelper.LongTimeout (or something like that). But I don't want to hold back test fixes if rest of the team feels ok.

MihaZupan

LGTM, timeouts can be constantified later

ManickaP requested a review from a team June 26, 2020 16:38

ManickaP added the area-System.Net.Http label Jun 26, 2020

ManickaP mentioned this pull request Jun 26, 2020

Increase delays in flaky tests #38444

Merged

wfurt reviewed Jun 26, 2020

View reviewed changes

ManickaP mentioned this pull request Jun 28, 2020

Test failure: System.Net.Http.Functional.Tests.HttpClientTest.Send_CancelledRequestContent_Throws #38497

Closed

This was referenced Jun 30, 2020

RunGetBitLengthTests unit test failing on wasm #38605

Closed

System.DirectoryServices.Protocols.Tests crashes sometimes on Linux #37911

Closed

ManickaP force-pushed the mapichov/fixed_sync_outerloop branch from 2821bfe to 55f4b15 Compare June 30, 2020 13:22

ManickaP added 2 commits July 1, 2020 16:42

Longer timeout due to slow infra.

2164239

Added timeout to Wait.

a5d5ac2

ManickaP force-pushed the mapichov/fixed_sync_outerloop branch 3 times, most recently from feb4152 to e9c8582 Compare July 3, 2020 10:15

Timing experiments.

b645643

ManickaP force-pushed the mapichov/fixed_sync_outerloop branch from e9c8582 to b645643 Compare July 3, 2020 10:17

Flushy flush, might help, or not, we'll see.

71370fa

ManickaP requested review from a team, wfurt and scalablecory July 4, 2020 16:04

MihaZupan approved these changes Jul 7, 2020

View reviewed changes

ManickaP merged commit 2254059 into dotnet:master Jul 7, 2020

ManickaP deleted the mapichov/fixed_sync_outerloop branch July 7, 2020 13:08

karelz added this to the 5.0.0 milestone Aug 18, 2020

ghost locked as resolved and limited conversation to collaborators Dec 8, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fixed outerloop tests for Sync HttpClient API #38453

Fixed outerloop tests for Sync HttpClient API #38453

ManickaP commented Jun 26, 2020 •

edited

Loading

Dotnet-GitSync-Bot commented Jun 26, 2020

ghost commented Jun 26, 2020

ManickaP commented Jun 26, 2020

azure-pipelines bot commented Jun 26, 2020

wfurt Jun 26, 2020

ManickaP Jun 26, 2020

ManickaP Jun 26, 2020

scalablecory Jun 29, 2020

ManickaP Jun 29, 2020

ManickaP Jun 29, 2020

wfurt Jul 1, 2020

ManickaP Jul 4, 2020

antonfirsov commented Jun 29, 2020 •

edited

Loading

ManickaP commented Jun 29, 2020

azure-pipelines bot commented Jun 29, 2020

alnikola commented Jul 1, 2020

azure-pipelines bot commented Jul 1, 2020

ManickaP commented Jul 3, 2020

azure-pipelines bot commented Jul 3, 2020

ManickaP commented Jul 4, 2020 •

edited

Loading

ManickaP commented Jul 4, 2020

ManickaP commented Jul 4, 2020

wfurt commented Jul 7, 2020

MihaZupan left a comment

Fixed outerloop tests for Sync HttpClient API #38453

Fixed outerloop tests for Sync HttpClient API #38453

Conversation

ManickaP commented Jun 26, 2020 • edited Loading

Dotnet-GitSync-Bot commented Jun 26, 2020

ghost commented Jun 26, 2020

ManickaP commented Jun 26, 2020

azure-pipelines bot commented Jun 26, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

antonfirsov commented Jun 29, 2020 • edited Loading

ManickaP commented Jun 29, 2020

azure-pipelines bot commented Jun 29, 2020

alnikola commented Jul 1, 2020

azure-pipelines bot commented Jul 1, 2020

ManickaP commented Jul 3, 2020

azure-pipelines bot commented Jul 3, 2020

ManickaP commented Jul 4, 2020 • edited Loading

ManickaP commented Jul 4, 2020

ManickaP commented Jul 4, 2020

wfurt commented Jul 7, 2020

MihaZupan left a comment

Choose a reason for hiding this comment

ManickaP commented Jun 26, 2020 •

edited

Loading

antonfirsov commented Jun 29, 2020 •

edited

Loading

ManickaP commented Jul 4, 2020 •

edited

Loading