Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sync HttpClient Send #34948

Merged
merged 80 commits into from
Jun 24, 2020
Merged

Sync HttpClient Send #34948

merged 80 commits into from
Jun 24, 2020

Conversation

ManickaP
Copy link
Member

Introduces sync version of HttpClient.Send and all necessary changes to make it work synchronously (e.g.: HttpContent, HttpMessageHandler and their child classes).

The change works properly only for HTTP1.1, for HTTP2 does sync-over-async.
Also for client authentication uses sync-over-async, this is an ongoing issue being separately handled in #34638.

Testing the change uses existing tests calling HttpClient.SendAsync and introduces test argument calling the same test twice, once with HttpClient.SendAsync and then with HttpClient.Send.

Resolves #32125

…socketshttphandlersync

stephentoub/corefx@0e4d640

Little compilation fix.


Compilaris


Test compilaris

Fixed buffer sending not passing offset and count.
Conflicts:
	src/libraries/Common/tests/System/Net/Http/LoopbackServer.cs
	src/libraries/System.Net.Http/src/System.Net.Http.csproj
	src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs
# Conflicts:
#	src/libraries/System.Net.Http/ref/System.Net.Http.cs
#	src/libraries/System.Net.Http/src/System.Net.Http.csproj
#	src/libraries/System.Net.Http/src/System/Net/Http/DiagnosticsHandler.cs
#	src/libraries/System.Net.Http/src/System/Net/Http/HttpClient.cs
#	src/libraries/System.Net.Http/src/System/Net/Http/HttpContent.cs
#	src/libraries/System.Net.Http/src/System/Net/Http/MultipartContent.cs
#	src/libraries/System.Net.Http/src/System/Net/Http/ReadOnlyMemoryContent.cs
#	src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/AuthenticationHelper.NtAuth.cs
#	src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/AuthenticationHelper.cs
#	src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/Http3Connection.cs
#	src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnection.cs
#	src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionBase.cs
#	src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs
#	src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPoolManager.cs
#	src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpContentWriteStream.cs
#	src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/SocketsHttpHandler.cs
#	src/libraries/System.Net.Http/src/System/Net/Http/StreamContent.cs
# Conflicts:
#	src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionBase.cs
#	src/libraries/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/HttpConnectionPool.cs
@ManickaP ManickaP requested a review from a team April 14, 2020 13:46
@ManickaP ManickaP self-assigned this Apr 14, 2020
@Dotnet-GitSync-Bot
Copy link
Collaborator

Note regarding the new-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, to please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

@ghost
Copy link

ghost commented Apr 14, 2020

Tagging subscribers to this area: @dotnet/ncl
Notify danmosemsft if you want to be subscribed.

@ManickaP
Copy link
Member Author

/azp list

@ManickaP
Copy link
Member Author

/azp run runtime-libraries outerloop

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@ManickaP ManickaP force-pushed the mapichov/sync_http_api branch 3 times, most recently from a25bcf2 to 007cfbe Compare June 19, 2020 18:51
@ManickaP ManickaP force-pushed the mapichov/sync_http_api branch 3 times, most recently from a3ebd67 to 3593372 Compare June 22, 2020 14:46
@ManickaP
Copy link
Member Author

/azp run runtime-libraries outerloop

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@davidsh
Copy link
Contributor

davidsh commented Jun 24, 2020

This PR has failures in DiagnosticHandler. I think this is caused by known issue #38205

@ManickaP
Copy link
Member Author

Results from our micro benchamarks, pr vs master:

BenchmarkDotNet=v0.12.1, OS=manjaro 
Intel Core i7-4771 CPU 3.50GHz (Haswell), 1 CPU, 8 logical and 4 physical cores
.NET Core SDK=5.0.100-preview.6.20310.4
  [Host]     : .NET Core 5.0.0 (CoreCLR 5.0.20.30506, CoreFX 5.0.20.30506), X64 RyuJIT
  Job-DJGXQP : .NET Core 5.0 (CoreCLR 42.42.42.42424, CoreFX 42.42.42.42424), X64 RyuJIT
  Job-LKRKYA : .NET Core 5.0 (CoreCLR 42.42.42.42424, CoreFX 42.42.42.42424), X64 RyuJIT

PowerPlanMode=00000000-0000-0000-0000-000000000000  Arguments=/p:DebugType=portable  IterationTime=250.0000 ms  
MaxIterationCount=20  MinIterationCount=15  WarmupCount=1  
Method Job Toolchain ssl chunkedResponse responseLength Mean Error StdDev Median Min Max Ratio RatioSD Gen 0 Gen 1 Gen 2 Allocated
Get Job-DJGXQP /master/corerun/corerun False False 1 51.68 μs 1.355 μs 1.506 μs 51.45 μs 49.07 μs 54.62 μs 1.00 0.00 0.4211 - - 2.02 KB
Get Job-LKRKYA /sync_pr/corerun/corerun False False 1 51.58 μs 1.601 μs 1.844 μs 51.41 μs 48.07 μs 54.85 μs 1.00 0.05 0.4006 - - 2.05 KB
Get Job-DJGXQP /master/corerun/corerun False False 100000 84.33 μs 1.019 μs 0.954 μs 84.42 μs 82.25 μs 85.68 μs 1.00 0.00 0.3359 - - 2.04 KB
Get Job-LKRKYA /sync_pr/corerun/corerun False False 100000 85.54 μs 1.084 μs 0.906 μs 85.39 μs 84.35 μs 87.48 μs 1.01 0.01 0.4390 - - 2.07 KB
Get Job-DJGXQP /master/corerun/corerun False True 1 51.74 μs 1.130 μs 1.256 μs 51.67 μs 48.91 μs 54.08 μs 1.00 0.00 0.4246 - - 2.01 KB
Get Job-LKRKYA /sync_pr/corerun/corerun False True 1 52.02 μs 0.992 μs 0.974 μs 52.22 μs 50.49 μs 53.54 μs 1.01 0.02 0.4214 - - 2.04 KB
Get Job-DJGXQP /master/corerun/corerun False True 100000 113.25 μs 0.796 μs 0.744 μs 113.25 μs 112.06 μs 114.50 μs 1.00 0.00 0.4619 - - 2.02 KB
Get Job-LKRKYA /sync_pr/corerun/corerun False True 100000 114.99 μs 0.997 μs 0.833 μs 115.06 μs 113.38 μs 116.15 μs 1.01 0.01 0.4615 - - 2.05 KB
Get Job-DJGXQP /master/corerun/corerun True False 1 242.44 μs 34.959 μs 38.857 μs 237.31 μs 177.12 μs 336.79 μs 1.00 0.00 - - - 5.23 KB
Get Job-LKRKYA /sync_pr/corerun/corerun True False 1 243.98 μs 29.132 μs 29.917 μs 244.62 μs 187.99 μs 316.79 μs 1.02 0.22 - - - 5.26 KB
Get Job-DJGXQP /master/corerun/corerun True False 100000 674.75 μs 254.020 μs 249.481 μs 631.25 μs 387.43 μs 1,163.31 μs 1.00 0.00 - - - 71.29 KB
Get Job-LKRKYA /sync_pr/corerun/corerun True False 100000 513.09 μs 146.182 μs 156.413 μs 440.12 μs 346.33 μs 737.11 μs 0.91 0.44 - - - 5.27 KB
Get Job-DJGXQP /master/corerun/corerun True True 1 220.30 μs 37.672 μs 41.872 μs 228.83 μs 149.07 μs 282.84 μs 1.00 0.00 - - - 5.22 KB
Get Job-LKRKYA /sync_pr/corerun/corerun True True 1 225.02 μs 38.637 μs 44.495 μs 204.35 μs 175.35 μs 307.61 μs 1.05 0.16 - - - 5.25 KB
Get Job-DJGXQP /master/corerun/corerun True True 100000 548.13 μs 149.034 μs 146.372 μs 507.55 μs 328.93 μs 958.75 μs 1.00 0.00 - - - 5.22 KB
Get Job-LKRKYA /sync_pr/corerun/corerun True True 100000 507.01 μs 102.817 μs 110.013 μs 479.67 μs 353.45 μs 681.92 μs 0.99 0.29 - - - 5.25 KB

I ran it several times. Sometimes it would show a regression in one test, other times in another. When I isolated them, I got somewhat unstable results (sometimes was my PR faster, sometimes master). Due to this, I don't think there's a perf regression per se.

@stephentoub
Copy link
Member

Ok, thanks for checking. Any differences appear to be within noise.

@ManickaP
Copy link
Member Author

All outerloop errors are unrelated (#38205).

@ManickaP ManickaP merged commit 313b165 into dotnet:master Jun 24, 2020
@ManickaP ManickaP deleted the mapichov/sync_http_api branch June 24, 2020 15:00
ManickaP added a commit to ManickaP/runtime that referenced this pull request Jul 2, 2020
Introduces sync version of HttpClient.Send and all necessary changes to make it work synchronously (e.g.: HttpContent, HttpMessageHandler and their child classes).

The change works properly only for HTTP1.1, for unsupported scenarios like HTTP2 throws.

Testing the change uses existing tests calling HttpClient.SendAsync and introduces test argument calling the same test twice, once with HttpClient.SendAsync and then with HttpClient.Send.

Resolves dotnet#32125
@ManickaP ManickaP mentioned this pull request Jul 2, 2020
@@ -537,9 +551,17 @@ public Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, HttpCompl
}

// Buffer the response content if we've been asked to and we have a Content to buffer.
if (response.Content != null)
if (buffered && response.Content != null)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should it also check: response.Content is not EmptyContent ?

After updating an app from .NET Core 3.1 to .NET 5.0, I am hitting ObjectDisposedException:

  ---- System.ObjectDisposedException : Cannot access a disposed object.
  Object name: 'System.Net.Http.EmptyContent'.
    Stack Trace:
       at System.Net.Http.HttpContent.CheckDisposed()
     at System.Net.Http.HttpContent.LoadIntoBufferAsync(Int64 maxBufferSize, CancellationToken cancellationToken)
     at System.Net.Http.HttpClient.SendAsyncCore(HttpRequestMessage request, HttpCompletionOption completionOption, Boolean async, Boolean emitTelemetryStartStop, CancellationToken cancellationToken)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nvm, it was a bug in test, which 3.1 was tolerating and 5.0 makes sure the objects are disposed properly: https://github.com/actions/runner/pull/799/files#diff-f7999ffa872c1d85e7368f4c351978e3d2a4ad3e3b9a27547234ede8e26bdaf5.

@ghost ghost locked as resolved and limited conversation to collaborators Dec 11, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Sync API for HttpClient
9 participants