Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use generated PInvokes and exchange types #50685

Merged
merged 27 commits into from
Nov 3, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions eng/Dependencies.props
Original file line number Diff line number Diff line change
Expand Up @@ -200,6 +200,7 @@ and are generated based on the last package release.
<LatestPackageReference Include="MessagePack" />
<LatestPackageReference Include="MessagePackAnalyzer" />
<LatestPackageReference Include="Microsoft.Data.SqlClient" />
<LatestPackageReference Include="Microsoft.Windows.CsWin32" />
Tratcher marked this conversation as resolved.
Show resolved Hide resolved
<LatestPackageReference Include="Mono.Cecil" />
<LatestPackageReference Include="Mono.TextTemplating" />
<LatestPackageReference Include="Moq" />
Expand Down
1 change: 1 addition & 0 deletions eng/Versions.props
Original file line number Diff line number Diff line change
Expand Up @@ -302,6 +302,7 @@
<MicrosoftIdentityWebGraphServiceClientVersion>2.13.4</MicrosoftIdentityWebGraphServiceClientVersion>
<MicrosoftIdentityWebUIVersion>2.13.4</MicrosoftIdentityWebUIVersion>
<MicrosoftIdentityWebDownstreamApiVersion>2.13.4</MicrosoftIdentityWebDownstreamApiVersion>
<MicrosoftWindowsCsWin32Version>0.3.46-beta</MicrosoftWindowsCsWin32Version>
<MessagePackAnalyzerVersion>$(MessagePackVersion)</MessagePackAnalyzerVersion>
<MoqVersion>4.10.0</MoqVersion>
<MonoCecilVersion>0.11.2</MonoCecilVersion>
Expand Down
1 change: 1 addition & 0 deletions src/Servers/HttpSys/HttpSysServer.slnf
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
"solution": {
"path": "..\\..\\..\\AspNetCore.sln",
"projects": [
"src\\Antiforgery\\src\\Microsoft.AspNetCore.Antiforgery.csproj",
"src\\DataProtection\\Abstractions\\src\\Microsoft.AspNetCore.DataProtection.Abstractions.csproj",
"src\\DataProtection\\Cryptography.Internal\\src\\Microsoft.AspNetCore.Cryptography.Internal.csproj",
"src\\DataProtection\\DataProtection\\src\\Microsoft.AspNetCore.DataProtection.csproj",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@
using System.Text;
using BenchmarkDotNet.Attributes;
using Microsoft.AspNetCore.HttpSys.Internal;
using Windows.Win32.Foundation;
using Windows.Win32.Networking.HttpServer;
using RequestHeaders = Microsoft.AspNetCore.HttpSys.Internal.RequestHeaders;

[SimpleJob, MemoryDiagnoser]
Expand Down Expand Up @@ -54,7 +56,7 @@ private unsafe RequestHeaders CreateRequestHeader(int unknowHeaderCount)
var nativeContext = new NativeRequestContext(MemoryPool<byte>.Shared, null, 0, false);
var nativeMemory = new Span<byte>(nativeContext.NativeRequest, (int)nativeContext.Size + 8);

var requestStructure = new HttpApiTypes.HTTP_REQUEST();
var requestStructure = new HTTP_REQUEST_V1();
var remainingMemory = SetUnknownHeaders(nativeMemory, ref requestStructure, GenerateUnknownHeaders(unknowHeaderCount));
SetHostHeader(remainingMemory, ref requestStructure);
MemoryMarshal.Write(nativeMemory, in requestStructure);
Expand All @@ -64,64 +66,64 @@ private unsafe RequestHeaders CreateRequestHeader(int unknowHeaderCount)
return requestHeaders;
}

private unsafe Span<byte> SetHostHeader(Span<byte> nativeMemory, ref HttpApiTypes.HTTP_REQUEST requestStructure)
private unsafe Span<byte> SetHostHeader(Span<byte> nativeMemory, ref HTTP_REQUEST_V1 requestStructure)
{
// Writing localhost to Host header
var dataDestination = nativeMemory.Slice(Marshal.SizeOf<HttpApiTypes.HTTP_REQUEST>());
int length = Encoding.ASCII.GetBytes("localhost:5001", dataDestination);
var dataDestination = nativeMemory[Marshal.SizeOf<HTTP_REQUEST_V1>()..];
var length = Encoding.ASCII.GetBytes("localhost:5001", dataDestination);
fixed (byte* address = &MemoryMarshal.GetReference(dataDestination))
{
requestStructure.Headers.KnownHeaders_29.pRawValue = address;
requestStructure.Headers.KnownHeaders_29.RawValueLength = (ushort)length;
requestStructure.Headers.KnownHeaders._28.pRawValue = (PCSTR)address;
requestStructure.Headers.KnownHeaders._28.RawValueLength = (ushort)length;
}
return dataDestination;
}

/// <summary>
/// Writes an array HTTP_UNKNOWN_HEADER and an array of header key-value pairs to nativeMemory. Pointers in the HTTP_UNKNOWN_HEADER structure points to the corresponding key-value pair.
/// </summary>
private unsafe Span<byte> SetUnknownHeaders(Span<byte> nativeMemory, ref HttpApiTypes.HTTP_REQUEST requestStructure, IReadOnlyCollection<(string Key, string Value)> headerNames)
private unsafe Span<byte> SetUnknownHeaders(Span<byte> nativeMemory, ref HTTP_REQUEST_V1 requestStructure, IReadOnlyCollection<(string Key, string Value)> headerNames)
{
var unknownHeaderStructureDestination = nativeMemory.Slice(Marshal.SizeOf<HttpApiTypes.HTTP_REQUEST>());
var unknownHeaderStructureDestination = nativeMemory[Marshal.SizeOf<HTTP_REQUEST_V1>()..];
fixed (byte* address = &MemoryMarshal.GetReference(unknownHeaderStructureDestination))
{
requestStructure.Headers.pUnknownHeaders = (HttpApiTypes.HTTP_UNKNOWN_HEADER*)address;
requestStructure.Headers.pUnknownHeaders = (HTTP_UNKNOWN_HEADER*)address;
}
requestStructure.Headers.UnknownHeaderCount += (ushort)headerNames.Count;

var unknownHeadersSize = Marshal.SizeOf<HttpApiTypes.HTTP_UNKNOWN_HEADER>();
var dataDestination = unknownHeaderStructureDestination.Slice(unknownHeadersSize * headerNames.Count);
foreach (var headerName in headerNames)
var unknownHeadersSize = Marshal.SizeOf<HTTP_UNKNOWN_HEADER>();
var dataDestination = unknownHeaderStructureDestination[(unknownHeadersSize * headerNames.Count)..];
foreach (var (headerKey, headerValue) in headerNames)
{
var unknownHeaderStructure = new HttpApiTypes.HTTP_UNKNOWN_HEADER();
int nameLength = Encoding.ASCII.GetBytes(headerName.Key, dataDestination);
var unknownHeaderStructure = new HTTP_UNKNOWN_HEADER();
var nameLength = Encoding.ASCII.GetBytes(headerKey, dataDestination);
fixed (byte* address = &MemoryMarshal.GetReference(dataDestination))
{
unknownHeaderStructure.pName = address;
unknownHeaderStructure.pName = (PCSTR)address;
unknownHeaderStructure.NameLength = (ushort)nameLength;
}
dataDestination = dataDestination.Slice(nameLength);
dataDestination = dataDestination[nameLength..];

if (!string.IsNullOrEmpty(headerName.Value))
if (!string.IsNullOrEmpty(headerValue))
{
int valueLength = Encoding.ASCII.GetBytes(headerName.Value, dataDestination);
var valueLength = Encoding.ASCII.GetBytes(headerValue, dataDestination);
fixed (byte* address = &MemoryMarshal.GetReference(dataDestination))
{
unknownHeaderStructure.pRawValue = address;
unknownHeaderStructure.pRawValue = (PCSTR)address;
unknownHeaderStructure.RawValueLength = (ushort)valueLength;
}
dataDestination = dataDestination.Slice(nameLength);
dataDestination = dataDestination[nameLength..];
}
MemoryMarshal.Write(unknownHeaderStructureDestination, in unknownHeaderStructure);
unknownHeaderStructureDestination = unknownHeaderStructureDestination.Slice(unknownHeadersSize);
unknownHeaderStructureDestination = unknownHeaderStructureDestination[unknownHeadersSize..];
}
return dataDestination;
}

private IReadOnlyCollection<(string, string)> GenerateUnknownHeaders(int count)
private static List<(string, string)> GenerateUnknownHeaders(int count)
{
var result = new List<(string, string)>();
for (int i = 0; i < count; i++)
for (var i = 0; i < count; i++)
{
result.Add(($"X-Custom-{i}", $"Value-{i}"));
}
Expand Down
129 changes: 62 additions & 67 deletions src/Servers/HttpSys/samples/TestClient/Program.cs
Original file line number Diff line number Diff line change
@@ -1,94 +1,89 @@
// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.

using System;
using System.Net;
using System.Net.Http;
using System.Net.WebSockets;
using System.Text;
using System.Threading;
using System.Threading.Tasks;

namespace TestClient
namespace TestClient;

public class Program
{
public class Program
private const string Address =
"http://localhost:5000/public/1kb.txt";
// "https://localhost:9090/public/1kb.txt";

public static void Main(string[] args)
{
private const string Address =
"http://localhost:5000/public/1kb.txt";
// "https://localhost:9090/public/1kb.txt";
Console.WriteLine("Ready");
Console.ReadKey();

public static void Main(string[] args)
{
Console.WriteLine("Ready");
Console.ReadKey();
var handler = new HttpClientHandler();
handler.MaxConnectionsPerServer = 500;
handler.ServerCertificateCustomValidationCallback = HttpClientHandler.DangerousAcceptAnyServerCertificateValidator;
// handler.UseDefaultCredentials = true;
HttpClient client = new HttpClient(handler);

var handler = new HttpClientHandler();
handler.MaxConnectionsPerServer = 500;
handler.ServerCertificateCustomValidationCallback = HttpClientHandler.DangerousAcceptAnyServerCertificateValidator;
// handler.UseDefaultCredentials = true;
HttpClient client = new HttpClient(handler);
RunParallelRequests(client);

RunParallelRequests(client);
// RunManualRequests(client);

// RunManualRequests(client);
// RunWebSocketClient().Wait();

// RunWebSocketClient().Wait();
Console.WriteLine("Done");
// Console.ReadKey();
}

Console.WriteLine("Done");
// Console.ReadKey();
private static void RunManualRequests(HttpClient client)
{
while (true)
{
Console.WriteLine("Press any key to send request");
Console.ReadKey();
var result = client.GetAsync(Address).Result;
Console.WriteLine(result);
}
}

private static void RunManualRequests(HttpClient client)
private static void RunParallelRequests(HttpClient client)
{
int completionCount = 0;
int iterations = 100000;
for (int i = 0; i < iterations; i++)
{
while (true)
{
Console.WriteLine("Press any key to send request");
Console.ReadKey();
var result = client.GetAsync(Address).Result;
Console.WriteLine(result);
}
client.GetAsync(Address)
.ContinueWith(t => Interlocked.Increment(ref completionCount));
}

private static void RunParallelRequests(HttpClient client)
while (completionCount < iterations)
{
int completionCount = 0;
int iterations = 100000;
for (int i = 0; i < iterations; i++)
{
client.GetAsync(Address)
.ContinueWith(t => Interlocked.Increment(ref completionCount));
}

while (completionCount < iterations)
{
Thread.Sleep(10);
}
Thread.Sleep(10);
}
}

public static async Task RunWebSocketClient()
{
ClientWebSocket websocket = new ClientWebSocket();

string url = "ws://localhost:5000/";
Console.WriteLine("Connecting to: " + url);
await websocket.ConnectAsync(new Uri(url), CancellationToken.None);

public static async Task RunWebSocketClient()
string message = "Hello World";
Console.WriteLine("Sending message: " + message);
byte[] messageBytes = Encoding.UTF8.GetBytes(message);
await websocket.SendAsync(new ArraySegment<byte>(messageBytes), WebSocketMessageType.Text, true, CancellationToken.None);

byte[] incomingData = new byte[1024];
WebSocketReceiveResult result = await websocket.ReceiveAsync(new ArraySegment<byte>(incomingData), CancellationToken.None);

if (result.CloseStatus.HasValue)
{
Console.WriteLine("Closed; Status: " + result.CloseStatus + ", " + result.CloseStatusDescription);
}
else
{
ClientWebSocket websocket = new ClientWebSocket();

string url = "ws://localhost:5000/";
Console.WriteLine("Connecting to: " + url);
await websocket.ConnectAsync(new Uri(url), CancellationToken.None);

string message = "Hello World";
Console.WriteLine("Sending message: " + message);
byte[] messageBytes = Encoding.UTF8.GetBytes(message);
await websocket.SendAsync(new ArraySegment<byte>(messageBytes), WebSocketMessageType.Text, true, CancellationToken.None);

byte[] incomingData = new byte[1024];
WebSocketReceiveResult result = await websocket.ReceiveAsync(new ArraySegment<byte>(incomingData), CancellationToken.None);

if (result.CloseStatus.HasValue)
{
Console.WriteLine("Closed; Status: " + result.CloseStatus + ", " + result.CloseStatusDescription);
}
else
{
Console.WriteLine("Received message: " + Encoding.UTF8.GetString(incomingData, 0, result.Count));
}
Console.WriteLine("Received message: " + Encoding.UTF8.GetString(incomingData, 0, result.Count));
}
}
}
29 changes: 14 additions & 15 deletions src/Servers/HttpSys/src/AsyncAcceptContext.cs
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@

using System.Diagnostics;
using System.Threading.Tasks.Sources;
using Microsoft.AspNetCore.HttpSys.Internal;

namespace Microsoft.AspNetCore.Server.HttpSys;

Expand Down Expand Up @@ -39,9 +38,9 @@ internal ValueTask<RequestContext> AcceptAsync()

AllocateNativeRequest();

uint statusCode = QueueBeginGetContext();
if (statusCode != UnsafeNclNativeMethods.ErrorCodes.ERROR_SUCCESS &&
statusCode != UnsafeNclNativeMethods.ErrorCodes.ERROR_IO_PENDING)
var statusCode = QueueBeginGetContext();
if (statusCode != ErrorCodes.ERROR_SUCCESS &&
statusCode != ErrorCodes.ERROR_IO_PENDING)
{
// some other bad error, possible(?) return values are:
// ERROR_INVALID_HANDLE, ERROR_INSUFFICIENT_BUFFER, ERROR_OPERATION_ABORTED
Expand All @@ -55,16 +54,16 @@ private void IOCompleted(uint errorCode, uint numBytes)
{
try
{
if (errorCode != UnsafeNclNativeMethods.ErrorCodes.ERROR_SUCCESS &&
errorCode != UnsafeNclNativeMethods.ErrorCodes.ERROR_MORE_DATA)
if (errorCode != ErrorCodes.ERROR_SUCCESS &&
errorCode != ErrorCodes.ERROR_MORE_DATA)
{
_mrvts.SetException(new HttpSysException((int)errorCode));
return;
}

Debug.Assert(_requestContext != null);

if (errorCode == UnsafeNclNativeMethods.ErrorCodes.ERROR_SUCCESS)
if (errorCode == ErrorCodes.ERROR_SUCCESS)
{
var requestContext = _requestContext;
// It's important that we clear the request context before we set the result
Expand All @@ -79,10 +78,10 @@ private void IOCompleted(uint errorCode, uint numBytes)
AllocateNativeRequest(numBytes, _requestContext.RequestId);

// We need to issue a new request, either because auth failed, or because our buffer was too small the first time.
uint statusCode = QueueBeginGetContext();
var statusCode = QueueBeginGetContext();

if (statusCode != UnsafeNclNativeMethods.ErrorCodes.ERROR_SUCCESS &&
statusCode != UnsafeNclNativeMethods.ErrorCodes.ERROR_IO_PENDING)
if (statusCode != ErrorCodes.ERROR_SUCCESS &&
statusCode != ErrorCodes.ERROR_IO_PENDING)
{
// someother bad error, possible(?) return values are:
// ERROR_INVALID_HANDLE, ERROR_INSUFFICIENT_BUFFER, ERROR_OPERATION_ABORTED
Expand Down Expand Up @@ -117,14 +116,14 @@ private uint QueueBeginGetContext()
_requestContext.RequestId,
// Small perf impact by not using HTTP_RECEIVE_REQUEST_FLAG_COPY_BODY
// if the request sends header+body in a single TCP packet
(uint)HttpApiTypes.HTTP_FLAGS.NONE,
0u,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feels like a small loss in code quality. Is there not a "none" flag in the httpsys headers?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, there isn't. There isn't one defined natively either.
https://learn.microsoft.com/en-us/windows/win32/api/http/nf-http-httpreceivehttprequest

Value Meaning
0 (zero) Only the request headers are retrieved; the entity body is not copied.
HTTP_RECEIVE_REQUEST_FLAG_COPY_BODY The available entity body is copied along with the request headers. The pEntityChunks member of the HTTP_REQUEST structure points to the entity body.
HTTP_RECEIVE_REQUEST_FLAG_FLUSH_BODY All of the entity bodies are copied along with the request headers. The pEntityChunks member of the HTTP_REQUEST structure points to the entity body.

_requestContext.NativeRequest,
_requestContext.Size,
&bytesTransferred,
_overlapped);

if ((statusCode == UnsafeNclNativeMethods.ErrorCodes.ERROR_CONNECTION_INVALID
|| statusCode == UnsafeNclNativeMethods.ErrorCodes.ERROR_INVALID_PARAMETER)
if ((statusCode == ErrorCodes.ERROR_CONNECTION_INVALID
|| statusCode == ErrorCodes.ERROR_INVALID_PARAMETER)
&& _requestContext.RequestId != 0)
{
// ERROR_CONNECTION_INVALID:
Expand All @@ -139,15 +138,15 @@ private uint QueueBeginGetContext()
_requestContext.RequestId = 0;
retry = true;
}
else if (statusCode == UnsafeNclNativeMethods.ErrorCodes.ERROR_MORE_DATA)
else if (statusCode == ErrorCodes.ERROR_MORE_DATA)
{
// the buffer was not big enough to fit the headers, we need
// to read the RequestId returned, allocate a new buffer of the required size
// (uint)backingBuffer.Length - AlignmentPadding
AllocateNativeRequest(bytesTransferred);
retry = true;
}
else if (statusCode == UnsafeNclNativeMethods.ErrorCodes.ERROR_SUCCESS
else if (statusCode == ErrorCodes.ERROR_SUCCESS
&& HttpSysListener.SkipIOCPCallbackOnSuccess)
{
// IO operation completed synchronously - callback won't be called to signal completion.
Expand Down
Loading
Loading