[API Proposal]: add direct buffer access to SocketAddress

[mgravell]

Background and motivation

Currently, SocketAddress has public int Size and byte this[int index] accessors; if you need the data, accessing the underlying contents (without IVTA) means using your own buffer of the .Size, and then looping to copy the data out. This is inefficient and unnecessary.

Only read access is being considered here; it is not seen as necessary (by me, at least) to support write access (which would make the internal hash optimization awkward), nor to support storage between calls - hence ReadOnlySpan<byte> seems ideal.

Originating context: Kestrel (aspnetcore) investigation into socket overheads and perf/scalability, looking into alternative underlying implementations.

API Proposal

Add a span accessor (note: naming is hard!):

public ReadOnlySpan<byte> Span => new ReadOnlySpan<byte>(Buffer, 0, InternalSize);

This enforces all the same characteristics of the existing get accessor, but allows the contents to be accessed much more efficiently.

API Usage

SocketAddress addr = ...
var bytes = addr.Span;
// use bytes

(contrast to)

SocketAddr addr = ...
var bytes = /* your choice of temporary buffer here, using addr.Size */
for (int i = 0 ; i < addr.Size; i++)
{
    bytes[i] = addr[i];
}
// use bytes

Alternative Designs

• could be AsSpan() method instead of property getter?
• could use CopyTo(Span<byte>), but that still forces a copy, rather than in-place direct usage

Risks

A caller could use unsafe / Unsafe to access the underlying buffer; changing the contents is already allowed (by the indexer set accessor), so that isn't by itself an additional risk - however, this would not trigger the hash-code optimization path. This, though, seems a case of "do stupid things, win stupid prizes", i.e. "don't do that". As soon as unsafe / Unsafe were used, all guarantees were gone.

---

Using a simple span accessor dictates that the memory has a contiguous representation; this restricts future flexibility for refactoring, although this seems pretty unlikely. But maybe a bool TryGetSpan(out ReadOnlySpan<byte>) (or TryGetBuffer, naming) would be more future-proof there? so: some hypothetical future implementation can at least say "nope" without faulting?

---

Not a risk, but additionally, Buffer and InternalSize are currently internal rather than private - I wonder how much code that uses those could be updated to use this new API, and perhaps benefit from bounds-check elision (a loop that uses InternalSize will not benefit, but a loop over a right-sized span: will).

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[ghost]

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Background and motivation

Currently, SocketAddress has public int Size and byte this[int index] accessors; if you need the data, accessing the underlying contents (without IVTA) means using your own buffer of the .Size, and then looping to copy the data out. This is inefficient and unnecessary.

Only read access is being considered here; it is not seen as necessary (by me, at least) to support write access (which would make the internal hash optimization awkward), nor to support storage between calls - hence ReadOnlySpan<byte> seems ideal.

API Proposal

Add a span accessor (note: naming is hard!):

public ReadOnlySpan<byte> Span => new ReadOnlySpan<byte>(Buffer, 0, InternalSize);

This enforces all the same characteristics of the existing get accessor, but allows the contents to be accessed much more efficiently.

API Usage

SocketAddress addr = ...
var bytes = addr.Span;
// use bytes

(contrast to)

SocketAddr addr = ...
var bytes = /* your choice of temporary buffer here, using addr.Size */
for (int i = 0 ; i < addr.Size; i++)
{
    bytes[i] = addr[i];
}
// use bytes

Alternative Designs

• could be AsSpan() method instead of property getter?
• could use CopyTo(Span<byte>), but that still forces a copy, rather than in-place direct usage

Risks

A caller could use unsafe / Unsafe to access the underlying buffer; changing the contents is already allowed (by the indexer set accessor), so that isn't by itself an additional risk - however, this would not trigger the hash-code optimization path. This, though, seems a case of "do stupid things, win stupid prizes", i.e. "don't do that". As soon as unsafe / Unsafe were used, all guarantees were gone.

---

Using a simple span accessor dictates that the memory has a contiguous representation; this restricts future flexibility for refactoring, although this seems pretty unlikely. But maybe a bool TryGetSpan(out ReadOnlySpan<byte>) (or TryGetBuffer, naming) would be more future-proof there? so: some hypothetical future implementation can at least say "nope" without faulting?

---

Not a risk, but additionally, Buffer and InternalSize are currently internal rather than private - I wonder how much code that uses those could be updated to use this new API, and perhaps benefit from bounds-check elision (a loop that uses InternalSize will not benefit, but a loop over a right-sized span: will).

Author:	mgravell
Assignees:	-
Labels:	api-suggestion, area-System.Net.Sockets, untriaged
Milestone:	-

[mgravell]

Performance characteristics; data first

BenchmarkDotNet=v0.13.2, OS=Windows 11 (10.0.22621.819)
Intel Core i9-9900K CPU 3.60GHz (Coffee Lake), 1 CPU, 16 logical and 8 physical cores
.NET SDK=7.0.100
[Host] : .NET 7.0.0 (7.0.22.51805), X64 RyuJIT AVX2
DefaultJob : .NET 7.0.0 (7.0.22.51805), X64 RyuJIT AVX2

Method	V6	Mean	Error	StdDev	Ratio
Indexer	False	32.378 ns	0.2220 ns	0.1968 ns	1.00
Span	False	1.089 ns	0.0105 ns	0.0093 ns	0.03
Indexer	True	39.078 ns	0.4026 ns	0.3766 ns	1.00
Span	True	1.083 ns	0.0079 ns	0.0066 ns	0.03

code (note: context here is talking via P/Invoke, so the unsafe isn't atypical):

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
using System.Net;
using System.Reflection;
using System.Runtime.CompilerServices;

BenchmarkRunner.Run<Scenario>();

public sealed class SpoofedSocketAddress // like SocketAddress, but with the new API
{
    internal int InternalSize;
    internal byte[] Buffer;
    public SpoofedSocketAddress(SocketAddress value)
    {
        // just copy out the raw values
        InternalSize = (int)typeof(SocketAddress).GetField(nameof(InternalSize), BindingFlags.NonPublic | BindingFlags.Instance)!.GetValue(value)!;
        Buffer = (byte[])typeof(SocketAddress).GetField(nameof(Buffer), BindingFlags.NonPublic | BindingFlags.Instance)!.GetValue(value)!;
    }
    public ReadOnlySpan<byte> Span => new ReadOnlySpan<byte>(Buffer, 0, InternalSize);

    public int Size
    {
        get
        {
            return InternalSize;
        }
    }
    public byte this[int offset]
    {
        get
        {
            if (offset < 0 || offset >= Size)
            {
                throw new IndexOutOfRangeException();
            }
            return Buffer[offset];
        }
    }
}
[SimpleJob]
public class Scenario
{
    private SpoofedSocketAddress? addr;
    [GlobalSetup]
    public void Setup()
    {
        var endpoint = new IPEndPoint(V6 ? IPAddress.IPv6Loopback : IPAddress.Loopback, 8080);
        addr = new SpoofedSocketAddress(endpoint.Serialize());
    }

    [Params(false, true)]
    public bool V6 { get; set; }

    [Benchmark(Baseline = true)]
    [SkipLocalsInit] // to be as fair as possible
    public unsafe void Indexer()
    {
        var addr = this.addr!;
        var len = addr.Size;
        byte* buffer = stackalloc byte[len];
        for (int i = 0; i < len; i++)
        {
            buffer[i] = addr[i];
        }
        DoTheThing(buffer, len);
    }
    [Benchmark]
    public unsafe void Span()
    {
        var span = this.addr!.Span;
        fixed (byte* ptr = span)
        {
            DoTheThing(ptr, span.Length);
        }
    }

    // in real scenario, this would be extern
    [MethodImpl(MethodImplOptions.NoInlining)]
    static unsafe void DoTheThing(byte* ptr, int len) { }
}

[antonfirsov]

I don't see much value in the SocketAddress type. I wonder if we should consider something like the following instead?

public abstract class EndPoint
{
    // Existing:
    // public virtual SocketAddress Serialize();
    // public virtual EndPoint Create(SocketAddress socketAddress);

    public virtual bool TryGetSocketAddressSize(out int size);
    public virtual void Serialize(Span<byte> socketAddress);
    public virtual EndPoint Create(ReadOnlySpan<byte> socketAddress);
}


// Usage

if (endPoint.TryGetSocketAddressSize(out int size))
{
    Span<byte> socketAddress = size < 256 ? stackalloc byte[size] : new byte[size];
    endPoint.Serialize(socketAddress);
    
    // use socketAddress
}

This could enable us getting rid of many internal SocketAddress allocations, and help with #30797 without special-casing IPAddress.

@mgravell would such an API be more usable for you as well, or you prefer to work with SocketAddress?

[wfurt]

How do you intend to use is @mgravell? While SocketAddress was probably created to deal with platform, but as @antonfirsov pointed out it also has some issues - like allocation of new class instance.

Now, EndPoint even does not have Address property so asking for AddressSize seems strange. The Serialize could take ref Span<byte> or it can return the actual length.

I assume the Create would be static to create new instance @antonfirsov? Perhaps TryCreate to avoid throwing?
The #30797 is more tricky as we may need to mutate EndPoint passed in via ref. Creating new instance is easy but allowing mutability via public API is new level of complexity.

And you you care about families beyond IPEndPoint? For IP, the life is easy, to support generic families is somewhat more tricky. It seems like we would already throw for cases like DnsEndPoint....

It seems like Kestrel may get the SocketAddress from Http.sysinto native buffers. I would assume the_localEndPoint` would be identical for all connection to given server so I'm wondering if we could avoid allocation and share single instance when possible.

[antonfirsov]

I assume the Create would be static to create new instance @antonfirsov?

No, this is a different overload for the existing virtual EndPoint.Create(SocketAddress). The existing EndPoint instance is used as prototype to create new instance of the same type with a different socket address. Weird but this is how it works now :) With a static method, it wouldn't be possible to deserialize custom endpoints.

Now, EndPoint even does not have Address property so asking for AddressSize seems strange.

Successful TryGetSocketAddressSize would indicate that this endpoint can be serialized into an address buffer. This is true for IPEndPoint, UnixDomainSocketEndpoint and even exotic socket types like the ones requested in #58378.

The Serialize could take ref Span<byte> or it can return the actual length.

This way wouldn't be possible to delegate the buffer allocation to the user. With TryGetSocketAddressSize(out int size) / Serialize(Span<byte> socketAddress) callers can stackalloc or use pooled buffers (see my usage example).

[mgravell]

@wfurt the specific scenario I had in mind was "demikernel", which has similar API surface to windsock in some regards, including the address layout; if we wanted to go direct from EndPoint (or at least IPEndPoint) to the byte representation, perhaps with a little help from SocketPal, that would also work; I was initially thinking in terms of avoiding the CPU work of fetch, but avoiding the instance too has some merit.

[antonfirsov]

perhaps with a little help from SocketPal

@mgravell can you elaborate what do you mean by this? Edit: Do you need to query particular info from the sockaddr buffer before it's deserailized back into an EndPoint? (Note before serialization you can query the EndPoint, and the SocketAddress class also doesn't tell much, only Family and Size.)

[mgravell]

@antonfirsov IIRC (I might be misremembering the code) the actual byte population of Buffer is handed to SocketPal. If I'm wrong there, mea culpa - but basically what I'm after is the same fundamental bytes that SocketAddress exposes via the combination of Size and indexer - the bytes commonly handed to the underlying socket APIs

[antonfirsov]

the actual byte population of Buffer is handed to SocketPal

@mgravell yes but it's an implementation detail in SocketAddress. In my proposal it would happen in the Serialize(Span<byte> socketAddress) method.

Out of curiosity: where did you encounter this bottleneck in your investigation? Can you show some context/code for your scenario?

[Wraith2]

I've hit this problem while writing hyper-v socket endpoints, https://github.com/Wraith2/HyperVSockets/blob/dff2d010cb916976fbeba5ab7c323d2a19c6bf8f/HyperVEndPoint.cs#L35

Needing to do byte copies back and forth through the accessor was a minor annoyance so I decided not to open issue since it didn't appear that anyone else was having a problem. Now that someone else is I've added my 👍

[mgravell]

@antonfirsov I'm not claiming it is a super bottleneck in Kestrel specifically; indeed, I haven't even got far enough along to get things compiling, let alone working - it simply stood out at me as a particularly striking API gap. For network code, getting this particular serialized byte form is pretty common, and right now, if you're not blessed by IVTA, the way of doing it is incredibly indirect. For TCP, this is indeed unlikely to be a tight pinch, as TCP connections are usually reasonably long lived. For UDP, this is much more noticeable, as this byte sequence is needed per packet. And yes, you could extract and store it, but now you've got yet another copy of everything. Mostly, I was thinking in terms of "paper cuts": here's an API that is demanded by networking scenarios, that could be hugely improved with a one-line addition.

[antonfirsov]

A ReadOnlySpan<byte> property wouldn't help with cases where SocketAddress needs to be created from a native buffer, or at least not without unsafe code pinning or converting the span. Example with Kestrel's own copy of SocketAddress here.

I would add a new constructor SocketAddress(ReadOnlySpan<byte>) for that.

[wfurt]

The questions is if SocketAddress is the goal or just artifact of the past. The documentation does not seems to guarantee that the outcome is actually sockaddr and https://learn.microsoft.com/en-us/dotnet/api/system.net.socketaddress?view=net-6.0 description is wrong for macOS where sin_family is single byte.

I would not mind to add some more functions directly to EndPoint. Aside from DnsEndPoint, I'm wondering if there is reasonable case when there is no corresponding sockaddr_XXX or way to dump something to buffer.
However, the existing Create creates new instance. That make sense as making EndPoint publicly mutable opens big can of worms IMHO. What Sockets do internally is different from public API IMHO.

And there are cases like #78448: The size differs between actual length vs maximum needed.

I would not mind the API as proposed either @mgravell but I want to make sure it solves some real problem. (small preference on AsSpan() but naming can be finalized during API review)
Since the proposal is ReadOnlySpan<byte> it would not allow modifications e.g no worries about HashCode.

[antonfirsov]

Triage: the exact API changes may depend on the fate of #30797 and #78993. Without knowing clear specific use cases where this can improve end-to-end scenarios, we don't feel committed to implement this for 8.0.

[stephentoub]

This was addressed in .NET 8.