Reduce the size of the pipe

[davidfowl]

• Use the pipe itself as the synchronization object
• Store the options instance as a way to reference shared settings
• Added a field to PipeOptions for storing if the Pool is the ArrayPool implementation of the MemoryPool
• Reduce the size of PipeCompletion since triggering callbacks is deprecated
• Change the size of the stack to 4 segments by default (which represents 16K of memory vs the 65K it is today). This also matches the pause writer threshold.

This makes it possible to share common options across multiple pipe instances and not pay that extra size cost.

Using the ObjectLayoutInspector:

Before
Size: 368 bytes. Paddings: 27 bytes (%7 of empty space)

After
Size: 256 bytes. Paddings: 22 bytes (%8 of empty space)
Size: 264 bytes. Paddings: 22 bytes (%8 of empty space)

Simulation of 1M connections using pipes:

var options = new PipeOptions(useSynchronizationContext: false);

for (int i = 0; i < 1_000_000; i++)
{
    var transportToApp = new Pipe(options);
    var appToTransport = new Pipe(options);

    static async Task DoRead1(PipeReader reader)
    {
        var result = await reader.ReadAsync();
        reader.AdvanceTo(result.Buffer.End);
    }

    static async Task DoRead2(PipeReader reader)
    {
        var result = await reader.ReadAsync();
        reader.AdvanceTo(result.Buffer.End);
    }

    _ = DoRead1(transportToApp.Reader);
    _ = DoRead2(appToTransport.Reader);

    _connections.Add((transportToApp, appToTransport));
}

Before:

Name	Working Set
ConsoleApp28.exe	1,590,136 K

After:

Name	Working Set
ConsoleApp28.exe	1,197,036 K

[davidfowl]

Something is broken but I'm having a hard time running tests locally 😢

[benaadams]

Before 368 bytes. After 256 bytes

Nice! 30% saving, and options is likely a shared object by all the pipes so going via the ref should mean its in cache for all pipelines rather than being a different 48 byte block for each Pipe; and since PipeOptions is immutable, seem safe.

Does that mean SocketAsyncEventArgs is now top chonk at 320 bytes?

[davidfowl]

Yep am looking into the per connection memory and SocketAsyncEventArgs is up there.

[davidfowl]

OK I'm done tweaking this thing.

[stephentoub]

I'm in no way saying it's worth it, but if you really wanted to avoid this new object() and you wanted to use that buffer segment array you tried to in a previous commit, you could instead still keep this field and just store that array into this field in the ctor. Then you're reusing the same array instance even the buffer stack is resized. There are downsides, though, namely you'd be keeping alive what I assume is a much larger object, and if that array referenced other stuff, anything it referenced.

[stephentoub]

Some build failures, but otherwise LGTM

[davidfowl]

Is this crossgen issue a known thing?

[benaadams]

Is this crossgen issue a known thing?

runtime-dev-innerloop (Build windows x86 release Runtime_Debug) is a timeout cull because it ran for more than 60mins; seen it pop up a few times on PRs and go away, looks like its on the timeout edge?

e.g. runs for 58mins on successful builds