Unexpected pattern

[thomashilke]

While peeking at the code of MoreLinq, I've spotted more than once a code construct for which I fail to identify the reason of being. An example is the implementation of Pairwise:

        public static IEnumerable<TResult> Pairwise<TSource, TResult>(this IEnumerable<TSource> source, Func<TSource, TSource, TResult> resultSelector)
        {
            if (source == null) throw new ArgumentNullException(nameof(source));
            if (resultSelector == null) throw new ArgumentNullException(nameof(resultSelector));

            return _(); IEnumerable<TResult> _()
            {
                using var e = source.GetEnumerator();

                if (!e.MoveNext())
                    yield break;

                var previous = e.Current;
                while (e.MoveNext())
                {
                    yield return resultSelector(previous, e.Current);
                    previous = e.Current;
                }
            }
        }

but I have also seen it in From, Exclude, etc.

The question is: why define a local lambda _ and call it immediately? In what sens is it not equivalent to:

        public static IEnumerable<TResult> Pairwise<TSource, TResult>(this IEnumerable<TSource> source, Func<TSource, TSource, TResult> resultSelector)
        {
            if (source == null) throw new ArgumentNullException(nameof(source));
            if (resultSelector == null) throw new ArgumentNullException(nameof(resultSelector));

            using var e = source.GetEnumerator();

            if (!e.MoveNext())
                yield break;

            var previous = e.Current;
            while (e.MoveNext())
            {
                yield return resultSelector(previous, e.Current);
                previous = e.Current;
            }
        }

Thanks in advance!
Thomas

[i3arnon]

The local lambda in this case is an iterator method.
It's a special method in C# that allows pausing and resuming execution.
You can see that this method doesn't return an IEnumerable anywhere in its code while the method's signature guarantees it does.
The yield return statement tells the compiler to return a single element in the resulting enumerable and then pause execution until that enumerable continues being iterated.

So, what this pattern achieves is immediate argument validation while maintaining LINQ's deferred execution.
Currently if there's an issue the exception is thrown immediately and in your version you need to iterate the resulting enumerable to get an exception.

You can try running this code yourself on both versions to see the difference:

MoreEnumerable.Pairwise<int,int>(null, null);

I have a relevant (and longer) post focusing on the async versions of these issues: https://blog.i3arnon.com/2021/07/12/async-linq-operator/

[viceroypenguin]

This is a pattern you will find even in the .net runtime in the enumerable methods. The most important thing to realize is that if you do a yield return, none of the code in the method will run until the first time the consumer actually enumerates the data. This means that arguments are not checked immediately in the normal method. Doing the local method pattern allows for immediate argument validation while still using the yield return pattern.

Example consumer code:

var data = source.Pairwise(null);

// Do something else

return data;

Under the default version (what you provided), then the null argument will not be checked in this method, but will be checked when the caller tries to use the enumerable. This means the exception is separated from the place it occurred, and is much less useful. Using the local method, the null argument is checked in the method, so the exception happens at a useful place and can be debugged quickly.

[thomashilke]

@viceroypenguin Thank you, I am too young to be aware of all the historical heritage :)
@i3arnon The Reset method of the IEnumerator generated by the compiler seems to simply throw an NotSupportedException:

        [DebuggerHidden]
        void IEnumerator.Reset()
        {
            throw new NotSupportedException();
        }

[i3arnon]

@i3arnon The Reset method of the IEnumerator generated by the compiler seems to simply throw an NotSupportedException:

This is unrelated.
When you iterate an enumerable you start by calling GetEnumerator that returns an IEnumerator. Each iteration gets a separate enumerator instance, there's no need for a reset.
Reset is meaningless in this case as once you started executing the method you can't "go back". It still allows for separate executions for separate iterations by returning new enumerator instances from the same enumerable.

[viceroypenguin]

@i3arnon I stand corrected - the IEnumerable<> can be reused. Per your suggestion of evidence, I went to check and verified that you are correct. My memory may be fuzzy, but I think when first implemented, they couldn't. I remember that being a distinction about iterator methods when LINQ was first introduced, that you had to be careful with using them in a LINQ pattern because of that. I wonder if that was fixed with roslyn. Or like I said, my memory is fuzzy, and I'm just remembering things incorrectly. Anyway, TIL! :)

[thomashilke]

@i3arnon Yes, I think we are all on the same page now.

Thank you both for taking the time to answer my question!

[atifaziz]

I thought that the behavior was similar as async method, where the prologue is executed synchronously until the first await is reached. The discrepancy is a bit shocking IMO, given the similarity of two constructs and their implementation by converting the method into a finite state machine (for the curious, here is the desugared version of my version of the Pairwise code).

@thomashilke Actually, async methods/functions behave pretty much the same way! While you're right that the prologue is executed synchronously until the first await is reached, if an argument validation throws an exception in that passed of code, it will get wrapped into a task object like Task or ValueTask! Unless the caller also awaits on the result immediately on the invoked method/function, the exception won't be thrown until some point later when the returned task is awaited.

var foo = Foo(42);
var boo = Foo(-42);                   // no exception thrown
var t = await Task.WhenAny(foo, boo); // no exception thrown
await t;                              // ArgumentOutOfRangeException thrown here

static async Task Foo(int bar)
{
    if (bar < 0)
        throw new ArgumentOutOfRangeException(nameof(bar), bar, null);
    await Task.Yield();
}

You can see why this happens by exploring the following code in sharplab.io:

using System;
using System.Threading.Tasks;
public class C
{
    public async Task M1(object input)
    {
        if (input is null) throw new ArgumentNullException(nameof(input));
        await Task.Yield();
    }

    public Task M2(object input)
    {
        if (input is null) throw new ArgumentNullException(nameof(input));

        return _(); async Task _()
        {
            await Task.Yield();
        }
    }
}

that produces:

using System;
using System.Diagnostics;
using System.Reflection;
using System.Runtime.CompilerServices;
using System.Runtime.InteropServices;
using System.Security;
using System.Security.Permissions;
using System.Threading.Tasks;

[assembly: CompilationRelaxations(8)]
[assembly: RuntimeCompatibility(WrapNonExceptionThrows = true)]
[assembly: Debuggable(DebuggableAttribute.DebuggingModes.IgnoreSymbolStoreSequencePoints)]
[assembly: SecurityPermission(SecurityAction.RequestMinimum, SkipVerification = true)]
[assembly: AssemblyVersion("0.0.0.0")]
[module: UnverifiableCode]
public class C
{
    [StructLayout(LayoutKind.Auto)]
    [CompilerGenerated]
    private struct <M1>d__0 : IAsyncStateMachine
    {
        public int <>1__state;

        public AsyncTaskMethodBuilder <>t__builder;

        public object input;

        private YieldAwaitable.YieldAwaiter <>u__1;

        private void MoveNext()
        {
            int num = <>1__state;
            try
            {
                YieldAwaitable.YieldAwaiter awaiter;
                if (num != 0)
                {
                    if (input == null)
                    {
                        throw new ArgumentNullException("input");
                    }
                    awaiter = Task.Yield().GetAwaiter();
                    if (!awaiter.IsCompleted)
                    {
                        num = (<>1__state = 0);
                        <>u__1 = awaiter;
                        <>t__builder.AwaitUnsafeOnCompleted(ref awaiter, ref this);
                        return;
                    }
                }
                else
                {
                    awaiter = <>u__1;
                    <>u__1 = default(YieldAwaitable.YieldAwaiter);
                    num = (<>1__state = -1);
                }
                awaiter.GetResult();
            }
            catch (Exception exception)
            {
                <>1__state = -2;
                <>t__builder.SetException(exception);
                return;
            }
            <>1__state = -2;
            <>t__builder.SetResult();
        }

        void IAsyncStateMachine.MoveNext()
        {
            //ILSpy generated this explicit interface implementation from .override directive in MoveNext
            this.MoveNext();
        }

        [DebuggerHidden]
        private void SetStateMachine(IAsyncStateMachine stateMachine)
        {
            <>t__builder.SetStateMachine(stateMachine);
        }

        void IAsyncStateMachine.SetStateMachine(IAsyncStateMachine stateMachine)
        {
            //ILSpy generated this explicit interface implementation from .override directive in SetStateMachine
            this.SetStateMachine(stateMachine);
        }
    }

    [StructLayout(LayoutKind.Auto)]
    [CompilerGenerated]
    private struct <<M2>g___|1_0>d : IAsyncStateMachine
    {
        public int <>1__state;

        public AsyncTaskMethodBuilder <>t__builder;

        private YieldAwaitable.YieldAwaiter <>u__1;

        private void MoveNext()
        {
            int num = <>1__state;
            try
            {
                YieldAwaitable.YieldAwaiter awaiter;
                if (num != 0)
                {
                    awaiter = Task.Yield().GetAwaiter();
                    if (!awaiter.IsCompleted)
                    {
                        num = (<>1__state = 0);
                        <>u__1 = awaiter;
                        <>t__builder.AwaitUnsafeOnCompleted(ref awaiter, ref this);
                        return;
                    }
                }
                else
                {
                    awaiter = <>u__1;
                    <>u__1 = default(YieldAwaitable.YieldAwaiter);
                    num = (<>1__state = -1);
                }
                awaiter.GetResult();
            }
            catch (Exception exception)
            {
                <>1__state = -2;
                <>t__builder.SetException(exception);
                return;
            }
            <>1__state = -2;
            <>t__builder.SetResult();
        }

        void IAsyncStateMachine.MoveNext()
        {
            //ILSpy generated this explicit interface implementation from .override directive in MoveNext
            this.MoveNext();
        }

        [DebuggerHidden]
        private void SetStateMachine(IAsyncStateMachine stateMachine)
        {
            <>t__builder.SetStateMachine(stateMachine);
        }

        void IAsyncStateMachine.SetStateMachine(IAsyncStateMachine stateMachine)
        {
            //ILSpy generated this explicit interface implementation from .override directive in SetStateMachine
            this.SetStateMachine(stateMachine);
        }
    }

    [AsyncStateMachine(typeof(<M1>d__0))]
    public Task M1(object input)
    {
        <M1>d__0 stateMachine = default(<M1>d__0);
        stateMachine.<>t__builder = AsyncTaskMethodBuilder.Create();
        stateMachine.input = input;
        stateMachine.<>1__state = -1;
        stateMachine.<>t__builder.Start(ref stateMachine);
        return stateMachine.<>t__builder.Task;
    }

    public Task M2(object input)
    {
        if (input == null)
        {
            throw new ArgumentNullException("input");
        }
        return <M2>g___|1_0();
    }

    [AsyncStateMachine(typeof(<<M2>g___|1_0>d))]
    [CompilerGenerated]
    internal static Task <M2>g___|1_0()
    {
        <<M2>g___|1_0>d stateMachine = default(<<M2>g___|1_0>d);
        stateMachine.<>t__builder = AsyncTaskMethodBuilder.Create();
        stateMachine.<>1__state = -1;
        stateMachine.<>t__builder.Start(ref stateMachine);
        return stateMachine.<>t__builder.Task;
    }
}

You can see that M1 creates and starts the state machine and the argument validation happens within the initial state, which is then guarded by a try-catch that captures any thrown exception and returns. In contrast, M2 does the argument validation and then proceeds to calls another method (<M2>g___|1_0) that sets up and starts the state machine.

[atifaziz]

While @viceroypenguin and @i3arnon have provided excellent insights already, see also my answer to the question “Can anyone explain this enumerator syntax?” on StackOveflow.

The new parameter null-checking syntactic sugar (!!) (in preview at the moment) will probably make the local function split obsolete for most cases and then iterators could be defined like regular methods once again.

[atifaziz]

The new parameter null-checking syntactic sugar (!!) (in preview at the moment) will probably make the local function split obsolete for most cases and then iterators could be defined like regular methods once again.

For better or worse, this ain't happening. Parameter null-checking or bang-bang (!!) has been removed from C# 11.