Substitute.For delegate much slower than for interfaces and classes

[jcansdale]

First, congratulations on a very slick project. :)

I've been doing some performance tests using BenchmarkDotNet and noticed that substituting delegates is much slower than for other types (something like 80 times slower). I wondered if you know why this might be and if it could be improved?

Here are the results and benchmarking code:

Host Process Environment Information:
BenchmarkDotNet.Core=v0.9.9.0
OS=Microsoft Windows NT 6.2.9200.0
Processor=Intel(R) Core(TM) i7-4500U CPU 1.80GHz, ProcessorCount=4
Frequency=2338341 ticks, Resolution=427.6536 ns, Timer=TSC
CLR=MS.NET 4.0.30319.42000, Arch=32-bit ?
GC=Concurrent Workstation
JitModules=clrjit-v4.6.1586.0

Type=NSubstituteBenchmark  Mode=Throughput  LaunchCount=1  
WarmupCount=3  TargetCount=3  

Method	Median	StdDev	Scaled	Scaled-SD	Min	Max
SubstituteForInterface	6.3367 us	0.0920 us	1.00	0.00	6.2362 us	6.4199 us
SubstituteForClass	6.5393 us	0.0584 us	1.03	0.01	6.4411 us	6.5448 us
SubstituteForDelegate	545.3938 us	53.6336 us	83.17	6.99	465.9712 us	568.1222 us

namespace StaticMocks.Benchmarks
{
    using System;
    using NSubstitute;
    using BenchmarkDotNet.Attributes;
    using BenchmarkDotNet.Attributes.Columns;

    [MinColumn, MaxColumn]
    public class NSubstituteBenchmark
    {
        [Benchmark]
        public object SubstituteForInterface()
        {
            return Substitute.For<IServiceProvider>();
        }

        [Benchmark]
        public object SubstituteForClass()
        {
            return Substitute.For<Type>();
        }

        [Benchmark]
        public object SubstituteForDelegate()
        {
            return Substitute.For<Action>();
        }
    }
}

[jcansdale]

I'm wondering if the commonly used Action and Func delegates could be special cased? I can imagine that substituting an unknown delegate might be tricky.

[dtchepak]

Hi Jamie,
Thanks a lot for this. The DelegateProxyFactory does a lot of extra work. I think special cases would help. Possibly adding some caching of the proxy expression too.

[alexandrnikitin]

It spends most of the time in System.Reflection compiling the delegate

There are faster compilers e.g. https://github.com/dadhi/FastExpressionCompiler but I'm not sure we should touch it at all. What do you think?

[jcansdale]

I now have a workaround that lets me substitute an interface rather than a delegate (which is ~80 times faster).

It looks something like this:

        static Delegate createDelegateFor(Type delegateType)
        {
            var dele = quickCreateDelegateFor(delegateType);
            if(dele != null)
            {
                return dele;
            }

            return (Delegate)Substitute.For(new Type[] { delegateType }, new object[0]);
        }

        // TODO: Add support for all Actions and Funcs.
        static Delegate quickCreateDelegateFor(Type delegateType)
        {
            Type type;
            switch (delegateType.Name)
            {
                case "Func`1":
                    type = typeof(IFunc<>).MakeGenericType(delegateType.GenericTypeArguments);
                    break;
                case "Func`2":
                    type = typeof(IFunc<,>).MakeGenericType(delegateType.GenericTypeArguments);
                    break;
                default:
                    return null;
            }

            var target = Substitute.For(new Type[] { type }, new object[0]);
            var invokeMethod = type.GetTypeInfo().GetMethod("Invoke");
            return invokeMethod.CreateDelegate(delegateType, target);
        }

        public interface IFunc<R> { R Invoke(); }

        public interface IFunc<T, R> { R Invoke(T t); }

I'm guessing there may be dragons that I'm missing, but this seems to work for my purposes. 😄

[zvirja]

I've investigated this issue a bit and basically I see two options:

1. Use the FastExpressionCompiler mentioned above;
2. Rewrite code to use the Reflection.Emit directly.

The second option is probably more lightweight, however it will force to have a mess in our code, as it's non-supportable. Also I'm not sure that the emitted code will be 100% valid (so it doesn't fail on some Mono environment), as sometimes opcodes are very specific depending on the operand type (e.g. different opcode is used to read int, long and string argument value).

Therefore, the safest way (and the preferable one) would be to use the FastExpressionCompiler which has been already tested and proved to work. We can import the whole FastExpressionCompiler.cs file to our solution (making the declarations internal), so we will not depend on it directly.

@alexandrnikitin @dtchepak What do you think? 😉

[dtchepak]

@zvirja Sounds good 👍