Exception in Client vs. Server Evaluation with async/await in EF Core 2.0.0

[JonPSmith]

I upgraded my application to version EF Core 2.0.0 and I have problems running some of my queries with async - throw an exception. This query worked in EF Core 1.1.0.

I have narrowed the problem down to areas that use Client vs. Server Evaluation that involve a collection navigation property. I have two parts - one which produces an exception and one that hangs.

I will describe the one that throws an exception - If you need the information on the one that hangs I can provide that as well.

Stack trace

System.ArgumentException : Expression of type 'System.Collections.Generic.IAsyncEnumerable`1[DataLayer.EfClasses.BookAuthor]' cannot be used for parameter of type 'System.Collections.Generic.IEnumerable`1[System.Object]' of method 'System.Collections.Generic.ICollection`1[DataLayer.EfClasses.BookAuthor] MaterializeCollectionNavigation[BookAuthor](Microsoft.EntityFrameworkCore.Metadata.INavigation, System.Collections.Generic.IEnumerable`1[System.Object])'
Parameter name: arg1
   at System.Dynamic.Utils.ExpressionUtils.ValidateOneArgument(MethodBase method, ExpressionType nodeKind, Expression arguments, ParameterInfo pi, String methodParamName, String argumentParamName, Int32 index)
   at System.Linq.Expressions.Expression.Call(MethodInfo method, Expression arg0, Expression arg1)
   at System.Linq.Expressions.MethodCallExpression2.Rewrite(Expression instance, IReadOnlyList`1 args)
   at System.Linq.Expressions.ExpressionVisitor.VisitMethodCall(MethodCallExpression node)
   at Microsoft.EntityFrameworkCore.Query.ExpressionVisitors.RelationalEntityQueryableExpressionVisitor.VisitMethodCall(MethodCallExpression node)
   at System.Linq.Expressions.MethodCallExpression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at Microsoft.EntityFrameworkCore.Query.EntityQueryModelVisitor.ReplaceClauseReferences(Expression expression, IQuerySource querySource, Boolean inProjection)
   at Microsoft.EntityFrameworkCore.Query.EntityQueryModelVisitor.CompileMainFromClauseExpression(MainFromClause mainFromClause, QueryModel queryModel)
   at Microsoft.EntityFrameworkCore.Query.RelationalQueryModelVisitor.CompileMainFromClauseExpression(MainFromClause mainFromClause, QueryModel queryModel)
   at Microsoft.EntityFrameworkCore.Query.EntityQueryModelVisitor.VisitMainFromClause(MainFromClause fromClause, QueryModel queryModel)
   at Remotion.Linq.Clauses.MainFromClause.Accept(IQueryModelVisitor visitor, QueryModel queryModel)
   at Remotion.Linq.QueryModelVisitorBase.VisitQueryModel(QueryModel queryModel)
   at Microsoft.EntityFrameworkCore.Query.EntityQueryModelVisitor.VisitQueryModel(QueryModel queryModel)
   at Microsoft.EntityFrameworkCore.Query.RelationalQueryModelVisitor.VisitQueryModel(QueryModel queryModel)
   at Microsoft.EntityFrameworkCore.Query.ExpressionVisitors.ProjectionExpressionVisitor.VisitSubQuery(SubQueryExpression expression)
   at Remotion.Linq.Clauses.Expressions.SubQueryExpression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at Microsoft.EntityFrameworkCore.Query.ExpressionVisitors.RelationalProjectionExpressionVisitor.Visit(Expression expression)
   at System.Dynamic.Utils.ExpressionVisitorUtils.VisitArguments(ExpressionVisitor visitor, IArgumentProvider nodes)
   at System.Linq.Expressions.ExpressionVisitor.VisitMethodCall(MethodCallExpression node)
   at Microsoft.EntityFrameworkCore.Query.ExpressionVisitors.RelationalProjectionExpressionVisitor.VisitMethodCall(MethodCallExpression methodCallExpression)
   at System.Linq.Expressions.MethodCallExpression.Accept(ExpressionVisitor visitor)
   at System.Linq.Expressions.ExpressionVisitor.Visit(Expression node)
   at Microsoft.EntityFrameworkCore.Query.ExpressionVisitors.RelationalProjectionExpressionVisitor.Visit(Expression expression)
   at Microsoft.EntityFrameworkCore.Query.EntityQueryModelVisitor.VisitSelectClause(SelectClause selectClause, QueryModel queryModel)
   at Microsoft.EntityFrameworkCore.Query.RelationalQueryModelVisitor.VisitSelectClause(SelectClause selectClause, QueryModel queryModel)
   at Remotion.Linq.Clauses.SelectClause.Accept(IQueryModelVisitor visitor, QueryModel queryModel)
   at Remotion.Linq.QueryModelVisitorBase.VisitQueryModel(QueryModel queryModel)
   at Microsoft.EntityFrameworkCore.Query.EntityQueryModelVisitor.VisitQueryModel(QueryModel queryModel)
   at Microsoft.EntityFrameworkCore.Query.RelationalQueryModelVisitor.VisitQueryModel(QueryModel queryModel)
   at Microsoft.EntityFrameworkCore.Query.EntityQueryModelVisitor.CreateAsyncQueryExecutor[TResult](QueryModel queryModel)
   at Microsoft.EntityFrameworkCore.Storage.Database.CompileAsyncQuery[TResult](QueryModel queryModel)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.CompileAsyncQueryCore[TResult](Expression query, INodeTypeProvider nodeTypeProvider, IDatabase database)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.<>c__DisplayClass24_0`1.<CompileAsyncQuery>b__0()
   at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQueryCore[TFunc](Object cacheKey, Func`1 compiler)
   at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddAsyncQuery[TResult](Object cacheKey, Func`1 compiler)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.CompileAsyncQuery[TResult](Expression query)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.ExecuteAsync[TResult](Expression query)
   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryProvider.ExecuteAsync[TResult](Expression expression)
   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryable`1.System.Collections.Generic.IAsyncEnumerable<TResult>.GetEnumerator()
   at System.Linq.AsyncEnumerable.<Aggregate_>d__6`3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at test.UnitTests.DataLayer.Ch05_AsyncAwait.<RunClientServerCollectionAsync>d__1.MoveNext() in C:\Users\jonsm\Documents\Visual Studio 2017\Projects\EfCoreInAction\Test\UnitTests\DataLayer\Ch05_AsyncAwait.cs:line 52
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

Steps to reproduce

I have a Book entity class with a Many-to-Many relationship to an Author entity class, via a BookAuthor entity class linking table. I am simply trying to get a comma separated list of all the authors of a book. (see my issue #9519, as this is using the same code).

My entity classes are:

public class Book                                   
{
    public int BookId { get; set; } 
    public string Title { get; set; }
    public string Description { get; set; }
    public DateTime PublishedOn { get; set; }
    public string Publisher { get; set; }
    public decimal Price { get; set; }
    public string ImageUrl { get; set; }

    //----------------------------------------------
    //relationships

    public PriceOffer Promotion { get; set; }       
    public ICollection<Review> Reviews { get; set; }
    public ICollection<BookAuthor> AuthorsLink { get; set; }                   
}

public class Author                   
{
    public int AuthorId { get; set; }
    public string Name { get; set; }

    //------------------------------
    //Relationships

    public ICollection<BookAuthor> 
        BooksLink { get; set; }       
}

Linking Table is

public class BookAuthor               
{
    public int BookId { get; set; }   
    public int AuthorId { get; set; } 
    public byte Order { get; set; }   

    //-----------------------------
    //Relationships

    public Book Book { get; set; }    
    public Author Author { get; set; }
}

The unit test contains just the element of the query that causes the exception.
Note: The method SeedDatabaseFourBooks adds four books, each with one Author linked via one BookAuthor table.

[Fact]
public async Task RunClientServerCollectionAsync()
{
    //SETUP
    var inMemDb = new SqliteInMemory();

    using (var context = inMemDb.GetContextWithSetup())
    {
        context.SeedDatabaseFourBooks();

        //ATTEMPT
        var dtos = await context.Books
            .Select(p => 
                string.Join(", ",
                    p.AuthorsLink
                        .Select(q => q.Author.Name))).ToListAsync();

        //VERIFY
        dtos.Count.ShouldEqual(4);
    }
}

Further technical details

EF Core version: 2.0.0 (worked in 1.1.0)
Database Provider: Microsoft.EntityFrameworkCore.Sqlite
Operating system: Windows 10
IDE: VS2017 15.3

[ajcvickers]

@anpete Can you write risk/justification for this one?

[anpete]

@ajcvickers Here you go:

Risk: Low: Adds a small amount of new code to the query compiler that detects the offending query patterns and performs a transformation from sync to async operators.
Justification: Regression. Fixes a crash that can occur in reasonably common customer async scenarios. After the fix, such queries become fully async, which is the ideal behavior.

[Eilon]

This patch bug is approved for the 2.0.x patch. Please send a PR to the feature/2.0.1 branch and get it reviewed and merged. When we have the rel/2.0.1 branches ready please port the commit to that branch.

[Eilon]

Hi, we have a public test feed that you can use to try out the ASP.NET/EF Core 2.0.3 patch!

To try out the pre-release patch, please refer to the following guide:

• ASP.NET/EF Core 2.0.3 patch pre-release test feed aspnet/Announcements#274 (info)
• ASP.NET/EF Core 2.0.3 patch pre-release test feed - discussion aspnetcore#2254 (discussions and questions)

We are looking for feedback on this patch. We'd like to know if you have any issues with this patch by updating your apps and libraries to the latest packages and seeing if it fixes the issues you've had, or if it introduces any new issues. If you have any issues or questions, please reply on this issue to let us know as soon as possible.

Thanks,
Eilon

[chris31389]

Hi @Eilon @anpete ,

I've come across this bug today. I was running this code:

public class Request
{
    public Guid Id { get; set; }
    public List<Data> Data { get; set; }
}

public class Data
{
    public Guid Id { get; set; }
    public string Key { get; set; }
    public string ValueAsString { get; set; }
    public DateTime? ValueAsDateTime { get; set; }
    public int? ValueAsInt { get; set; }
    public decimal? ValueAsDecimal { get; set; }
}

int count = await dbContext.Set<Request>.Include(x => x.Data)
    .Where(x => (x.Data.FirstOrDefault(y => y.Key == "supplier").ValueAsString ?? "").StartsWith("an"))
    .CountAsync()

When I use .UseInMemoryDatabase and run tests it works fine, however when I run .UseSqlServer it throws

Is there a known workaround?

How long until the fix is publicly released? (I've seen this is set for 2.0.3, yet I can't see 2.0.1 on the nuget feed?)

[Eilon]

@chris31389 we skipped over 2.0.1 and 2.0.2 so the first patch will officially by called 2.0.3. The patch is nearly ready for release - it's scheduled for November.