Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Xamarin Android: Exceptions thrown if SDK Assemblies are linked, after updating all NuGet-Packages #12460

Closed
daniels7 opened this issue Jun 25, 2018 · 21 comments

Comments

@daniels7
Copy link

daniels7 commented Jun 25, 2018

If I build a Xamarin Android App with linked SDK Assemblies it will crash after EF Core is Updated to newest version.

Exception message:
Stack trace:
06-25 16:48:31.244  9279  9279 I MonoDroid: UNHANDLED EXCEPTION:
06-25 16:48:31.257  9279  9279 I MonoDroid: System.TypeInitializationException: The type initializer for 'Microsoft.EntityFrameworkCore.Sqlite.Query.ExpressionTranslators.Internal.SqliteMathTranslator' threw an exception. ---> System.ArgumentException: An item with the same key has already been added. Key: Int32 Abs(Int32)
06-25 16:48:31.257  9279  9279 I MonoDroid:   at System.Collections.Generic.Dictionary`2[TKey,TValue].TryInsert (TKey key, TValue value, System.Collections.Generic.InsertionBehavior behavior) <0xceeb8b14 + 0x0045c> in <c099d544051e40b89a67a87a43581f01>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at System.Collections.Generic.Dictionary`2[TKey,TValue].Add (TKey key, TValue value) <0xceeb7fd0 + 0x00013> in <c099d544051e40b89a67a87a43581f01>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Microsoft.EntityFrameworkCore.Sqlite.Query.ExpressionTranslators.Internal.SqliteMathTranslator..cctor () <0xcd753afc + 0x00207> in <135a24207446461281bfc2c90f71e590>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:    --- End of inner exception stack trace ---
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Microsoft.EntityFrameworkCore.Query.ExpressionTranslators.RelationalCompositeMethodCallTranslator+<>c__DisplayClass5_0.<Translate>b__0 (Microsoft.EntityFrameworkCore.Query.ExpressionTranslators.IMethodCallTranslator translator) <0xcd61f2dc + 0x00044> in <b4c4c93ef655443999d2211e0c2e61cd>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at System.Linq.Enumerable+SelectListIterator`2[TSource,TResult].MoveNext () <0xce42d194 + 0x000ff> in <dd44b4af145c4bd6a0a60da9e4b5131f>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at System.Linq.Enumerable.TryGetFirst[TSource] (System.Collections.Generic.IEnumerable`1[T] source, System.Func`2[T,TResult] predicate, System.Boolean& found) <0xce415f5c + 0x000f3> in <dd44b4af145c4bd6a0a60da9e4b5131f>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at System.Linq.Enumerable.FirstOrDefault[TSource] (System.Collections.Generic.IEnumerable`1[T] source, System.Func`2[T,TResult] predicate) <0xce415c6c + 0x0003f> in <dd44b4af145c4bd6a0a60da9e4b5131f>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Microsoft.EntityFrameworkCore.Query.ExpressionTranslators.RelationalCompositeMethodCallTranslator.Translate (System.Linq.Expressions.MethodCallExpression methodCallExpression, Microsoft.EntityFrameworkCore.Metadata.IModel model) <0xcd61ef6c + 0x002bf> in <b4c4c93ef655443999d2211e0c2e61cd>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Microsoft.EntityFrameworkCore.Query.ExpressionVisitors.SqlTranslatingExpressionVisitor.VisitMethodCall (System.Linq.Expressions.MethodCallExpression methodCallExpression) <0xcd614aac + 0x00393> in <b4c4c93ef655443999d2211e0c2e61cd>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at System.Linq.Expressions.MethodCallExpression.Accept (System.Linq.Expressions.ExpressionVisitor visitor) <0xce4a5000 + 0x0001b> in <dd44b4af145c4bd6a0a60da9e4b5131f>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at System.Linq.Expressions.ExpressionVisitor.Visit (System.Linq.Expressions.Expression node) <0xce49f890 + 0x00027> in <dd44b4af145c4bd6a0a60da9e4b5131f>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Remotion.Linq.Parsing.ThrowingExpressionVisitor.Visit (System.Linq.Expressions.Expression expression) <0xce62b070 + 0x0006b> in <1081ac98085b4c0b96c42f3872e13655>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Microsoft.EntityFrameworkCore.Query.ExpressionVisitors.SqlTranslatingExpressionVisitor.Visit (System.Linq.Expressions.Expression expression) <0xcd613b00 + 0x0010f> in <b4c4c93ef655443999d2211e0c2e61cd>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Microsoft.EntityFrameworkCore.Query.RelationalQueryModelVisitor.VisitWhereClause (Remotion.Linq.Clauses.WhereClause whereClause, Remotion.Linq.QueryModel queryModel, System.Int32 index) <0xcd5f7064 + 0x000ef> in <b4c4c93ef655443999d2211e0c2e61cd>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Remotion.Linq.Clauses.WhereClause.Accept (Remotion.Linq.IQueryModelVisitor visitor, Remotion.Linq.QueryModel queryModel, System.Int32 index) <0xce63c7cc + 0x00093> in <1081ac98085b4c0b96c42f3872e13655>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Remotion.Linq.QueryModelVisitorBase.VisitBodyClauses (System.Collections.ObjectModel.ObservableCollection`1[T] bodyClauses, Remotion.Linq.QueryModel queryModel) <0xce62e964 + 0x00167> in <1081ac98085b4c0b96c42f3872e13655>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Remotion.Linq.QueryModelVisitorBase.VisitQueryModel (Remotion.Linq.QueryModel queryModel) <0xce62e468 + 0x0006b> in <1081ac98085b4c0b96c42f3872e13655>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Microsoft.EntityFrameworkCore.Query.EntityQueryModelVisitor.VisitQueryModel (Remotion.Linq.QueryModel queryModel) <0xce6e2e08 + 0x00047> in <1a963d6bdb8d435e8dcdc28935e050ad>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Microsoft.EntityFrameworkCore.Query.RelationalQueryModelVisitor.VisitQueryModel (Remotion.Linq.QueryModel queryModel) <0xcd5f5da0 + 0x0005f> in <b4c4c93ef655443999d2211e0c2e61cd>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Microsoft.EntityFrameworkCore.Query.EntityQueryModelVisitor.CreateQueryExecutor[TResult] (Remotion.Linq.QueryModel queryModel) <0xce6e0fcc + 0x00177> in <1a963d6bdb8d435e8dcdc28935e050ad>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Microsoft.EntityFrameworkCore.Storage.Database.CompileQuery[TResult] (Remotion.Linq.QueryModel queryModel) <0xce6c8964 + 0x000bb> in <1a963d6bdb8d435e8dcdc28935e050ad>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.CompileQueryCore[TResult] (System.Linq.Expressions.Expression query, Microsoft.EntityFrameworkCore.Query.Internal.IQueryModelGenerator queryModelGenerator, Microsoft.EntityFrameworkCore.Storage.IDatabase database, Microsoft.EntityFrameworkCore.Diagnostics.IDiagnosticsLogger`1[TLoggerCategory] logger, System.Type contextType) <0xce70f75c + 0x0028f> in <1a963d6bdb8d435e8dcdc28935e050ad>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler+<>c__DisplayClass13_0`1[TResult].<Execute>b__0 () <0xce71037c + 0x0005b> in <1a963d6bdb8d435e8dcdc28935e050ad>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQueryCore[TFunc] (System.Object cacheKey, System.Func`1[TResult] compiler) <0xce6f70ac + 0x000c3> in <1a963d6bdb8d435e8dcdc28935e050ad>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQuery[TResult] (System.Object cacheKey, System.Func`1[TResult] compiler) <0xce6f7024 + 0x0003b> in <1a963d6bdb8d435e8dcdc28935e050ad>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.Execute[TResult] (System.Linq.Expressions.Expression query) <0xce70f47c + 0x001a7> in <1a963d6bdb8d435e8dcdc28935e050ad>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryProvider.Execute[TResult] (System.Linq.Expressions.Expression expression) <0xce6f7b58 + 0x00047> in <1a963d6bdb8d435e8dcdc28935e050ad>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at System.Linq.Queryable.FirstOrDefault[TSource] (System.Linq.IQueryable`1[T] source) <0xce4096f4 + 0x000cb> in <dd44b4af145c4bd6a0a60da9e4b5131f>:0

Steps to reproduce

I'll try to append a sample project tomorrow, though I somehow have the thought this might be somehow related to Issue #10207 ?

Does anyone have an idea on how to modify the linking behaviour so that the Application will work again? Without linking the resulting apk sadly is 3 times as big as usual.

Further technical details

EF Core version: (found in project.csproj or packages.config) 2.1.1
Database Provider: (e.g. Microsoft.EntityFrameworkCore.SqlServer) Microsoft.EntityFrameworkCore.Sqlite
Operating system: Windows 10 Pro x64
IDE: (e.g. Visual Studio 2017 15.4) VS 2017 15.7.4

@marfh
Copy link

marfh commented Jun 27, 2018

I have the same issue, works fine in 2.0.3 but not 2.1.0 or 2.1.1.

@ajcvickers
Copy link
Contributor

@daniels7 @dwaskel It would be great if you could post a sample project/solution that demonstrates this.

@cwrea Does this look like something you have run into?

@mfeingol
Copy link

I'm seeing the same thing. I don't have a simple repro on hand, but willing to provide access to a private VSTS repo to the right person.

@BarretHTW
Copy link

I have this problem as well, resulting in a too big apk...

@daniels7
Copy link
Author

daniels7 commented Jul 9, 2018

@ajcvickers

Issue12460.zip

Here is a sample project as requested. Just build it in Release-Mode to install on an Android-Phone and it crashes instantly.

Please fix this asap, thank you very much

@michaelrinderle
Copy link

im running into the same issue. release apk for google play ends up at ~90mb. eek!

proposed solutions didnt work for me, i.e. linkdescription.txt, --skip-link, etc

@daniels7
Copy link
Author

@ajcvickers
Any reaction from you or any other member of the team would be very nice.
This Bug is critical and fucks up every new release of our App because it is three times as big as it should be.

Come on, any reaction....

It's been 2 days since you got a sample project by me and more than two weeks since the Bug has been reported by multiple users.

@BarretHTW
Copy link

???

@divega divega self-assigned this Jul 14, 2018
@divega divega added this to the 2.2.0 milestone Jul 14, 2018
@divega
Copy link
Contributor

divega commented Jul 14, 2018

Everyone, sorry for the delay in answering. I have assigned this to @smitpatel and myself. I would like to ask from the people that repro if just running this code throws if you copy it inside your app:

    var supportedMethods = new Dictionary<MethodInfo, string>
        {
            { typeof(Math).GetMethod(nameof(Math.Abs), new[] { typeof(double) }), "abs" },
            { typeof(Math).GetMethod(nameof(Math.Abs), new[] { typeof(float) }), "abs" },
            { typeof(Math).GetMethod(nameof(Math.Abs), new[] { typeof(int) }), "abs" },
            { typeof(Math).GetMethod(nameof(Math.Abs), new[] { typeof(long) }), "abs" },
            { typeof(Math).GetMethod(nameof(Math.Abs), new[] { typeof(sbyte) }), "abs" },
            { typeof(Math).GetMethod(nameof(Math.Abs), new[] { typeof(short) }), "abs" },
            { typeof(Math).GetMethod(nameof(Math.Max), new[] { typeof(byte), typeof(byte) }), "max" },
            { typeof(Math).GetMethod(nameof(Math.Max), new[] { typeof(double), typeof(double) }), "max" },
            { typeof(Math).GetMethod(nameof(Math.Max), new[] { typeof(float), typeof(float) }), "max" },
            { typeof(Math).GetMethod(nameof(Math.Max), new[] { typeof(int), typeof(int) }), "max" },
            { typeof(Math).GetMethod(nameof(Math.Max), new[] { typeof(long), typeof(long) }), "max" },
            { typeof(Math).GetMethod(nameof(Math.Max), new[] { typeof(sbyte), typeof(sbyte) }), "max" },
            { typeof(Math).GetMethod(nameof(Math.Max), new[] { typeof(short), typeof(short) }), "max" },
            { typeof(Math).GetMethod(nameof(Math.Max), new[] { typeof(uint), typeof(uint) }), "max" },
            { typeof(Math).GetMethod(nameof(Math.Max), new[] { typeof(ushort), typeof(ushort) }), "max" },
            { typeof(Math).GetMethod(nameof(Math.Min), new[] { typeof(byte), typeof(byte) }), "min" },
            { typeof(Math).GetMethod(nameof(Math.Min), new[] { typeof(double), typeof(double) }), "min" },
            { typeof(Math).GetMethod(nameof(Math.Min), new[] { typeof(float), typeof(float) }), "min" },
            { typeof(Math).GetMethod(nameof(Math.Min), new[] { typeof(int), typeof(int) }), "min" },
            { typeof(Math).GetMethod(nameof(Math.Min), new[] { typeof(long), typeof(long) }), "min" },
            { typeof(Math).GetMethod(nameof(Math.Min), new[] { typeof(sbyte), typeof(sbyte) }), "min" },
            { typeof(Math).GetMethod(nameof(Math.Min), new[] { typeof(short), typeof(short) }), "min" },
            { typeof(Math).GetMethod(nameof(Math.Min), new[] { typeof(uint), typeof(uint) }), "min" },
            { typeof(Math).GetMethod(nameof(Math.Min), new[] { typeof(ushort), typeof(ushort) }), "min" },
            { typeof(Math).GetMethod(nameof(Math.Round), new[] { typeof(double) }), "round" },
            { typeof(Math).GetMethod(nameof(Math.Round), new[] { typeof(double), typeof(int) }), "round" }
        };

This is a copy of the code from which the exception is coming. It is creating a dictionary where the key is a MethodInfo. Since it is saying that Abs(int32) is a duplicate key, that means that either typeof(Math).GetMethod(nameof(Math.Abs), new[] { typeof(double) }) or typeof(Math).GetMethod(nameof(Math.Abs), new[] { typeof(float) }) returned the same method info as typeof(Math).GetMethod(nameof(Math.Abs), new[] { typeof(int) }), which seems very unexpected.

Even if the linker is removing some members, that doesn't seem to be an excuse for reflection to start doing fuzzy matching. That said, I am not super familiar with the technology and there could be a reason for this. In any case, I would say that part is an issue in Xamarin/Mono and I would be grateful if you can report it at https://github.com/xamarin/xamarin-android/issues.

Re how to workaround this, I would guess that using the options in the linker to skip linking certain assemblies. E.g. as documented in https://docs.microsoft.com/en-us/xamarin/ios/deploy-test/linker?tabs=vsmac, you could either use:

--linkskip=NameOfAssemblyToSkipWithoutFileExtension

Or a custom link configuration as described at https://docs.microsoft.com/en-us/xamarin/cross-platform/deploy-test/linker.

I am not sure what assembly/assemblies you should be excluding to make sure all overoads of Math.Abs are preserved. It could be mscorlib, in which case your LinkDescription.xml file woudl look something like this (I am basing this on @cwrea's sample app LinkDescription.xml file):

  <assembly fullname="mscorlib">
    <type fullname="System.String">
      <method name="Compare"></method>
      <method name="CompareTo"></method>
      <method name="ToUpper"></method>
      <method name="ToLower"></method>
    </type>
     <type fullname="System.Math">
      <method name="Abs"></method>
      <method name="Max"></method>
      <method name="Min"></method>
      <method name="Round"></method>
    </type>
 </assembly>
  <assembly fullname="System.Core">
    <type fullname="System.Linq.Expressions.Expression`1"></type>
    <type fullname="System.Linq.Queryable"></type>
  </assembly>
</linker>

Also, I haven't tried this workaround myself on Xamarin Android, so I am not sure with just specifying the names of the methods in the file is enough to preserve all overloads. Worst case, you may need to specify all the signatures (which is also documented in the previous link).

@divega
Copy link
Contributor

divega commented Jul 14, 2018

@smitpatel could you try to repro this and try the workaround when you have a chance?

@michaelrinderle
Copy link

michaelrinderle commented Jul 16, 2018

these solutions don't work. trying to skip assemblies is the worst, you keep compiling, marking an assembly/namespace in ef core until you run up against an assembly that just work link correctly and you're stuck after hours of work for nothing.

as i recall xamarin pushed this back on ef core team since they have to mark reflection that needs to be kept in source so this doesnt happen.

ef core and xamarin are a ms flagship now, and you can't link with the ef assm and end up with a bloated apk on top of the mono framework because of it? this kind of stinks!

@divega
Copy link
Contributor

divega commented Jul 16, 2018

@ainsophical just to double-check, when you say that these solution's don't work, do you mean that you don't think this is the right long term solution or that you tried the workaround and it really didn't work?

FWIW, we also don't think that this is the ideal long term solution and that is why #10963 is still open. However all the proposals from Xamarin and experts in that issue are about moving the workarounds to EF Core and potentially all its dependencies. For example, we would ship our own LinkDescription.xml and/or annotate members in EF Core, Microsoft.Extensions.* and .NET Core BCL with PreserveAttribute.

If the workaround I suggested doesn't work at all, that is very important to know because it means that this particular issue is caused by a different problem and moving the workarounds into EF Core and its dependencies wouldn't help.

@michaelrinderle
Copy link

i've tried everything suggested on the topic from multiple sources including the above. I included my own linkdescription.xml file and started the compile process to see which assem. where getting stripped out during the linking process; the conclusion was that at a certain point i couldnt get ef core to work on my xam. app with linking (sdk / sdk & user assem.) since I was getting an exception for a reference to an assembly despite it being on my xml file. i dont recall the referenced assembly at the moment but it was in the ef namespace.

@david-bauduin
Copy link

david-bauduin commented Jul 19, 2018

I rolled back to 2.03 and work fine with <AndroidLinkMode>SdkOnly</AndroidLinkMode>
->
<PackageReference Include="Microsoft.EntityFrameworkCore"> <Version>2.0.3</Version> </PackageReference> <PackageReference Include="Microsoft.EntityFrameworkCore.Sqlite"> <Version>2.0.3</Version> </PackageReference>

Waiting for the fix now !

@divega
Copy link
Contributor

divega commented Jul 23, 2018

I found that the issue hasn't been reported to Xamarin so I created dotnet/android#1990. I would suggest everyone that reported the issue here to track that one.

@divega
Copy link
Contributor

divega commented Jul 27, 2018

From dotnet/android#1990 (comment)

@JonDouglas commented
I can confirm the behavior outlined in the reproduction steps on Xamarin.Android 8.3.3.2:

07-26 15:56:50.903 29686-29686/? E/AndroidRuntime: FATAL EXCEPTION: main
   Process: Issue12460.Issue12460, PID: 29686
   android.runtime.JavaProxyThrowable: System.TypeInitializationException: The type initializer for 'Microsoft.EntityFrameworkCore.Sqlite.Query.ExpressionTranslators.Internal.SqliteCompositeMethodCallTranslator' threw an exception. ---> System.TypeInitializationException: The type initializer for 'Microsoft.EntityFrameworkCore.Sqlite.Query.ExpressionTranslators.Internal.SqliteMathTranslator' threw an exception. ---> System.ArgumentException: An item with the same key has already been added. Key: Double Abs(Double)

However, the behavior does not exhibit on Xamarin.Android 9.0.0.15. Perhaps the bump to Mono 5.12 resolved this issue upstream @jonpryor ?
@divega Have you tried 15.8 Previews(4 or 5) with this issue? I can't seem to reproduce there.

@daniels7 @dwaskel @mfeingol @BarretHTW @stddd @ainsophical and anyone that has hit the issue, could you please answer @JonDouglas's question on whether this repros for you using 15.8 Previews 4 or 5?

@divega
Copy link
Contributor

divega commented Jul 31, 2018

Per the previous comment, upgrading to Xamarin.Android 9 fixes this issue. In the meanwhile, downgrading to a previous build of EF Core can serve as a workaround.

Closing the issue, since there doesn't seem to be anything actionable on the EF Core side.

@divega divega removed this from the 2.2.0 milestone Jul 31, 2018
@mister-giga
Copy link

@divega I am targeting Android 9, but this error still appears for me.

@michaelrinderle
Copy link

michaelrinderle commented Apr 16, 2019

what IDE and version are you using? this problem went away for me a long time ago.
more importantly, what version of EF core are you on?

@mister-giga
Copy link

@ainsophical I am using visual studio 2019 for mac. I was using 2.2.4 version of packages.
I am stuck with version 2.0.3 for months already.

@divega
Copy link
Contributor

divega commented Apr 16, 2019

@mister-giga just in case it helps, there was a very similar exception reported becasue the linker removes System.Date. The workaround is described at dotnet/android#2620 (comment).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests