Xamarin iOS : Crash when i add row to table with nullable type or with a lot of columns

[Guillaume-Fortin]

Update

According to https://bugzilla.xamarin.com/show_bug.cgi?id=59184#c57, the more complete fix in Mono 2017-12 is now merged and has been verified on Xamarin.iOS.

According to https://bugzilla.xamarin.com/show_bug.cgi?id=59184#c58, the fix should be included in the 15.7 release of Xamarin, which I believe maps to Xamarin.iOS 11.9 (at least I found that on https://github.com/xamarin/xamarin-macios/wiki).

Original issue

EntityFramework crash when i insert a row inside table. it crash only on iOS Device (iPod Touch and iPad mini 4).

I have try successfully onAndroid Device and iOS Simulator.
But when i use on iOS device, it fall to Xamarin iOS limitation.

First way : when i have a nullable type inside TrainingDayRow table (exemple : int? unit)

Exception message: 
Stack trace:

{System.ExecutionEngineException: Attempting to JIT compile method '(wrapper runtime-invoke) <Module>:runtime_invoke_void__this___object_int_int_int_int_DateTime_DateTime_DateTime_Nullable`1<int> (object,intptr,intptr,intptr)' while running in aot-only mode. See https://developer.xamarin.com/guides/ios/advanced_topics/limitations/ for more information.
at (wrapper managed-to-native) System.Reflection.MonoCMethod:InternalInvoke (System.Reflection.MonoCMethod,object,object[],System.Exception&)
at System.Reflection.MonoCMethod.InternalInvoke (System.Object obj, System.Object[] parameters) [0x00002] in /Library/Frameworks/Xamarin.iOS.framework/Versions/10.12.0.14/src/mono/mcs/class/corlib/System.Reflection/MonoMethod.cs:661 
 --- End of stack trace from previous location where exception was thrown ---

Second way : Have a table with a lot of column (here on TrainingExerciseRow with 14 columns)

Exception message: 
Stack trace:

{System.ExecutionEngineException: Attempting to JIT compile method '(wrapper runtime-invoke) <Module>:runtime_invoke_void__this___object_int_int_int_int_DateTime_DateTime_DateTime_int (object,intptr,intptr,intptr)' while running in aot-only mode. See https://developer.xamarin.com/guides/ios/advanced_topics/limitations/ for more information.
at (wrapper managed-to-native) System.Reflection.MonoCMethod:InternalInvoke (System.Reflection.MonoCMethod,object,object[],System.Exception&)
 at System.Reflection.MonoCMethod.InternalInvoke (System.Object obj, System.Object[] parameters) [0x00002] in /Library/Frameworks/Xamarin.iOS.framework/Versions/10.12.0.14/src/mono/mcs/class/corlib/System.Reflection/MonoMethod.cs:661 
--- End of stack trace from previous location where exception was thrown ---

Steps to reproduce

Clone my Github source and just click to "+".

Source code
https://github.com/Thetyne/XamarinTodo

Further technical details

EntityFrameworkCore 2.0.0-preview2-final
NetStandard 2.0.0.0-preview2-25401-01
Database Provider: Microsoft.EntityFrameworkCore.Sqlite
Operating system: iOS/Macosx
IDE: Visual Studio Community 2017 for Mac Version 7.1 Preview (7.1 build 1281)

[divega]

@Thetyne thanks for reporting this.

I have created an issue for this for Xamarin iOS at https://bugzilla.xamarin.com/show_bug.cgi?id=58424 (this issue has been resolved as a duplicate of https://bugzilla.xamarin.com/show_bug.cgi?id=59184). Hopefully we will figure out if this is something that can be addressed by either an improvement in Xamarin, a chance in EF Core or a workaround in the application code.

cc @marek-safar as he has helped with similar issues in the past.

[divega]

Note for triage: This is due to a limitation in the number of parameters for methods that are invoked dynamically in the implementation of Xamario.iOS.

From @vargaz on the Xamarin issue:

I can reproduce, it tries to do a method invoke on this method:
"Microsoft.EntityFrameworkCore.ChangeTracking.Internal.Snapshot`20<int, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string, string>:.ctor (int,string,string,string,string,string,string,string,string,string,string,string,string,string,string,string,string,string,string,string)"

xamarin.ios currently has some limits on what methods can be dynamically invoked, this method has too arguments.

The limit for parameters is about 14.

It will take some time to fix this since it is low level code and it needs to work on multiple platforms. It might be better to try to rewrite the c# code to use less arguments.

So, it seems that using a smaller value for Snapshot.MaxGenericTypes (about 14 instead of 30) could help run EF Core in Xamarin.iOS with larger entities.

More details about the limitation:

On arm64, its 8+6=14, or arm its 4+10=14. This includes the this pointer as well.

[ajcvickers]

We will add a quirk mode to workaround this issue until it is fixed in Xamarin.iOS.

[ns-wgilmour]

For anyone here looking for a quick fix on iOS, I managed to get my project working by pulling EFCore, changing Snapshot.MaxGenericTypes to 2 (I tried 14 and 5, but they didn't work, which may just be my model), building and packaging it, and using the new NuGet packages in my project.

[ajcvickers]

@ns-wgilmour When you changed MaxGenericTypes to 14 and 5, did you still see the same failure as reported above? Can you post the exception message and stack trace if there is one?

@divega It may not be feasible to go all the way down to 2 generic types. Also, at this level there may be other generics we create that have similar problems.

[ns-wgilmour]

@ajcvickers I did, although I only tested it on my current project rather than with the test project I created here to showcase the errors (I catch the errors in that test project, so don't expect it to throw). My current project would crash when trying to do anything related to a particular model with 3 optional relations in it, the keys for which are represented in the model as 3 Nullable<long>. The model has several other properties that bring its total count to 12 (that are stored in the database).

I think there is slightly more to this problem than meets the eye, or perhaps a misunderstanding of it. @divega, when commenting on the Xamarin issue, said that up to 30 generic arguments could be used, as specified by Snapshot.MaxGenericTypes, and when I first saw that, I thought it referred to the total number of properties on the model, but that did not seem to be the case with my model of 12 properties, as setting the value to 14 and 5 seemed to have no effect. I then thought it might refer to the number of properties on the model that were generically-typed, such as Nullable<>, and set it to a number that was below the number of generic types my problem model had, in this case 2, and it then worked.

Also, whilst that particular case seems to be related to generic types, in my test project there is a model with roughly 20 string properties, no Nullables or generic types, and that also fails with the same error.

I'm not sure if the circumstances are just coincidental but to me there seems to be two different causes. I am unfortunately swamped at the moment so haven't really had time to properly investigate with my test project. I must also confess I know very little about how all this reflection and dynamic invoking generically typed methods works, so my apologies if what I am saying seems nonsensical, I am just trying to put down my observations and thought process.

[divega]

I agree we don't understand the limitation in Xamarin.iOS well enough.

According to @vargaz it is about the number of arguments on a method that is invoked dynamically. The original repro hit this on the constructor of one of the Snapshot<T,...Tn> types. I.e. it is only indirectly related to the number of generic arguments, and there are other ways to hit it.

Hi hope @vargaz has more suggestions here, or information on when the limitation could be lifted.

[divega]

@ns-wgilmour maybe you can help us by providing a new repro here and in the Xamarin issue at https://bugzilla.xamarin.com/show_bug.cgi?id=58424.

[vargaz]

The limitation is on the number of arguments to the method which is invoked dynamically. There is no need for another testcase right now.

[divega]

@vargaz thanks for responding. We are trying to understand why (according to #9249 (comment)), limiting the number of generic arguments to 14, or even 5 doesn't help.

[divega]

Update: mono/mono#5612 will fix this for ARM64 devices in an upcoming release of Xamarin.iOS.

[MasanoriOnuki]

I tested dynamic invocation on iOS devices.

Test code is here.

The number of arguments that doesn't throw exception is 3 on ARM64 or 1 on ARMv7.
If the number of arguments is over 13, always throw exception.

I think SnapShot instance itself is one of the arguments. so 4-1=3 and 14-1=13.
I think whether dynamic invocation throws an exception depends on the size of the argument type.

To quick fix this problem for ARM64, SnapShot.MaxGenericTypes is set to 3.

[divega]

The fixes for this issue have been incorporated in Mono. Not sure what builds of Xamarin will contain it but they are in https://github.com/mono/mono/tree/2017-10.

I believe there is nothing actionable on the EF Core side other than validating the fix.

Thanks @vargaz!

[divega]

Removed the area-external label to repurpose the issue just to validate the fix on our side.

[marek-safar]

That's interesting. Are you sure it's the same failure?

@brendanzagaeski could you help us to verify this?

[aritchie]

It appears to be. I use long & DateTimeOffset with nullables in this one particular model. I switched everything to int & datetime, now everything works. I'll setup a sample for you hopefully in the next couple of days.

[ajcvickers]

@divega Can this be moved out of the 2.1 milestone. (We are trying to make 2.1 only include issues that we must fix for 2.1)

[aritchie]

If you guys open up Snapshot.MaxGenericTypes to allow external setting, this issue can be moved entirely to the xamarin iOS team. It is framework bug with them, not with EFCore.

[divega]

As @ajcvickers already explained in #9249 (comment) we won’t be making changes for this. We only kept this open for verification purposes, but we have at least one report indicating that this may not be fixed in the 2.1 timeframe.

@aritchie I recommend you follow up in the Xamarin issue tracker. I will move this to our backlog for now.

[ghost]

I tried today with Visual Studio 15.7.2, Xamarin.Forms 3.0.0.482510, Xamarin.iOS 11.10.1.178 - still no luck...

[0:] System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.ExecutionEngineException: Attempting to JIT compile method '(wrapper runtime-invoke) :runtime_invoke_void__this___Guid_Nullable1<Guid>_object_object_double_int_single_object_object_object_object_Nullable1_object_DateTime_Nullable`1 (object,intptr,intptr,intptr)' while running in aot-only mode. See https://developer.xamarin.com/guides/ios/advanced_topics/limitations/ for more information.

;-(

[Juansero29]

Got this problem as well when using a Devart.SQLite ADO library. To be exact:

Unhandled Exception:

System.ExecutionEngineException: Attempting to JIT compile method '(wrapper native-to-managed) �  :� (intptr,int,intptr)' while running in aot-only mode. See https://developer.xamarin.com/guides/ios/advanced_topics/limitations/ for more information.
 occurred

This error only comes when I test my app on a physical device though. On emulators everything works like a charm.

[vargaz]

Thats a different problem.

[Kelsch]

Still seeing this issue on visual studio version 15.8.1 with xamarin 4.11.0.732 and xamarin.ios 11.14.0.13

[cfdelaune]

Like @Kelsch, I am still seeing the problem with the latest VS and Xamarin versions. I am using EF Core 2.1.2, and would be willing to try the EF Core 2.2 preview, but at this point, it is unclear to me where the problem resides. Is this a VS issue, Xamarin.iOS issue, or an EF Core issue? Until today, I have not been able to test because I kept running a compile error (described here) that prevented me from running on a physical iOS device. It appears that was resolved with VS 15.8.x.

Any information is much appreciated!

[divega]

@Kelsch @cfdelaune I am sorry to hear that this isn't fixed for you in recent releases of Xamarin. If you are seeing the exact same symptoms described here, it is mostly likely still a Xamarin issue. I know that the Xamarin team has made several fixes trying to address this, but perhaps there is one more case that they need to look at.

If you haven't already, please report it by creating a new issue on Xamarin's new issue tracker at https://github.com/xamarin/xamarin-macios/issues. You can mention that you believe this is similar to https://bugzilla.xamarin.com/show_bug.cgi?id=58424 and https://bugzilla.xamarin.com/show_bug.cgi?id=59184, but please also include a repro project so that they can investigate why the fixes already included in the product aren't helping.

Also, feel free to @ mention me in the issue so I can follow up with the Xamarin team.

cc @marek-safar, @vargaz , @mandel-macaque

[vargaz]

Also, could you paste the exact error message you are getting, i.e. the {System.ExecutionEngineException: Attempting to JIT compile method
message.

[cfdelaune]

Thank you for following up. I will post a new issue asap. For now, here is a screenshot of the error I am getting on an iPhone 7 when attempting to add a record to a table with the the following column types (8 total):

• Guid
• string
• DateTime
• Guid?
• DateTime
• DateTime?
• int?
• string

[cfdelaune]

After digging in a bit more, I have been able to isolate that the problem only occurs when there is a column of type Nullable Guid (Guid?). With this configuration I was able to get it working on a physical device with a table consisting of 20+ columns and including Nullable DateTime? and int? types, but throwing a Guid? column into the mix causes this error.

As @divega suggests, I will submit this as a new issue. In the meantime if anyone has ideas for workarounds, I would love to hear them.

[cfdelaune]

I just submitted this as a new issue here.