App Crash upon launch in microsoft.net.sdk.ios version 17.2.8078

[IainS1986]

This being raised as I saw someone mention similar in this ticket #20848 and I believe we are seeing the same behavior.

Builds on Dev ops on the 5th July were working fine, a build then on 12th july onwards (including building the exact same commit point) crash on launch.

At first I assumed it was an apple certificate issue (something expiring) and was even about to go commit by commit doing builds but then saw someone mention in the issue above that they too are now seeing crashes on startup.

The build that worked fine had the following workloads....

Installing workload manifest microsoft.net.sdk.android version 34.0.113...
Installing workload manifest microsoft.net.sdk.ios version 17.2.8053...
Installing workload manifest microsoft.net.sdk.maccatalyst version 17.2.8053...
Installing workload manifest microsoft.net.sdk.macos version 14.2.8053...
Installing workload manifest microsoft.net.sdk.maui version 8.0.40...
Installing workload manifest microsoft.net.sdk.tvos version 17.2.8053...
Installing workload manifest microsoft.net.sdk.aspire version 8.0.2...

The build that now crash on startup...

Installing workload manifest microsoft.net.sdk.android version 34.0.113...
Installing workload manifest microsoft.net.sdk.ios version 17.2.8078...
Installing workload manifest microsoft.net.sdk.maccatalyst version 17.2.8078...
Installing workload manifest microsoft.net.sdk.macos version 14.2.8078...
Installing workload manifest microsoft.net.sdk.maui version 8.0.61...
Installing workload manifest microsoft.net.sdk.tvos version 17.2.8078...
Installing workload manifest microsoft.net.workload.mono.toolchain.current version 8.0.7...
Installing workload manifest microsoft.net.workload.emscripten.current version 8.0.7...
Installing workload manifest microsoft.net.workload.emscripten.net6 version 8.0.7...
Installing workload manifest microsoft.net.workload.emscripten.net7 version 8.0.7...
Installing workload manifest microsoft.net.workload.mono.toolchain.net6 version 8.0.7...
Installing workload manifest microsoft.net.workload.mono.toolchain.net7 version 8.0.7...
Installing workload manifest microsoft.net.sdk.aspire version 8.0.2...

Also just to note with regards to the Slow/Large builds in the ticket above (#20848) we've been experiencing this since we ported to .net around March time. Our ipa went up about 50% in size and our builds regularly take an hour. I noticed the person in that ticket was on a very old version, so the longer build times i'm not convinced were intorduced in 8078.

I'm trying to get crash logs from the device to see why the app is actually dying at startup - but I can't actually see any crash logs on the device - the app seems to be dying without a crash log being created. I've tried attaching the console and getting output - but its not easy to see what might be relevant - theres nothing obvious but theres a lot of noise.

[IainS1986]

Roll back has stopped the startup crash.

  - task: DotNetCoreCLI@2
    displayName: "Install Workloads"
    inputs:
      command: 'custom'
      custom: 'workload'
      arguments: 'install ios maui-ios android maui-android maccatalyst --from-rollback-file https://maui.blob.core.windows.net/metadata/rollbacks/8.0.61.json --source https://api.nuget.org/v3/index.json'

[ivanpovazan]

Do you have <UseInterpreter>true</UseInterpreter> set in your project file?

[IainS1986]

So I saw discussion around that on Discord and the other ticket, I don't have that but do use MTouchInterpreter to AOT everythig except a few Dlls...

<MtouchInterpreter>-all,WW.Pdf,WW.Cad,WW</MtouchInterpreter>

The 'full' settings are here (this was ported from Xamarin and the project has been ongoing for years so this is a bit of a frankenstein list of settings....)

    <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">
        <CreatePackage>false</CreatePackage>
        <MtouchLink>SdkOnly</MtouchLink>
        <MtouchEnableSGenConc>false</MtouchEnableSGenConc>
        <CodesignKey>iPhone Distribution</CodesignKey>
        <RuntimeIdentifier>ios-arm64</RuntimeIdentifier>
        <MtouchExtraArgs>--dlsym:Microsoft.Win32.Registry.dll</MtouchExtraArgs>
        <DebugType>none</DebugType>
        <WarningLevel>4</WarningLevel>
        <CodesignProvision>Automatic</CodesignProvision>
        <CodesignEntitlements>Entitlements.plist</CodesignEntitlements>
        <PlatformTarget>ARM64</PlatformTarget>
        <ForceSimulatorX64ArchitectureInIDE>true</ForceSimulatorX64ArchitectureInIDE>
        <MtouchUseLlvm>true</MtouchUseLlvm>
        <MtouchEnableBitcode>false</MtouchEnableBitcode>
        <!-- AOT everything, except WW dlls, which will be interpreted -->
        <MtouchInterpreter>-all,WW.Pdf,WW.Cad,WW</MtouchInterpreter>
    </PropertyGroup>

[ivanpovazan]

Ok, I just wanted to check if it is not possibly a duplicate of: dotnet/maui#23577

Would it be possible for you to provide a smaller repro project?

On the other hand, regarding getting the crash reports and the Console.app
You can:

• Open Xcode
• Choose Window -> Devices and Simulators from the menu bar
• Under the Devices section in the left column, choose the device.
• From the right panel, you should see an option for "Open Recent Logs"
• It may take some time for Xcode to fetch the data, but once it is done, you can search for your app name to find the crash report
• Once you find it you can download it to your Mac (drag and drop to Desktop for example)
• Then open the crash report in your Console.app

[maroy1986]

Roll back has stopped the startup crash.

  - task: DotNetCoreCLI@2
    displayName: "Install Workloads"
    inputs:
      command: 'custom'
      custom: 'workload'
      arguments: 'install ios maui-ios android maui-android maccatalyst --from-rollback-file https://maui.blob.core.windows.net/metadata/rollbacks/8.0.61.json --source https://api.nuget.org/v3/index.json'

Just wanted to chime in here. The same happened to us for .Net Maui Blazor Hybrid app built with ABP Framework. We built a version on July 4 and everything was fine. We had to do another build on July 10 and then it started having these symptoms of startup crash. Thinking it was the latest changes, I went and tried to build the same code we built previously on the 4th to end up with a crashing app as well. After a few hours of digging and log pipeline analysis, we figured out the iOS workloads changed from 17.2.8055 to 17.2.8078

It took a while but I did end up, after several hours again, to find the undocumented --from-rollback-file switch on the restore command. This also solved our issue, although I'm considering this more of a temporary fix.

Gladly we didn't have to make a hotfix release at this time, otherwise, this would have been catastrophic.

Launching on a USB Connected iPhone from Visual Studio 2022 was still working fine, it was only when building on Azure DevOps that the resulting build was crashing, if that can help in any way.

[DDHSchmidt]

We experienced a similar crash in our project after the merge of a big PR. Only in Release mode and it would also disappear if we used a rollback file to revert the workload back to 17.2.8022.
Like @IainS1986 We had the config setting <MtouchInterpreter>-all,{someExcludedDlls...}</MtouchInterpreter> in our csproj which obviously triggers this crash during release mode.

@ivanpovazan gave the hint to try setting <MtouchExtraArgs>--setenv=MONO_LOG_LEVEL=debug</MtouchExtraArgs> and watch the console log in XCode with that, while the app is starting/crashing.
At first this looked like a dead end: It added thousands of debug lines akin to AOT: FOUND method ... or AOT NOT FOUND method ... in the console log and nothing juicy like an exception or a stacktrace.
Today I looked at it again and noticed the last line like that, that was output in the log, metioned a property ".... SortControl.IsAscending...." and I remembered: One of the changes that were part of that big PR was a new SortControl which had an IsAscending property implemented like this.

public bool IsAscending
    {
        get
        {
            var listType = typeof(ListSortOption<>);
            var optionType = SelectedSortOption?.GetType();
            if ((optionType?.IsGenericType ?? false) && optionType.GetGenericTypeDefinition().IsAssignableFrom(listType))
            {
                var isAscendingProp = optionType.GetProperty(nameof(ListSortOption<object>.IsAscending));
                return (bool)isAscendingProp.GetValue(SelectedSortOption);
            }
            return false;
        }
    }

To our defense: ListSortOption is a generic class (ListSortOption<T> to be precise) and the SortControl doesn't know which concrete implementation it's dealing with, so we used Reflection to just get the bool property we wanted to meddle with.

Since that smelled, I introduced a new (stupid) Interface to ListSortOption:

public interface IAscendable
{
    bool IsAscending { get; set; }
}

Simplifying the previous property in our SortControl:

public bool IsAscending
    {
        get
        {
            return (SelectedSortOption as IAscendable)?.IsAscending ?? false;
        }
    }

This now successfully runs with the latest workload version 17.2.8078 and does not crash anymore.

To summarize:

1. The previous, generic approach still worked fine in workload 17.2.8022 but produces crashes in 17.2.8078 (Release mode only)
2. Setting <MtouchInterpreter>all</MtouchInterpreter> or omitting that in favour of <UseInterpreter>true</UseInterpreter> also got rid of the crash.
3. Using <MtouchExtraArgs>--setenv=MONO_LOG_LEVEL=debug</MtouchExtraArgs> and paying close attention to the last output of the generated AOT debug message gave me a valuable hint as to who the culprit was.

UPDATE
Just wanted to add that I'm aware of the constraint
There's limited generics support. Not every possible generic instantiation can be determined at compile time. Many of the iOS-specific issues encountered in .NET MAUI release builds are due to this limitation.
I looked back at the build output of the crashing build and noticed no particular warning about the generic construct we used. Can that be tracked/recognized and output as a warning when MTouchInterpreter is set to "-all" ?
Or is there already a warning in the build output and I just didn't spot it?

[ivanpovazan]

@DDHSchmidt thats a great finding! I am glad you managed to find what was causing the crash.

Would you mind sharing the exact message in the logs regarding:

metioned a property ".... SortControl.IsAscending...."

I would really like to try to create a smaller repro and identify where the issue is coming from, so we can look into how we can improve handling of such constructs and prevent this from happening again.

[DDHSchmidt]

Sorry for the late reply.
As I mentioned: There were thousands of AOT debug messages in the console. The last 3 ones were:

debug: AOT NOT FOUND: (wrapper delegate-invoke) System.Reflection.RuntimePropertyInfo/Getter`2<CompanyApp.Models.UI.ListSortOption`1<CompanyApp.Models.Database.BusinessObject>, bool>:invoke_callvirt_R_T (CompanyApp.Models.UI.ListSortOption`1<CompanyApp.Models.Database.BusinessObject>).
debug: AOT NOT FOUND: (wrapper other) object:interp_in (intptr).
debug: AOT: FOUND method CompanyApp.Models.UI.ListSortOption`1:get_IsAscending () [0x108896c7c - 0x108896cac 0x11020a958]

Incidentally: The last line was also the last line ever from my app process. All the following lines were about some internal iOS processes that output info about my app process being terminated.

In hindsight: Our SortControl was not explicitly mentioned in these log lines. I remembered that wrong.
But the IsAscending-property was bound via XAML to our SortControl and the lines made me suspicious enough to trace it back to the generic code that I mentioned above.

[rolfbjarne]

To all of those experiencing this crash, and just to narrow down what's happing, does interpreting all assemblies make the crash go away?

In other words, does this resolve the crash?

<MtouchInterpreter>all</MtouchInterpreter>

[microsoft-github-policy-service]

Hi @IainS1986. We have added the "need-info" label to this issue, which indicates that we have an open question for you before we can take further action. This issue will be closed automatically in 7 days if we do not hear back from you by then - please feel free to re-open it if you come back to this issue after that time.

[rolfbjarne]

To all of those experiencing this crash, and just to narrow down what's happing, does interpreting all assemblies make the crash go away?

In other words, does this resolve the crash?

<MtouchInterpreter>all</MtouchInterpreter>

I just noticed this from a previous comment:

Setting all or omitting that in favour of true also got rid of the crash.

So I believe this is a different variation of #20848, where disabling code deduplication with AOT ended up causing crashes.

AOT code deduplication will be re-enabled again in the next service release, so it's possible this crash will be fixed then.

However, AOT code deduplication is an optimization, which means disabling it shouldn't cause a crash. Thus I believe there's still an underlying issue here, even if the crash itself is gone after the next service release.

Unfortunately these kind of AOT compiler issues can be really hard to reproduce without a complete test project (I talked to @ivanpovazan and he wasn't able to create a test case with the information given so far), so that's what I'm asking for: a complete project/solution we can use to reproduce the crash (this can be your full project (sent to us privately if that's preferrable) it doesn't have to be a minimized test project).

[microsoft-github-policy-service]

Hi @IainS1986. Due to inactivity, we will be closing this issue. Please feel free to re-open this issue if the issue persists. For enhanced visibility, if over 7 days have passed, please open a new issue and link this issue there. Thank you.