[nativeaot][tests] Add smoke unit runtime tests for Native AOT on iOS platforms #87260

kotlarmilos · 2023-06-08T08:36:26Z

Description

This PR aims to add smoke unit runtime tests for Native AOT on ioslike platforms. The src/tests pipeline is modified to build tests for the native AOT ioslike platforms. This is achieved by using the ILC publish targets and adjusting the ILC path to use the target runtime packs for the compilation. The build job template is updated to support hostedOs and hostedArch properties used as a build target platform due to ILC limitations to be built for devices. Additionally, a new target for creating an app bundle is added.

This PR also introduces runtime-ioslike-mono and runtime-ioslike-coreclr azp runs to balance the load on devices. The tests are enabled on ios/iossimulator and tvos/tvossimulator platforms. The maccatalyst platform is temporarily disabled, awaiting a new preview version of SDK to be used in the runtime.

Tests that fail on a device are disabled due to dynamic code generation, globalization issues, or invalid handling of the SIGSEGV error.

ghost · 2023-06-08T08:36:40Z

Tagging subscribers to this area: @hoyosjs
See info in area-owners.md if you want to be subscribed.

Issue Details

Work in progress.

This PR aims to include a subset of smoke runtime tests for Native AOT on iOS. Currently, it includes some temporary workarounds that will be resolved once the CI reports green.

Description

TBD

Author:	kotlarmilos
Assignees:	kotlarmilos
Labels:	`NO-MERGE`, `NO-REVIEW`, `area-Infrastructure-coreclr`, `os-ios`
Milestone:	8.0.0

azure-pipelines · 2023-07-13T08:08:24Z

Azure Pipelines successfully started running 2 pipeline(s).

eng/pipelines/extra-platforms/runtime-extra-platforms-ioslike.yml

eng/pipelines/runtime-ioslike-coreclr.yml

ivanpovazan · 2023-07-13T09:21:31Z

src/tasks/AppleAppBuilder/Templates/main-simple.m

@@ -48,7 +48,6 @@ - (void)viewDidLoad {
    [button addTarget:self action:@selector(buttonClicked:) forControlEvents:UIControlEventTouchUpInside];
    [button setFrame:CGRectMake(50, 300, 200, 50)];
    [button setTitle:@"Click me (wire me up)" forState:UIControlStateNormal];
-    [button setExclusiveTouch:YES];


Why are we removing this?

setExclusiveTouch property is not supported on tvOS. It doesn't change tests functionality and we wanted to avoid #ifdef macros if possible.

ivanpovazan · 2023-07-13T09:32:21Z

src/tests/build.proj

+      <_LinkerFlagsToDrop Include="@(NativeFramework->'-framework %(Identity)')" />
+      <LinkerArg Remove="@(_LinkerFlagsToDrop)" />
+      <ExtraAppLinkerArgs Include="@(LinkerArg)" />
+      <GlobalizationkNativeLibs Include="$(MicrosoftNetCoreAppRuntimePackDir)runtimes/$(TargetOS)-$(TargetArchitecture)/native/icudt.dat" />


Why is this needed? I don't think we are doing this explicitly for mono. The file should be picked up by the build.
Additional nit: GlobalizationkNativeLibs is misspelled.

I think Mono and Native AOT use different publish paths. During the Mono publish the icudt.dat file ends up in the publish directory. After the Native AOT publish, it has to be added explicitly to the publish directory to be picked up by the AppleAppBuilder.

InnerManagedTestBuild.3.binlog.zip

I see, if I am not mistaken that should be handled by the common ComputeResolvedFilesToPublishList target, so if we need to explicitly include that file, it would probably be better to do something like:

<ResolvedFileToPublish Include="$(MicrosoftNetCoreAppRuntimePackDir)runtimes/$(TargetOS)-$(TargetArchitecture)/native/icudt.dat">

This way copying to the publish folder would be done by the build (so the manual Copy bellow will not be needed).

The right place to include the file explicitly is probably somewhere in AppleBuild.targets as we would want to have the same behaviour for the HelloiOS sample as well.

The native libs get populated in ResolveRuntimePackAssets and copied to the publish directory in _CopyFilesMarkedCopyLocal.

During the build, the ResolveRuntimePackAssets is skipped due to the condition below.

<Target Name="ResolveRuntimePackAssets" DependsOnTargets="ResolveFrameworkReferences" Condition="'@(RuntimePack)' != ''">

I assume that the RuntimePack is null since the CI builds don't use the runtime packs. If icudt.dat is explicitly referenced in ResolvedFileToPublish item, I think it happens after the _CopyFilesMarkedCopyLocal target and the file doesn't end up in the publish directory.

Thanks for checking that out and my apologies as I was not aware separate builds are used in this case, so setting up what I suggested would not work.
For mono we also collect all the native dependencies:

runtime/src/tests/build.proj

Line 319 in 4d5d2dc

<RuntimePackNativeLibs Include="$(MicrosoftNetCoreAppRuntimePackDir)/**/*.dll;$(MicrosoftNetCoreAppRuntimePackDir)/native/**/*.a;$(MicrosoftNetCoreAppRuntimePackDir)/native/**/*.dylib" />

and then recreate and copy them over into what supposed to be the new publish folder.

kotlarmilos · 2023-07-13T10:49:09Z

/azp run runtime-ioslikesimulator

azure-pipelines · 2023-07-13T10:49:18Z

Azure Pipelines successfully started running 1 pipeline(s).

ivanpovazan

LGTM! Thanks! Great job 🚀

akoeplinger · 2023-07-14T08:30:52Z

eng/liveBuilds.targets

@@ -128,7 +128,7 @@
      <MonoIncludeFiles Include="$(MonoArtifactsPath)\include\**\*.*" />
    </ItemGroup>

-    <Error Condition="'@(RuntimeFiles)' == ''" Text="The '$(RuntimeFlavor)' subset must be built before building this project." />
+    <Error Condition="'@(RuntimeFiles)' == '' and !('$(RuntimeFlavor)' == 'CoreCLR' and '$(TargetsAppleMobile)' == 'true')" Text="The '$(RuntimeFlavor)' subset must be built before building this project." />


this sounds wrong to me, I think I hit the same issue when adding the runtime packs which is why I added the BuildNativeAOTRuntimePack that makes the RuntimeFiles pick up NativeAOT assets instead of CoreCLR on L80.

You could set that property too, though I wonder if longer-term we should introduce RuntimeFlavor=NativeAOT

Thanks, updated!

…tead of CoreCLR

kotlarmilos · 2023-07-14T08:54:12Z

/azp run runtime-ioslikesimulator

azure-pipelines · 2023-07-14T08:54:18Z

No commit pushedDate could be found for PR 87260 in repo dotnet/runtime

kotlarmilos · 2023-07-14T08:56:28Z

/azp run runtime-ioslikesimulator

azure-pipelines · 2023-07-14T08:56:34Z

No commit pushedDate could be found for PR 87260 in repo dotnet/runtime

kotlarmilos · 2023-07-14T08:58:19Z

/azp run runtime-ioslikesimulator

azure-pipelines · 2023-07-14T08:58:28Z

Azure Pipelines successfully started running 1 pipeline(s).

steveisok

Looks good!

kotlarmilos · 2023-07-14T14:24:34Z

The Native AOT tests are passing and the failures shouldn't be related.

Thank you all for valuable feedback and assistance!

Initial commit

7b4e749

kotlarmilos added NO-MERGE The PR is not ready for merge yet (see discussion for detailed reasons) NO-REVIEW Experimental/testing PR, do NOT review it area-Infrastructure-coreclr os-ios Apple iOS labels Jun 8, 2023

kotlarmilos added this to the 8.0.0 milestone Jun 8, 2023

kotlarmilos self-assigned this Jun 8, 2023

kotlarmilos added 10 commits June 8, 2023 10:41

Merge branch 'main' into feature/nativeaot-runtime-tests-ios

0b18e34

Enable restore projects

a47e8a0

Temorary disable other tests

f8bc436

Temorary disable other tests

5f8f2d1

Fix build command

40c9979

Fix build command

2673823

Remove NativeAOT runtime flavor

35923f9

Fix build command

bdc038f

Test build configuration

ef9d492

Test build configuration

d7356ac

kotlarmilos changed the title ~~[wip][nativeaot][tests] Add Native AOT smoke runtime tests for iOS~~ [wip][nativeaot][tests] Add smoke runtime tests for Native AOT on iOS Jun 8, 2023

kotlarmilos added 11 commits June 8, 2023 12:12

Test build configuration

eee4341

Test build configuration

f4d3d8c

Test build configuration

12db78e

Test build configuration

20d6ce5

Test build configuration

a722343

Test build configuration

881e58d

Test build configuration

fb478a1

Merge branch 'main' into feature/nativeaot-runtime-tests-ios

200fba9

Test build configuration

33426b5

Test build configuration

5a7fd1c

Test build configuration

4f99bc9

ivanpovazan reviewed Jul 13, 2023

View reviewed changes

build-analysis bot mentioned this pull request Jul 13, 2023

File check failures in x64 jit tests #88783

Closed

Simplify build process and fix a typo

3559afa

build-analysis bot mentioned this pull request Jul 13, 2023

[wasm] S.R.IS.JavaScript.Tests.JSImportExportTest.JsImportTaskTypes #77334

Closed

ivanpovazan approved these changes Jul 13, 2023

View reviewed changes

kotlarmilos added 5 commits July 13, 2023 17:46

Test Native AOT perf job

be76e86

Update perf build command for Native AOT

df3f62b

Test Native AOT perf job

7b232fb

Update IlcToolsPath to use cross-build ilc path

ac2b9a3

Test Native AOT perf job

f5e8fba

build-analysis bot mentioned this pull request Jul 13, 2023

ComInterfaceGenerator.Unit.Tests.CompileFails.VerifyByValueMarshallingAttributeUsage tests are failing #88867

Closed

akoeplinger approved these changes Jul 14, 2023

View reviewed changes

Use /p:BuildNativeAOTRuntimePack=true to pick up NativeAOT assets ins…

3004990

…tead of CoreCLR

Merge branch 'dotnet:main' into feature/nativeaot-runtime-tests-ios

06b9868

build-analysis bot mentioned this pull request Jul 14, 2023

Tests.System.TimeProviderTests.TestProviderTimer test failure #87477

Closed

kotlarmilos mentioned this pull request Jul 14, 2023

[ios][tests] Run functional tests with Mono and Native AOT on Helix #87773

Merged

steveisok approved these changes Jul 14, 2023

View reviewed changes

kotlarmilos merged commit 9c8c973 into dotnet:main Jul 14, 2023

radical mentioned this pull request Jul 17, 2023

CI: runtime-ioslike-coreclr, and runtime-ioslike-mono running on unrelated PRs #88995

Closed

ghost locked as resolved and limited conversation to collaborators Aug 14, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[nativeaot][tests] Add smoke unit runtime tests for Native AOT on iOS platforms #87260

[nativeaot][tests] Add smoke unit runtime tests for Native AOT on iOS platforms #87260

kotlarmilos commented Jun 8, 2023 •

edited

Loading

ghost commented Jun 8, 2023

Description

azure-pipelines bot commented Jul 13, 2023

ivanpovazan Jul 13, 2023

kotlarmilos Jul 13, 2023

ivanpovazan Jul 13, 2023

kotlarmilos Jul 13, 2023

ivanpovazan Jul 13, 2023

kotlarmilos Jul 13, 2023

ivanpovazan Jul 13, 2023

kotlarmilos commented Jul 13, 2023

azure-pipelines bot commented Jul 13, 2023

ivanpovazan left a comment

akoeplinger Jul 14, 2023

kotlarmilos Jul 14, 2023

kotlarmilos commented Jul 14, 2023

azure-pipelines bot commented Jul 14, 2023

kotlarmilos commented Jul 14, 2023

azure-pipelines bot commented Jul 14, 2023

kotlarmilos commented Jul 14, 2023

azure-pipelines bot commented Jul 14, 2023

steveisok left a comment

kotlarmilos commented Jul 14, 2023

[nativeaot][tests] Add smoke unit runtime tests for Native AOT on iOS platforms #87260

[nativeaot][tests] Add smoke unit runtime tests for Native AOT on iOS platforms #87260

Conversation

kotlarmilos commented Jun 8, 2023 • edited Loading

Description

ghost commented Jun 8, 2023

Description

azure-pipelines bot commented Jul 13, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kotlarmilos commented Jul 13, 2023

azure-pipelines bot commented Jul 13, 2023

ivanpovazan left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kotlarmilos commented Jul 14, 2023

azure-pipelines bot commented Jul 14, 2023

kotlarmilos commented Jul 14, 2023

azure-pipelines bot commented Jul 14, 2023

kotlarmilos commented Jul 14, 2023

azure-pipelines bot commented Jul 14, 2023

steveisok left a comment

Choose a reason for hiding this comment

kotlarmilos commented Jul 14, 2023

kotlarmilos commented Jun 8, 2023 •

edited

Loading