Avoid special casing work items in sendtohelixhelp.proj

[radical]

    <ItemGroup Condition="'$(TargetOS)' == 'Browser'">
      <!-- Create a work item for run-only WASM apps  -->
      <_RunOnlyWorkItem Include="$(TestArchiveRoot)runonly/**/*.zip" />
      <HelixWorkItem Include="@(_RunOnlyWorkItem -> '%(FileName)')" >
        <PayloadArchive>%(Identity)</PayloadArchive>
        <!-- No RunTests script generated for the sample project so we just use the direct command -->
        <Command>dotnet exec $XHARNESS_CLI_PATH wasm $XHARNESS_COMMAND  --app=. --engine=V8  --engine-arg=--stack-trace-limit=1000 --js-file=runtime.js --output-directory=$XHARNESS_OUT -- --run WasmSample.dll</Command>
      </HelixWorkItem>
    </ItemGroup>

Instead of this, we could try generating RunTests.sh which is done by

runtime/eng/testing/tests.targets

Lines 51 to 52 in 93e767b

	<UsingTask TaskName="GenerateRunScript" AssemblyFile="$(InstallerTasksAssemblyPath)"/>
	<Target Name="GenerateRunScript">

.

Based on @safern's comment #45768 (comment)

I think I would rather do that and try to be consistent, it seems cumbersome to have that "workaround" in sendhelix to include the sample as part of the payload and hardcode the HelixCommand.

Can we open a follow up issue to clean this up and make it more generic for more samples? I see we also added this for iOS and it is doing somewhat similar instead of use the targets we already have.

[Dotnet-GitSync-Bot]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[radical]

/cc @akoeplinger @MaximLipnin

[premun]

@radical please note that RunTest.sh is not used for Helix but only in local development.

For iOS and Android, we use the Helix SDK to take care of running XHarness.

For iOS, we have something like RunTest.sh (but static):
https://github.com/dotnet/arcade/blob/master/src/Microsoft.DotNet.Helix/Sdk/tools/xharness-runner/ios-helix-job-payload.sh

For Android, we call XHarness directly in the Helix command:
https://github.com/dotnet/arcade/blob/16b1ec7dc37cd19927eb7a25e76a67236a9c74a0/src/Microsoft.DotNet.Helix/Sdk/CreateXHarnessAndroidWorkItems.cs#L106
but to my best knowledge, @MattGal might be adding a similar script soon to do some emulator restarting.

For WASM, I would suggest to go similar route and add support on Arcade level as well.

[ghost]

Tagging subscribers to this area: @directhex
See info in area-owners.md if you want to be subscribed.

Issue Details

---

    <ItemGroup Condition="'$(TargetOS)' == 'Browser'">
      <!-- Create a work item for run-only WASM apps  -->
      <_RunOnlyWorkItem Include="$(TestArchiveRoot)runonly/**/*.zip" />
      <HelixWorkItem Include="@(_RunOnlyWorkItem -> '%(FileName)')" >
        <PayloadArchive>%(Identity)</PayloadArchive>
        <!-- No RunTests script generated for the sample project so we just use the direct command -->
        <Command>dotnet exec $XHARNESS_CLI_PATH wasm $XHARNESS_COMMAND  --app=. --engine=V8  --engine-arg=--stack-trace-limit=1000 --js-file=runtime.js --output-directory=$XHARNESS_OUT -- --run WasmSample.dll</Command>
      </HelixWorkItem>
    </ItemGroup>

Instead of this, we could try generating RunTests.sh which is done by

runtime/eng/testing/tests.targets

Lines 51 to 52 in 93e767b

	<UsingTask TaskName="GenerateRunScript" AssemblyFile="$(InstallerTasksAssemblyPath)"/>
	<Target Name="GenerateRunScript">

.

Based on @safern's comment #45768 (comment)

I think I would rather do that and try to be consistent, it seems cumbersome to have that "workaround" in sendhelix to include the sample as part of the payload and hardcode the HelixCommand.

Can we open a follow up issue to clean this up and make it more generic for more samples? I see we also added this for iOS and it is doing somewhat similar instead of use the targets we already have.

Author:	radical
Assignees:	-
Labels:	area-Infrastructure-mono, untriaged
Milestone:	-

[safern]

@premun yeah that's correct. However for iOS we should try to also binplace the sample app into TestArchiveTestsRoot so that we don't have to duplicate logic in sendtohelixhelp.proj:

runtime/src/libraries/sendtohelixhelp.proj

Lines 181 to 203 in 03e4f5e

	<XHarnessAppBundleToTest Include="$([System.IO.Directory]::GetDirectories('$(TestArchiveTestsRoot)', '*.app', System.IO.SearchOption.AllDirectories))">
	<Targets Condition="'$(TargetArchitecture)' == 'arm'">ios-device</Targets>
	<Targets Condition="'$(TargetArchitecture)' == 'arm64'">ios-device</Targets>
	<Targets Condition="'$(TargetArchitecture)' == 'x64'">ios-simulator-64</Targets>
	<Targets Condition="'$(TargetArchitecture)' == 'x86'">ios-simulator-32</Targets>
	<TestTimeout>$(_workItemTimeout)</TestTimeout>
	<LaunchTimeout>$(_workItemTimeout)</LaunchTimeout>
	</XHarnessAppBundleToTest>
	<!-- Create work items for run-only apps -->
	<XHarnessAppBundleToTest Include="$([System.IO.Directory]::GetDirectories('$(TestArchiveRoot)runonly', '*.app', System.IO.SearchOption.AllDirectories))" >
	<Targets Condition="'$(TargetArchitecture)' == 'arm'">ios-device</Targets>
	<Targets Condition="'$(TargetArchitecture)' == 'arm64'">ios-device</Targets>
	<Targets Condition="'$(TargetArchitecture)' == 'x64'">ios-simulator-64</Targets>
	<Targets Condition="'$(TargetArchitecture)' == 'x86'">ios-simulator-32</Targets>
	<LaunchTimeout>$(_workItemTimeout)</LaunchTimeout>
	<!-- The sample app doesn't need test runner -->
	<IncludesTestRunner>false</IncludesTestRunner>
	<!-- The sample's C# Main method returns 42 so it should be considered by xharness as a success -->
	<ExpectedExitCode>42</ExpectedExitCode>
	</XHarnessAppBundleToTest>

[MaximLipnin]

@safern As for iOS, could you point me to other way besides putting to separate directories to differentiate the sample from test apps in order to set a couple of specific properties?

[mdh1418]

@radical I took a look and it seems like there were significant modifications to sendtohelix*(help/mobile/wasm).proj since this issue was first created. I'm not sure if I'm interpreting things correctly. It seems like this issue aims to remove hardcoded/special cased HelixWorkItem by using RunTest.sh generated through GenerateRunScript, but comments suggest this is more of an effort to remove duplicate code?

Is this still relevant, might be worth pushing to 8 if it isn't urgent.

[radical]

@mdh1418 I think we have reduced that quite a bit. But @premun's suggestion in #46156 (comment), we should probably still do that. But it's not urgent at all. Moving to Future milestone.