[android] Failing E2E Android - Devices Helix Tests Build_Debug tests

[ivanpovazan]

Build

https://dev.azure.com/dnceng-public/public/_build/results?buildId=750141

Build leg reported

E2E Android - Devices Helix Tests Build_Debug failed

Pull Request

dotnet/xharness#1233

Known issue core information

Fill out the known issue JSON section by following the step by step documentation on how to create a known issue

{
  "ErrorMessage": "crit: Install failure: Test command cannot continue",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

@dotnet/dnceng

Release Note Category

• Feature changes/additions
• Bug fixes
• Internal Infrastructure Improvements

Release Note Description

Additional information about the issue reported

No response

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=750141
Error message validated: [crit: Install failure: Test command cannot continue]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 7/29/2024 6:26:51 PM UTC

Report

Build	Definition	Test	Pull Request
863362	dotnet/xharness	System.Buffers.Tests-x86.WorkItemExecution
855598	dotnet/runtime	Microsoft.NETCore.Platforms.Tests.WorkItemExecution
850954	dotnet/xharness	System.Buffers.Tests-x86.WorkItemExecution

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	3

[ivanpovazan]

Problem

This issue seems to be occurring more and more, and xharness CI is mostly red.

The problem comes from the fact that in some jobs we are testing debug apks on physical Android devices, which gives:

adb: failed to install D:\h\w\A8C809B3\w\A92C0917\e\System.Buffers.Tests-arm64-v8a.apk: Failure [INSTALL_FAILED_VERIFICATION_FAILURE: Package Verification Result]
             
[02:24:27] crit: Install failure: Test command cannot continue

(link to the latest failure on DNCENGWIN-007 : https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-xharness-refs-heads-release-90-68d2120ee0c9454f80/System.Buffers.Tests-arm64-v8a.Attempt.3/1/console.03621459.log?helixlogtype=result

Installing such apks can be rejected by the device, if Verify apps over USB setting is enabled on it.

Assumption

My assumption is that the tethered devices were initially configured to have this verification disabled in device settings - allowing debug apks to be installed and tested, and that a factory reset happened recently, which caused switching this setting on again and caused the failures we are now seeing.

Ask

DNCENGWIN-070 had a successful run https://dev.azure.com/dnceng-public/public/_build/results?buildId=811051&view=logs&j=00f02412-5f5e-5b12-0780-d2e57de065ba&t=b1200827-0fdc-54c0-1776-c7b8da453b26
while DNCENGWIN-007 seems to fail constantly. Checking the Verify apps over USB setting on tethered devices on these machines would help us figure out the next steps.

@ilyas1974, could someone check this out?

[ilyas1974]

Hi Ivan, I have taken DNCENG-007 out of rotation so no additional jobs will use that system while we investigate the failure you mentioned above. I have created https://portal.microsofticm.com/imp/v5/incidents/details/546521338 to have the system in question looked at.

[ivanpovazan]

Thank you!
Should I add some context on the created incident ticket?

[ivanpovazan]

@ilyas1974 as we fixed #4253 this can be closed now.