Commits on Apr 27, 2018

1. [tests] Report errors last in RunApkTests (Take 2!) …

   Context: https://jenkins.mono-project.com/job/xamarin-android-pr-builder/2963/testReport/
   
   A build for PR dotnet#1489 ran 1110 tests; it *should* have run over 35,000.
   Where are the missing tests?
   
   What's missing are the `.apk` on-device tests; BCL tests make up
   *most* of our expected test count.
   
   Analysis of the build log shows that the on-device tests *are*
   running, with test results being downloaded:
   
   	…/android-toolchain/sdk/platform-tools/adb -s emulator-5570  pull "/data/data/Xamarin.Android.Bcl_Tests/files/.__override__/TestResults.xUnit.xml" "…/xamarin-android/tests/../bin/TestDebug/TestResult-Xamarin.Android.Bcl_Tests.xunit.xml"
   	...
   	/data/data/Xamarin.Android.Bcl_Tests/files/.__override__/TestResults.xUnit.xml: 1 file pulled. 53.5 MB/s (3273412 bytes in 0.058s)
   
   but those results don't appear on the `testReport` URL.
   
   The problem is that the `RenameApkTestCases` target is never being
   executed, and the `RenameApkTestCases` is responsible for copying the
   `TestResult*` files where Jenkins looks for them:
   
   	Done building target "ReleaseAndroidTarget" in project "RunApkTests.targets" -- FAILED.:
   
   The `RenameApkTestCases` target executes *after*
   `ReleaseAndroidTarget`, but because `ReleaseAndroidTarget` failed, the
   `RenameApkTestCases` target isn't executed.
   
   Meanwhile, `ReleaseAndroidTarget` reports an error to ensure that
   errors aren't overlooked; see commit 3b893cd.
   
   Allow the `RenameApkTestCases` target to execute by adding a new
   `ReportComponentFailures` target. The `ReleaseAndroidTarget` will no
   longer report errors, allowing the `RenameApkTestCases` target to
   execute, while if any tests *do* fail, the `ReportComponentFailures`
   target will generate an appropriate error.
   
   Furthermore, update all the `<MSBuild/>` invocations within
   `RunTests.targets` and add `ContinueOnError="ErrorAndContinue"` to all
   of them, to help ensure that we run all the tests we have, even if
   previous tests fail.

   

   jonpryor committed Apr 27, 2018
   Configuration menu

   • View commit details
   • Copy full SHA for 44c88a8
   • Browse repository at this point

   Copy the full SHA

   44c88a8 View commit details

   Browse the repository at this point in the history