Add Windows.Markup tests #138

hughbe · 2018-12-10T03:39:15Z

Bunch of tests for attributes, ValueSerializer, event args, converters etc in the System.Windows.Markup namespace

Assistance required: wrote these on my mac, they pass :) but not sure how to integrate into CI

Planning on writing more for the remaining stuff soon!

karelz · 2018-12-10T04:24:34Z

@hughbe hold on for a few days please. Looks like WPF team has plans for tests and we want to make sure your effort aligns with them.
There are more existing tests to be open-sourced for conversion. There may be limitations on how many new tests to take, etc. (due to long runtimes, etc.)
Please give us few days to come up either with a plan or plan for a plan. Thanks!

@dotnet/wpf-developers

hughbe · 2018-12-10T09:06:57Z

Okay no problem, from the readme:

We have published very few tests and have very limited coverage for PRs at this time as a result. We will add more tests in 2019, however, it will be a progressive process. We welcome test contributions to increase coverage and help us validate PRs more easily.

These tests are aimed at getting high coverage for simple attributes and classes used around System.Xaml. They’re unit, not integration tests and take a really short time running so runtime shouldn’t be an issue

rladuca · 2018-12-16T19:49:58Z

@hughbe We're working on getting public test guidance out and having a plan for accepting great contributions like this. Apologies for not being completely prepared in this regard, we'll get there as soon as we can.

Just to be clear, we're not concerned with the runtimes of these specific tests. Generally, the full CI test suite also contains a good amount of GUI end to end testing. This is necessary to validate core functionality of the product. Unfortunately, the expected runtime of such a set of tests is around 15-20 minutes. So we're hesitant to add to this test corpus until we have a better idea of what sort of impact everything has all up.

That being said, our overall goal for testing is to port everything WPF has. Our feature tests are much larger (~24,000 tests total) and obviously can't be run as part of every CI build. So these will be added as a disjointed test set (we haven't decided exactly how just yet) that developers can use as we do internally to validate more technical changes and contribute new test coverage.

I'll be working on this plan and broader test porting in the coming weeks and I'll update dotnet/wpf-test#105 accordingly.

illialarka · 2019-01-15T11:00:18Z

Hi, everyone!

Well, from this conversation I realized there is no published plan for testing, is not there? I would like to contribute to WPF test. Should I wait publishing of plans?

rladuca · 2019-01-15T19:34:31Z

@illialarka Yes, please wait on the plan to be published. We've been going over a set of plans for both getting the current tests we have from desktop framework ported and also allowing contributions in the short term from the community. In the next 2 or 3 weeks, we hope to have the guidance sorted and published.

hughbe · 2019-04-07T06:26:33Z

I'd like to move forward with this PR if there aren't any objections :)

Basically what I've done here is written unit tests that cover the classes found in the System.Windows.Markup namespace. For this reason I don't think it belongs in DRT tests, but rather in another test project. Following .NET Core open source conventions, I've named it System.Xaml.Tests.csproj.

I'm not sure whether this will automatically be run in the CI. I've tested it all locally, but thats something I'd appreciate a double check on that we run these unit tests in CI

karelz · 2019-04-07T09:01:33Z

Thanks for not giving up on this @hughbe!
@vatsan-madhavan is it something you would consider?

hughbe · 2019-06-26T11:12:42Z

Any progress with adding a unit test project for things like this?

stevenbrix · 2019-06-26T20:25:52Z

src/Microsoft.DotNet.Wpf/test/System.Xaml/System.Xaml.Tests.csproj

@@ -0,0 +1,15 @@
+<Project Sdk="Microsoft.NET.Sdk">


can you name this file "System.Xaml.UnitTests.csproj". i think that way it get's automatically picked up and ran by arcade.

See https://github.com/dotnet/arcade/blob/master/Documentation/ArcadeSdk.md#project-name-conventions

stevenbrix · 2019-06-26T20:35:03Z

src/Microsoft.DotNet.Wpf/test/System.Xaml/System.Xaml.Tests.csproj

+    <ProjectReference Include="..\..\src\System.Xaml\System.Xaml.csproj" />
+  </ItemGroup>
+  <ItemGroup>
+    <PackageReference Include="coverlet.msbuild" Version="2.4.0">


what is this package reference for? do other dotnet repos use this?

Yeah - its used for code coverage /cc @ViktorHofer

stevenbrix · 2019-06-26T20:37:52Z

src/Microsoft.DotNet.Wpf/test/System.Xaml/Common/CustomXamlNameResolver.cs

+using System.Collections.Generic;
+using Xunit;
+
+namespace System.Xaml.Tests.Common


what makes these "Common"?

oh i see, they are test helper classes, i thought these were tests 😂

What are your thoughts on using something like the Moq framework (like they use in winforms)

Would remove the need for this type of CustomXYZ class!

let me go take a look at how it's used in WinForms. just read the github page and it looks awesome!

I think I'm starting to understand how it's used, and that does look pretty nice! I think it'll be pretty clear if you're able to make a small change and push an update? Don't change everything to it just in case, but it looks fairly righteous

You'll need to push another change anyways to kick off another PR and CI run so we can see if the tests are running anyways :)

stevenbrix · 2019-06-26T20:38:10Z

Any progress with adding a unit test project for things like this?

@hughbe I'm not sure, honestly this looks awesome and I love it. This is the direction we need our tests heading in. I'm going to queue up another build and will let you know if i see the tests running.

ryalanms · 2021-01-18T20:57:39Z

/azp run

azure-pipelines · 2021-01-18T20:57:49Z

Azure Pipelines successfully started running 1 pipeline(s).

hughbe · 2023-05-02T15:33:38Z

Have rebased this PR and would welcome feedback & perhaps @ViktorHofer's help on the first commit to make sure I've got the new test project set up correctly

ViktorHofer · 2023-05-02T16:50:43Z

First commit looks good to me 👌

hughbe · 2023-05-02T16:52:09Z

src/Microsoft.DotNet.Wpf/test/System.Xaml/Common/CustomTypeDescriptorContext.cs

+
+namespace System.Xaml.Tests.Common
+{
+    public class CustomTypeDescriptorContext : ITypeDescriptorContext


Could look to use Moq for these instead of direct subclasses

hughbe · 2023-05-02T16:53:57Z

Thanks @ViktorHofer! So nothing else is required for these tests to run automatically in CI etc?

hughbe · 2023-07-18T10:52:08Z

Quick point: It says: dotnet-wpf CI Successful in 20m — Build #20230717.2 had test failures , but the report is green

singhashish-wpf · 2023-07-18T11:49:52Z

Quick point: It says: dotnet-wpf CI Successful in 20m — Build #20230717.2 had test failures , but the report is green

The failing tests are DRTs, As of now we aren't failing the builds due to failing DRTs. This will happen soon once we have all the DRTs in green state to have a baseline. We are in process of fixing the DRTs.
Regards
Ashish

dipeshmsft · 2023-07-18T18:41:49Z

@hughbe, as of now your tests are not running on the CI. Since System.Xaml.Tests.csproj is not added in the Microsoft.Dotnet.Wpf.sln file, they are not being built and run.

hughbe · 2023-07-25T10:43:57Z

OK we have it building, which is good!

Two errors, are they related?

C:\Program Files\Microsoft Visual Studio\2022\Preview\MSBuild\Current\Bin\Microsoft.Common.CurrentVersion.targets(2353,5): error MSB3270: (NETCORE_ENGINEERING_TELEMETRY=Build) There was a mismatch between the processor architecture of the project being built "MSIL" and the processor architecture of the reference "D:\a_work\1\s\artifacts\bin\System.Xaml\x64\Debug\net8.0\System.Xaml.dll", "AMD64". This mismatch may cause runtime failures. Please consider changing the targeted processor architecture of your project through the Configuration Manager so as to align the processor architectures between your project and references, or take a dependency on references with a processor architecture that matches the targeted processor architecture of your project.
C:\Program Files\Microsoft Visual Studio\2022\Preview\MSBuild\Current\Bin\Microsoft.Common.CurrentVersion.targets(2353,5): error MSB3270: (NETCORE_ENGINEERING_TELEMETRY=Build) There was a mismatch between the processor architecture of the project being built "MSIL" and the processor architecture of the reference "D:\a_work\1\s\artifacts\bin\System.Xaml\x64\Debug\net8.0\System.Xaml.dll", "AMD64". This mismatch may cause runtime failures. Please consider changing the targeted processor architecture of your project through the Configuration Manager so as to align the processor architectures between your project and references, or take a dependency on references with a processor architecture that matches the targeted processor architecture of your project.

The CI is acting up a bit showing which tests actually failed though

dipeshmsft · 2023-07-25T10:47:45Z

Apart from the ones on the CI, take a look at the artifacts :
There in the log generated for the tests, we get the following error :

xUnit.net Console Runner v2.4.0 (64-bit .NET 8.0.0-preview.5.23280.8)
System.TypeLoadException: Could not load type 'System.Runtime.CompilerServices.NullableContextAttribute' from assembly 'System.Runtime, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.

hughbe · 2023-07-25T10:58:52Z

Same error as dotnet/runtime#88518? /cc @ericstj

ThomasGoulet73 · 2023-07-25T12:54:11Z

Microsoft.Dotnet.Wpf.sln

@@ -1910,6 +1920,22 @@ Global
 		{AF9084C3-BF37-4A56-A851-89F3BAE731B3}.Release|x86.ActiveCfg = Release|Win32
 		{AF9084C3-BF37-4A56-A851-89F3BAE731B3}.Release|x86.Build.0 = Release|Win32
 		{AF9084C3-BF37-4A56-A851-89F3BAE731B3}.Release|x86.Deploy.0 = Release|Win32
+		{B2F2A89C-55C9-41B1-A645-0948609BD8BE}.Debug|Any CPU.ActiveCfg = Debug|Any CPU


@hughbe I think mapping these to the correct target platform (.sln x86 = .csproj x86, etc) should fix the error "There was a mismatch between the processor architecture of the project being built "MSIL" and the processor architecture of the reference".

dipeshmsft · 2023-08-02T11:49:57Z

/azp run

azure-pipelines · 2023-08-02T11:50:02Z

No commit pushedDate could be found for PR 138 in repo dotnet/wpf

hughbe · 2023-08-21T10:52:55Z

Looks like things are finally green!!

dipeshmsft · 2023-09-05T17:14:25Z

/azp run

azure-pipelines · 2023-09-05T17:14:37Z

Azure Pipelines successfully started running 1 pipeline(s).

pchaurasia14 · 2023-09-06T06:06:03Z

Thank you @hughbe! I apologize for the extended delay in merging this pull request.

hughbe · 2023-09-06T07:13:37Z

Woot woot! Glad this is in

rladuca mentioned this pull request Apr 5, 2023

Create a public test plan dotnet/wpf-test#105

Open

hughbe force-pushed the attribute-tests branch 2 times, most recently from ea40a0b to 6428bd1 Compare December 23, 2018 10:07

hughbe mentioned this pull request Dec 23, 2018

Cleanup System.Windows.Markup code #207

Merged

hughbe force-pushed the attribute-tests branch from 6428bd1 to 76429f2 Compare April 7, 2019 06:19

grubioe added the PR metadata: Label to tag PRs, to facilitate with triage label May 28, 2019

stevenbrix reviewed Jun 26, 2019

View reviewed changes

Base automatically changed from master to main March 17, 2021 17:38

ryalanms requested a review from a team as a code owner March 17, 2021 17:38

pchaurasia14 added the Community Contribution A label for all community Contributions label Jul 20, 2022

ghost assigned hughbe Jul 20, 2022

hughbe force-pushed the attribute-tests branch 2 times, most recently from b6caaf9 to a695515 Compare May 2, 2023 15:32

hughbe commented May 2, 2023

View reviewed changes

hughbe force-pushed the attribute-tests branch from 9fbc655 to 2843f9c Compare July 17, 2023 08:20

hughbe force-pushed the attribute-tests branch from 10eb066 to 13be217 Compare July 24, 2023 18:49

ThomasGoulet73 reviewed Jul 25, 2023

View reviewed changes

hughbe force-pushed the attribute-tests branch from f0ecaa6 to 94c2b9f Compare August 10, 2023 20:21

hughbe force-pushed the attribute-tests branch from 94c2b9f to f8dcdd8 Compare August 21, 2023 08:34

Add System.Xaml.Tests project

bc59fdd

hughbe force-pushed the attribute-tests branch from f8dcdd8 to bc59fdd Compare August 21, 2023 09:58

Fix sln file

9ca1671

rchauhan18 added the Queue for test pass label Aug 28, 2023

dipeshmsft approved these changes Sep 5, 2023

View reviewed changes

dipeshmsft merged commit bde0f12 into dotnet:main Sep 6, 2023
8 checks passed

hughbe deleted the attribute-tests branch September 6, 2023 05:19

rchauhan18 added Included in test pass and removed Queue for test pass labels Sep 7, 2023

hughbe mentioned this pull request Sep 12, 2023

Add more System.Xaml tests #8203

Open

ghost locked as resolved and limited conversation to collaborators Oct 7, 2023

dipeshmsft removed the Included in test pass label Aug 27, 2024

Add Windows.Markup tests #138

Add Windows.Markup tests #138

Conversation

hughbe commented Dec 10, 2018 • edited Loading

karelz commented Dec 10, 2018

hughbe commented Dec 10, 2018

rladuca commented Dec 16, 2018

illialarka commented Jan 15, 2019 • edited Loading

rladuca commented Jan 15, 2019

hughbe commented Apr 7, 2019 • edited Loading

karelz commented Apr 7, 2019

hughbe commented Jun 26, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stevenbrix commented Jun 26, 2019

ryalanms commented Jan 18, 2021

azure-pipelines bot commented Jan 18, 2021

hughbe commented May 2, 2023

ViktorHofer commented May 2, 2023

Choose a reason for hiding this comment

hughbe commented May 2, 2023

hughbe commented Jul 18, 2023

singhashish-wpf commented Jul 18, 2023

dipeshmsft commented Jul 18, 2023

hughbe commented Jul 25, 2023

dipeshmsft commented Jul 25, 2023

hughbe commented Jul 25, 2023

Choose a reason for hiding this comment

dipeshmsft commented Aug 2, 2023

azure-pipelines bot commented Aug 2, 2023

hughbe commented Aug 21, 2023

dipeshmsft commented Sep 5, 2023

azure-pipelines bot commented Sep 5, 2023

pchaurasia14 commented Sep 6, 2023

hughbe commented Sep 6, 2023

hughbe commented Dec 10, 2018 •

edited

Loading

illialarka commented Jan 15, 2019 •

edited

Loading

hughbe commented Apr 7, 2019 •

edited

Loading