Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use PlatformSpecific attribute to supress platform specific Quic tests #82933

Merged
merged 2 commits into from
Mar 3, 2023

Conversation

wfurt
Copy link
Member

@wfurt wfurt commented Mar 3, 2023

Using ConditionalFact creates noise on platforms where given test would never run.

   ===========================================================================================================
  ~/github/wfurt-runtime2/artifacts/bin/System.Net.Quic.Functional.Tests/Debug/net8.0-osx ~/github/wfurt-runtime2/src/libraries/System.Net.Quic/tests/FunctionalTests
    Discovering: System.Net.Quic.Functional.Tests (method display = ClassAndMethod, method display options = None)
    Discovered:  System.Net.Quic.Functional.Tests (found 4 of 122 test cases)
    Starting:    System.Net.Quic.Functional.Tests (parallel test collections = on, max threads = 16)
      System.Net.Quic.Tests.MsQuicPlatformDetectionTests.SupportedLinuxPlatformsWithMsQuic_IsSupportedIsTrue [SKIP]
        Condition(s) not met: "IsLinux"
      System.Net.Quic.Tests.MsQuicPlatformDetectionTests.SupportedWindowsPlatforms_IsSupportedIsTrue [SKIP]
        Condition(s) not met: "IsWindows", "SupportsTls13"
      System.Net.Quic.Tests.MsQuicPlatformDetectionTests.SupportedLinuxPlatforms_IsSupportedIsTrue [SKIP]
        Condition(s) not met: "IsLinux", "IsInHelix"
    Finished:    System.Net.Quic.Functional.Tests
  === TEST EXECUTION SUMMARY ===
     System.Net.Quic.Functional.Tests  Total: 4, Errors: 0, Failed: 0, Skipped: 3, Time: 3.639s
  ~/github/wfurt-runtime2/src/libraries/System.Net.Quic/tests/FunctionalTests

The PlatformSpecific attributes allows to silently suppress tests that are not even candidates

  ~/github/wfurt-runtime2/artifacts/bin/System.Net.Quic.Functional.Tests/Debug/net8.0-osx ~/github/wfurt-runtime2/src/libraries/System.Net.Quic/tests/FunctionalTests
    Discovering: System.Net.Quic.Functional.Tests (method display = ClassAndMethod, method display options = None)
    Discovered:  System.Net.Quic.Functional.Tests (found 1 of 122 test case)
    Starting:    System.Net.Quic.Functional.Tests (parallel test collections = on, max threads = 16)
    Finished:    System.Net.Quic.Functional.Tests
  === TEST EXECUTION SUMMARY ===
     System.Net.Quic.Functional.Tests  Total: 1, Errors: 0, Failed: 0, Skipped: 0, Time: 2.981s

so we have less noise and possibly less entries in test database.

@wfurt wfurt added test-enhancement Improvements of test source code area-System.Net.Quic labels Mar 3, 2023
@wfurt wfurt requested a review from a team March 3, 2023 06:25
@wfurt wfurt self-assigned this Mar 3, 2023
@ghost
Copy link

ghost commented Mar 3, 2023

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

Using ConditionalFact creates noise on platforms where given test would never run.

   ===========================================================================================================
  ~/github/wfurt-runtime2/artifacts/bin/System.Net.Quic.Functional.Tests/Debug/net8.0-osx ~/github/wfurt-runtime2/src/libraries/System.Net.Quic/tests/FunctionalTests
    Discovering: System.Net.Quic.Functional.Tests (method display = ClassAndMethod, method display options = None)
    Discovered:  System.Net.Quic.Functional.Tests (found 4 of 122 test cases)
    Starting:    System.Net.Quic.Functional.Tests (parallel test collections = on, max threads = 16)
      System.Net.Quic.Tests.MsQuicPlatformDetectionTests.SupportedLinuxPlatformsWithMsQuic_IsSupportedIsTrue [SKIP]
        Condition(s) not met: "IsLinux"
      System.Net.Quic.Tests.MsQuicPlatformDetectionTests.SupportedWindowsPlatforms_IsSupportedIsTrue [SKIP]
        Condition(s) not met: "IsWindows", "SupportsTls13"
      System.Net.Quic.Tests.MsQuicPlatformDetectionTests.SupportedLinuxPlatforms_IsSupportedIsTrue [SKIP]
        Condition(s) not met: "IsLinux", "IsInHelix"
    Finished:    System.Net.Quic.Functional.Tests
  === TEST EXECUTION SUMMARY ===
     System.Net.Quic.Functional.Tests  Total: 4, Errors: 0, Failed: 0, Skipped: 3, Time: 3.639s
  ~/github/wfurt-runtime2/src/libraries/System.Net.Quic/tests/FunctionalTests

The PlatformSpecific attributes allows to silently suppress tests that are not even candidates

  ~/github/wfurt-runtime2/artifacts/bin/System.Net.Quic.Functional.Tests/Debug/net8.0-osx ~/github/wfurt-runtime2/src/libraries/System.Net.Quic/tests/FunctionalTests
    Discovering: System.Net.Quic.Functional.Tests (method display = ClassAndMethod, method display options = None)
    Discovered:  System.Net.Quic.Functional.Tests (found 1 of 122 test case)
    Starting:    System.Net.Quic.Functional.Tests (parallel test collections = on, max threads = 16)
    Finished:    System.Net.Quic.Functional.Tests
  === TEST EXECUTION SUMMARY ===
     System.Net.Quic.Functional.Tests  Total: 1, Errors: 0, Failed: 0, Skipped: 0, Time: 2.981s

so we have less noise and possibly less entries in test database.

Author: wfurt
Assignees: wfurt
Labels:

test-enhancement, area-System.Net.Quic

Milestone: -

…formDetectionTests.cs

Co-authored-by: Stephen Toub <stoub@microsoft.com>
@wfurt wfurt merged commit 3853cb8 into dotnet:main Mar 3, 2023
@wfurt wfurt deleted the support branch March 3, 2023 21:00
@karelz karelz added this to the 8.0.0 milestone Mar 22, 2023
@ghost ghost locked as resolved and limited conversation to collaborators Apr 21, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-System.Net.Quic test-enhancement Improvements of test source code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants