Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

System.Security.Cryptography.X509Certificates.Tests.ChainTests.BuildInvalidSignatureTwice #48851

Closed
runfoapp bot opened this issue Feb 26, 2021 · 7 comments · Fixed by #55231
Closed

System.Security.Cryptography.X509Certificates.Tests.ChainTests.BuildInvalidSignatureTwice #48851

runfoapp bot opened this issue Feb 26, 2021 · 7 comments · Fixed by #55231
Labels
area-System.Security test-bug Problem in test source code (most likely)
Milestone
6.0.0

Comments

@runfoapp
Copy link

runfoapp bot commented Feb 26, 2021 •
edited
Loading

Runfo Tracking Issue: System.Security.Cryptography.X509Certificates.Tests.ChainTests.BuildInvalidSignatureTwice

Build Definition Kind Run Name
1082303 runtime PR 51073 net6.0-windows-Debug-x86-CoreCLR_release-Windows.7.Amd64.Open
1079654 runtime PR 50941 net6.0-windows-Debug-x86-CoreCLR_release-Windows.7.Amd64.Open
1079498 runtime PR 50985 net6.0-windows-Debug-x86-CoreCLR_release-Windows.7.Amd64.Open
1071805 runtime PR 50696 net6.0-windows-Debug-x86-CoreCLR_release-Windows.7.Amd64.Open
1071551 runtime Rolling net6.0-windows-Release-x86-CoreCLR_release-Windows.7.Amd64.Open
1071081 runtime Rolling net6.0-windows-Release-x86-CoreCLR_release-Windows.7.Amd64.Open
1068973 runtime PR 50623 net6.0-windows-Debug-x86-CoreCLR_release-Windows.7.Amd64.Open
1063705 runtime PR 50424 net6.0-windows-Debug-x86-CoreCLR_release-Windows.7.Amd64.Open
1062786 runtime Rolling net6.0-windows-Release-x86-CoreCLR_release-Windows.7.Amd64.Open
1056062 runtime PR 50236 net6.0-windows-Debug-x86-CoreCLR_release-Windows.7.Amd64.Open
1051150 runtime PR 50053 net6.0-windows-Debug-x86-CoreCLR_release-Windows.7.Amd64.Open
1049867 runtime PR 49958 net6.0-windows-Debug-x86-CoreCLR_release-Windows.7.Amd64.Open
1046663 runtime PR 49771 net6.0-windows-Debug-x86-CoreCLR_release-Windows.7.Amd64.Open
1045010 runtime PR 48369 net6.0-windows-Debug-x86-CoreCLR_release-Windows.7.Amd64.Open
1043220 runtime Rolling net6.0-windows-Release-x86-CoreCLR_release-Windows.7.Amd64.Open
1041098 runtime PR 49555 net6.0-windows-Debug-x86-CoreCLR_release-Windows.7.Amd64.Open
1037875 runtime Rolling net6.0-windows-Release-x86-CoreCLR_release-Windows.7.Amd64.Open
1033475 runtime PR 48883 net6.0-windows-Debug-x86-CoreCLR_release-Windows.7.Amd64.Open
1033274 runtime PR 49398 net6.0-windows-Debug-x86-CoreCLR_release-Windows.7.Amd64.Open
1028850 runtime PR 49095 net6.0-windows-Debug-x86-CoreCLR_release-Windows.7.Amd64.Open
1028710 runtime PR 49304 net6.0-windows-Debug-x86-CoreCLR_release-Windows.7.Amd64.Open
1026431 runtime PR 49264 net6.0-windows-Debug-x86-CoreCLR_release-Windows.7.Amd64.Open
1026202 runtime PR 49270 net6.0-windows-Debug-x86-CoreCLR_release-Windows.7.Amd64.Open
1024167 runtime PR 49190 net6.0-windows-Debug-x86-CoreCLR_release-Windows.7.Amd64.Open
1018893 runtime Rolling net6.0-windows-Release-x86-CoreCLR_release-Windows.7.Amd64.Open
1018282 runtime PR 48226 net6.0-windows-Debug-x86-CoreCLR_release-Windows.7.Amd64.Open
1016234 runtime PR 48883 net6.0-windows-Debug-x86-CoreCLR_release-Windows.7.Amd64.Open
1014122 runtime PR 48728 net6.0-windows-Debug-x86-CoreCLR_release-Windows.7.Amd64.Open
1009765 runtime Rolling net6.0-windows-Release-x86-CoreCLR_release-Windows.7.Amd64.Open

Build Result Summary

Day Hit Count Week Hit Count Month Hit Count
0 3 16
@dotnet-issue-labeler dotnet-issue-labeler bot added area-System.Security untriaged New issue has not been triaged by the area owner labels Feb 26, 2021
@ghost
Copy link

ghost commented Feb 26, 2021

Tagging subscribers to this area: @bartonjs, @vcsjones, @krwq, @GrabYourPitchforks
See info in area-owners.md if you want to be subscribed.

Issue Details

Runfo Creating Tracking Issue (data being generated)

Author: runfoapp[bot]
Assignees: -
Labels:

area-System.Security, untriaged
Milestone: -

@danmoseley
Copy link
Member 

Assert.Equal() Failure\r\nExpected: NotSignatureValid\r\nActual:   PartialChain


Stack trace
   at System.Security.Cryptography.X509Certificates.Tests.ChainTests.<BuildInvalidSignatureTwice>g__CheckChain|27_0(<>c__DisplayClass27_0& , <>c__DisplayClass27_1& , <>c__DisplayClass27_2& ) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/ChainTests.cs:line 936
   at System.Security.Cryptography.X509Certificates.Tests.ChainTests.BuildInvalidSignatureTwice() in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/ChainTests.cs:line 948

@vcsjones
Copy link
Member

This might be a result of #48460 that I did, I can look in to it this weekend. It switched chains, so maybe the new chain requires AIA fetching that wasn't happening before, and that fetch is flaky / network dependent.

@stephentoub stephentoub mentioned this issue Mar 18, 2021
Closed
@vcsjones vcsjones mentioned this issue Mar 22, 2021
Closed
@vcsjones
Copy link
Member

vcsjones commented Mar 22, 2021 •
edited
Loading

This might be a result of #48460 that I did

Doesn't look related actually, I was looking at a different BuilidInvalidSignatureTwice.

@vcsjones vcsjones mentioned this issue Mar 22, 2021
Closed
@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label Mar 22, 2021
@eerhardt eerhardt mentioned this issue Mar 23, 2021
Merged
@ghost ghost removed the in-pr There is an active PR which will close this issue when it is merged label Mar 23, 2021
@vcsjones
Copy link
Member

This should be fixed by #49532.

@eerhardt
Copy link
Member

I just hit this error in #51344:

https://dev.azure.com/dnceng/public/_build/results?buildId=1093096&view=logs&j=9d8498d2-c5b7-54d8-6df7-a2ce7e14e68c&t=34b459a3-e64e-5ccc-0adc-77ebca2601a8&l=71

https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-51344-merge-8a08e7c4273b4df58c/System.Security.Cryptography.X509Certificates.Tests/console.87c06a0f.log?sv=2019-07-07&se=2021-05-06T18%3A57%3A00Z&sr=c&sp=rl&sig=P%2BojIWkhhvYYq3aIy5nW4sURTLj6wy4VyrXpV3JjNI0%3D

C:\h\w\B0C10A10\w\AD8A094A\e>"C:\h\w\B0C10A10\p\dotnet.exe" exec --runtimeconfig System.Security.Cryptography.X509Certificates.Tests.runtimeconfig.json --depsfile System.Security.Cryptography.X509Certificates.Tests.deps.json xunit.console.dll System.Security.Cryptography.X509Certificates.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing  
  Discovering: System.Security.Cryptography.X509Certificates.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  System.Security.Cryptography.X509Certificates.Tests (found 615 of 668 test cases)
  Starting:    System.Security.Cryptography.X509Certificates.Tests (parallel test collections = on, max threads = 2)
    System.Security.Cryptography.X509Certificates.Tests.ChainTests.BuildInvalidSignatureTwice [FAIL]
      Assert.Equal() Failure
      Expected: NotSignatureValid
      Actual:   PartialChain
      Stack Trace:
        /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/ChainTests.cs(990,0): at System.Security.Cryptography.X509Certificates.Tests.ChainTests.<BuildInvalidSignatureTwice>g__CheckChain|28_0(<>c__DisplayClass28_0& , <>c__DisplayClass28_1& , <>c__DisplayClass28_2& )
        /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/ChainTests.cs(1002,0): at System.Security.Cryptography.X509Certificates.Tests.ChainTests.BuildInvalidSignatureTwice()
  Finished:    System.Security.Cryptography.X509Certificates.Tests
=== TEST EXECUTION SUMMARY ===
   System.Security.Cryptography.X509Certificates.Tests  Total: 1087, Errors: 0, Failed: 1, Skipped: 0, Time: 36.230s

Reopening.

@eerhardt eerhardt reopened this Apr 16, 2021
@eerhardt eerhardt mentioned this issue Apr 16, 2021
Merged
@GrabYourPitchforks GrabYourPitchforks mentioned this issue May 18, 2021
Merged
@vitek-karas vitek-karas mentioned this issue May 25, 2021
Merged
@bartonjs
Copy link
Member

bartonjs commented Jul 3, 2021

On the one hand, I'd love to know why the test isn't being deterministic. On the other (more pragmatic) hand, "PartialChain" is as valid as a chain with NotSignatureValid (see also the macOS and Android implementations).

So we should teach

runtime/src/libraries/System.Security.Cryptography.X509Certificates/tests/ChainTests.cs

Lines 991 to 997 in fe9a54d

Assert.Equal(
X509ChainStatusFlags.NotSignatureValid,
allFlags);
Assert.Equal(3, chain.ChainElements.Count);
Assert.False(valid, $"Chain is valid on execution {iter}");
that PartialChain is OK.

Something like

                    if (allFlags != PartialChain)
                    {
                        Assert.Equal(
                            X509ChainStatusFlags.NotSignatureValid,
                            allFlags);

                        Assert.Equal(3, chain.ChainElements.Count);
                    }

                    Assert.False(valid, $"Chain is valid on execution {iter}");

@bartonjs bartonjs added test-bug Problem in test source code (most likely) and removed untriaged New issue has not been triaged by the area owner labels Jul 3, 2021
@bartonjs bartonjs added this to the 6.0.0 milestone Jul 3, 2021
@vcsjones vcsjones mentioned this issue Jul 6, 2021
Merged
@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label Jul 6, 2021
@ghost ghost removed the in-pr There is an active PR which will close this issue when it is merged label Jul 7, 2021
@ghost ghost locked as resolved and limited conversation to collaborators Aug 6, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
area-System.Security test-bug Problem in test source code (most likely)
Projects
None yet
Milestone
6.0.0
5 participants
@vcsjones @stephentoub @danmoseley @eerhardt @bartonjs