Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test failure: System.Security.Cryptography.X509Certificates.Tests.CertificateCreation.CertificateRequestChainTests/CreateChain_Hybrid #25979

Closed
ghost opened this issue Apr 24, 2018 · 34 comments
Labels
area-System.Security Known Build Error Use this to report build issues in the .NET Helix tab tracking-external-issue The issue is caused by external problem (e.g. OS) - nothing we can do to fix it directly

Comments

@ghost
Copy link

ghost commented Apr 24, 2018

{
  "ErrorMessage": "Chain is valid on execution 1",
  "BuildRetry": false,
  "ErrorPattern": "",
  "ExcludeConsoleLog": false
}

The test System.Security.Cryptography.X509Certificates.Tests.CertificateCreation.CertificateRequestChainTests/CreateChain_Hybrid has failed.

Internal.Cryptography.CryptoThrowHelper+WindowsCryptographicException : An internal consistency check failed

        Stack Trace:

           at System.Security.Cryptography.CngKeyLite.GenerateNewExportableKey(String algorithm, Int32 keySize) in E:\A\_work\2\s\corefx\src\System.Security.Cryptography.Algorithms\src\System\Security\Cryptography\CngKeyLite.cs:line 138
       at System.Security.Cryptography.RSAImplementation.RSACng.GetDuplicatedKeyHandle() in E:\A\_work\2\s\corefx\src\System.Security.Cryptography.Algorithms\src\System\Security\Cryptography\RSACng.cs:line 34
       at System.Security.Cryptography.RSAImplementation.RSACng.ExportKeyBlob(Boolean includePrivateParameters) in E:\A\_work\2\s\corefx\src\System.Security.Cryptography.Algorithms\src\System\Security\Cryptography\RSACng.cs:line 48
       at System.Security.Cryptography.RSAImplementation.RSACng.ExportParameters(Boolean includePrivateParameters) in E:\A\_work\2\s\corefx\src\Common\src\System\Security\Cryptography\RSACng.ImportExport.cs:line 118
       at System.Security.Cryptography.X509Certificates.RSAPkcs1X509SignatureGenerator.BuildPublicKey(RSA rsa) in E:\A\_work\2\s\corefx\src\System.Security.Cryptography.X509Certificates\src\System\Security\Cryptography\X509Certificates\RSAPkcs1X509SignatureGenerator.cs:line 33
       at System.Security.Cryptography.X509Certificates.RSAPkcs1X509SignatureGenerator.BuildPublicKey() in E:\A\_work\2\s\corefx\src\System.Security.Cryptography.X509Certificates\src\System\Security\Cryptography\X509Certificates\RSAPkcs1X509SignatureGenerator.cs:line 28
       at System.Security.Cryptography.X509Certificates.CertificateRequest..ctor(String subjectName, RSA key, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding) in E:\A\_work\2\s\corefx\src\System.Security.Cryptography.X509Certificates\src\System\Security\Cryptography\X509Certificates\CertificateRequest.cs:line 136
       at System.Security.Cryptography.X509Certificates.Tests.CertificateCreation.CertificateRequestChainTests.OpenCertRequest(String dn, AsymmetricAlgorithm key, HashAlgorithmName hashAlgorithm) in E:\A\_work\2\s\corefx\src\System.Security.Cryptography.X509Certificates\tests\CertificateCreation\CertificateRequestChainTests.cs:line 210
       at System.Security.Cryptography.X509Certificates.Tests.CertificateCreation.CertificateRequestChainTests.CreateChainRequest(String dn, AsymmetricAlgorithm key, HashAlgorithmName hashAlgorithm, Boolean isCa, Nullable`1 pathLen) in E:\A\_work\2\s\corefx\src\System.Security.Cryptography.X509Certificates\tests\CertificateCreation\CertificateRequestChainTests.cs:line 252
       at System.Security.Cryptography.X509Certificates.Tests.CertificateCreation.CertificateRequestChainTests.CreateAndTestChain(AsymmetricAlgorithm rootPrivKey, AsymmetricAlgorithm intermed1PrivKey, AsymmetricAlgorithm intermed2PrivKey, AsymmetricAlgorithm leafPubKey) in E:\A\_work\2\s\corefx\src\System.Security.Cryptography.X509Certificates\tests\CertificateCreation\CertificateRequestChainTests.cs:line 361
       at System.Security.Cryptography.X509Certificates.Tests.CertificateCreation.CertificateRequestChainTests.CreateChain_Hybrid() in E:\A\_work\2\s\corefx\src\System.Security.Cryptography.X509Certificates\tests\CertificateCreation\CertificateRequestChainTests.cs:line 60
Build : Master - 20180423.04 (Core Tests)

Failing configurations:

  • Windows.81.Amd64-x86
    • Release

Runfo Tracking Issue: buildinvalidsignaturetwice

Build Definition Kind Run Name
267807 runtime PR 84864 net8.0-linux-Debug-arm64-Mono_release-(Ubuntu.1804.Arm64.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm64v8
267679 runtime PR 85858 net8.0-linux-Release-arm64-CoreCLR_checked-(Alpine.315.Arm64.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.15-helix-arm64v8
267024 runtime PR 85966 net8.0-linux-Debug-arm64-Mono_release-(Debian.11.Arm64.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:debian-11-helix-arm64v8
266120 runtime PR 84864 net8.0-linux-Release-arm64-CoreCLR_checked-(Alpine.315.Arm64.Open)Ubuntu.1804.ArmArch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.15-helix-arm64v8
247295 runtime PR 85151 net8.0-linux-Debug-x64-CoreCLR_checked-(Alpine.315.Amd64.Open)Ubuntu.1804.Amd64.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:alpine-3.15-helix-amd64
240573 runtime PR 84824 net8.0-windows-Release-x64-CoreCLR_checked-Windows.10.Amd64.Open
233949 runtime PR 84591 net8.0-windows-Debug-x86-CoreCLR_checked-Windows.10.Amd64.Open

Build Result Summary

Day Hit Count Week Hit Count Month Hit Count
3 4 7

Known Issue Error Message

Fill the error message using known issues guidance.

{
  "ErrorMessage": "",
  "BuildRetry": false
}
@danmoseley
Copy link
Member

Not much to go on but we can start tracking ..

@bartonjs
Copy link
Member

NTE_INTERNAL_ERROR from asking CNG to generate an RSA key. Without a stable repro it'll be hard to do anything. We can leave this open for a little bit to see if maybe we have a bad machine and can track down what this particular one means.

But NTE_INTERNAL_ERROR is the moral equivalent of Debug.Assert. A late check says that something previously went wrong and didn't notice, and now it doesn't know what went wrong or why, just that hum was not the right answer.

@ViktorHofer
Copy link
Member

Same cause on a different test System.Security.Cryptography.Dsa.Tests.DSASignVerify_Stream/InvalidKeySize_DoesNotInvalidateKey.

https://mc.dot.net/#/user/dotnet-bot/pr~2Fdotnet~2Fcorefx~2Frefs~2Fpull~2F37451~2Fmerge/test~2Ffunctional~2Fcli~2Finnerloop~2F/20190529.12/workItem/System.Security.Cryptography.Cng.Tests/analysis/xunit/System.Security.Cryptography.Dsa.Tests.DSASignVerify_Stream~2FInvalidKeySize_DoesNotInvalidateKey

Internal.Cryptography.CryptoThrowHelper+WindowsCryptographicException : An internal consistency check failed.

   at System.Security.Cryptography.CngKey.Create(CngAlgorithm algorithm, String keyName, CngKeyCreationParameters creationParameters) in /_/src/System.Security.Cryptography.Cng/src/System/Security/Cryptography/CngKey.Create.cs:line 55
   at Internal.Cryptography.CngAlgorithmCore.GetOrGenerateKey(Int32 keySize, CngAlgorithm algorithm) in /_/src/System.Security.Cryptography.Cng/src/Internal/Cryptography/CngAlgorithmCore.cs:line 63
   at System.Security.Cryptography.DSACng.get_Key() in /_/src/System.Security.Cryptography.Cng/src/System/Security/Cryptography/DSACng.Key.cs:line 25
   at System.Security.Cryptography.DSACng.GetDuplicatedKeyHandle() in /_/src/System.Security.Cryptography.Cng/src/System/Security/Cryptography/DSACng.Key.cs:line 52
   at System.Security.Cryptography.DSACng.ComputeQLength() in /_/src/Common/src/System/Security/Cryptography/DSACng.SignVerify.cs:line 118
   at System.Security.Cryptography.DSACng.AdjustHashSizeIfNecessary(ReadOnlySpan`1 hash, Span`1 stackBuf) in /_/src/Common/src/System/Security/Cryptography/DSACng.SignVerify.cs:line 96
   at System.Security.Cryptography.DSACng.CreateSignature(Byte[] rgbHash) in /_/src/Common/src/System/Security/Cryptography/DSACng.SignVerify.cs:line 33
   at System.Security.Cryptography.DSA.SignData(Stream data, HashAlgorithmName hashAlgorithm) in /_/src/System.Security.Cryptography.Algorithms/src/System/Security/Cryptography/DSA.cs:line 102
   at System.Security.Cryptography.Dsa.Tests.DSASignVerify_Stream.SignData(DSA dsa, Byte[] data, HashAlgorithmName hashAlgorithm) in /_/src/Common/tests/System/Security/Cryptography/AlgorithmImplementations/DSA/DSASignVerify.cs:line 51
   at System.Security.Cryptography.Dsa.Tests.DSASignVerify.InvalidKeySize_DoesNotInvalidateKey() in /_/src/Common/tests/System/Security/Cryptography/AlgorithmImplementations/DSA/DSASignVerify.cs:line 101

What can we about this kind of flakiness? It happened on the same configuration, thus should we disable it for Win81?

@bartonjs
Copy link
Member

I don't think we can disable all of crypto on Win8.1 😄.

I've reached out to Windows CNG to see if they have any advice.

@bartonjs
Copy link
Member

Interesting that it's the same test as before. While sending data to Windows I noticed that the second ever report of this error was for RSA (not DSA) and on Windows 10. The first was DSA, Windows version unknown, a different test.

If there's a semi-reliable way to force this to happen I'd love to send a repro to Windows and let them catch it in action.

@ViktorHofer
Copy link
Member

I would try to run the test on a Windows 81 machine in a loop (200-500 times should suffice).

@bartonjs
Copy link
Member

Windows suggests that this error code mostly means there's a disk problem (e.g. disk full). If the frequency has started to pick up on this then perhaps our Windows 8.1 machines/images need some sort of cleanup.

@ViktorHofer
Copy link
Member

@MattGal disk space problems on win81 (see above). Can you please verify?

@bartonjs
Copy link
Member

bartonjs commented Jun 3, 2019

Even if the disk/drive as a whole is fine, it may be that the CNG key directories are having issues (https://docs.microsoft.com/en-us/windows/desktop/seccng/key-storage-and-retrieval#key-directories-and-files).

The most likely culprit being %APPDATA%\Microsoft\Crypto\Keys for the user the test runs as.

@ViktorHofer
Copy link
Member

What's your recommendation? Move all tests to Outerloop for that one configuration? Or could there be a way to diagnose the health of the machine?

@bartonjs
Copy link
Member

bartonjs commented Jun 3, 2019

Move all tests to Outerloop for that one configuration?

It would literally mean every crypto test and every System.Net.Security test. So, I don't think that would be a good strategy (particularly if the problem is environmental).

Could there be a way to diagnose the health of the machine?

I'm open to suggestion. Adding a cctor which prints the size on disk remaining and number of files in %APPDATA%\Microsoft\Crypto\Keys until we get a hit is maybe reasonable, but since we've seen it from at least three different test assemblies it feels like maybe playing whack-a-mole. Printing it for absolutely every test library ever seems extreme (but with good coverage).

Can we inject a diagnostic dir command into RunTests.bat from corefx which we can see via MC for the helix job console capture?

@MattGal
Copy link
Member

MattGal commented Jun 3, 2019

@bartonjs / @ViktorHofer Unfortunately the machine(s) that ran your tests have long since been recycled, but I am very skeptical of the "disk full" theory going on here. The Windows 8.1 VMs get created with 100+ GB free disk space on the C:\ drive where %APPDATA% is located, refuse to take work if there's less than 3 GB available on this disk, and get deleted whenever there isn't active work to be done.

Have you actually run this on a Windows 8.1 / Server 2K12R2 machine? It might be an actual problem, though it could be caused by other work items messing with the machine state too.

@bartonjs
Copy link
Member

bartonjs commented Jun 3, 2019

@MattGal Yep, my main workstation is 2K12R2. I've never seen NTE_INTERNAL_ERROR on it.

@MattGal
Copy link
Member

MattGal commented Jun 3, 2019

@bartonjs good to hear it works on your box :) I'm standing up a machine to investigate and will share a repro if I get you one.

@MattGal
Copy link
Member

MattGal commented Jun 3, 2019

@bartonjs no repro, I'm letting it loop a few hundred times on a machine with the same image to check it out. I also checked the execution history of a00007K and noted only 40 other work items ran before it (none of which, in a spot check, logged "now ruining certs on the machine", though they could be at fault) ... this would indicate the odds of the disk being full when that happeend are close to 0.

I think the best possible way to investigate this would be to catch it at a time when the machine having the trouble is still alive, and let me know via Teams so I can make it not get cleaned up, and jump to it.

@ViktorHofer
Copy link
Member

I think the best possible way to investigate this would be to catch it at a time when the machine having the trouble is still alive, and let me know via Teams so I can make it not get cleaned up, and jump to it.

Thanks a lot for spending time on this. I will let you when it happens again.

@msftgits msftgits transferred this issue from dotnet/corefx Jan 31, 2020
@msftgits msftgits added this to the Future milestone Jan 31, 2020
@maryamariyan maryamariyan added the untriaged New issue has not been triaged by the area owner label Feb 23, 2020
@bartonjs bartonjs removed the untriaged New issue has not been triaged by the area owner label Jul 7, 2020
@trylek
Copy link
Member

trylek commented Jul 21, 2020

Just hit something similar in

https://dev.azure.com/dnceng/public/_build/results?buildId=739309&view=ms.vss-test-web.build-test-results-tab

in the test

System.Security.Cryptography.Dsa.Tests.DSASignVerify_Array.InvalidKeySize_DoesNotInvalidateKey
  Starting:    System.Security.Cryptography.Cng.Tests (parallel test collections = on, max threads = 2)
    System.Security.Cryptography.Dsa.Tests.DSASignVerify_Array.InvalidKeySize_DoesNotInvalidateKey [FAIL]
      Internal.Cryptography.CryptoThrowHelper+WindowsCryptographicException : An internal consistency check failed.
      Stack Trace:
        /_/src/libraries/System.Security.Cryptography.Cng/src/System/Security/Cryptography/CngKey.Create.cs(54,0): at System.Security.Cryptography.CngKey.Create(CngAlgorithm algorithm, String keyName, CngKeyCreationParameters creationParameters)
        /_/src/libraries/System.Security.Cryptography.Cng/src/Internal/Cryptography/CngAlgorithmCore.cs(72,0): at Internal.Cryptography.CngAlgorithmCore.GetOrGenerateKey(Int32 keySize, CngAlgorithm algorithm)
        /_/src/libraries/System.Security.Cryptography.Cng/src/System/Security/Cryptography/DSACng.Key.cs(24,0): at System.Security.Cryptography.DSACng.get_Key()
        /_/src/libraries/System.Security.Cryptography.Cng/src/System/Security/Cryptography/DSACng.Key.cs(51,0): at System.Security.Cryptography.DSACng.GetDuplicatedKeyHandle()
        /_/src/libraries/Common/src/System/Security/Cryptography/DSACng.SignVerify.cs(178,0): at System.Security.Cryptography.DSACng.ComputeQLength()
        /_/src/libraries/Common/src/System/Security/Cryptography/DSACng.SignVerify.cs(156,0): at System.Security.Cryptography.DSACng.AdjustHashSizeIfNecessary(ReadOnlySpan`1 hash, Span`1 stackBuf)
        /_/src/libraries/Common/src/System/Security/Cryptography/DSACng.SignVerify.cs(32,0): at System.Security.Cryptography.DSACng.CreateSignature(Byte[] rgbHash)
        /_/src/libraries/System.Security.Cryptography.Algorithms/src/System/Security/Cryptography/DSA.cs(136,0): at System.Security.Cryptography.DSA.SignData(Byte[] data, Int32 offset, Int32 count, HashAlgorithmName hashAlgorithm)
        /_/src/libraries/System.Security.Cryptography.Algorithms/src/System/Security/Cryptography/DSA.cs(88,0): at System.Security.Cryptography.DSA.SignData(Byte[] data, HashAlgorithmName hashAlgorithm)
        /_/src/libraries/Common/tests/System/Security/Cryptography/AlgorithmImplementations/DSA/DSASignVerify.cs(13,0): at System.Security.Cryptography.Dsa.Tests.DSASignVerify_Array.SignData(DSA dsa, Byte[] data, HashAlgorithmName hashAlgorithm)
        /_/src/libraries/Common/tests/System/Security/Cryptography/AlgorithmImplementations/DSA/DSASignVerify.cs(119,0): at System.Security.Cryptography.Dsa.Tests.DSASignVerify.InvalidKeySize_DoesNotInvalidateKey()
    System.Security.Cryptography.Rsa.Tests.KeyGeneration.GenerateMaxKey [SKIP]
      Condition(s) not met: "IsStressModeEnabled"
  Finished:    System.Security.Cryptography.Cng.Tests
=== TEST EXECUTION SUMMARY ===
   System.Security.Cryptography.Cng.Tests  Total: 1165, Errors: 0, Failed: 1, Skipped: 1, Time: 78.518s

@bartonjs bartonjs added the tracking-external-issue The issue is caused by external problem (e.g. OS) - nothing we can do to fix it directly label Jul 21, 2020
@MattGal
Copy link
Member

MattGal commented Jul 21, 2020

Interesting. This is the only failure like this in the past 1000+ runnings of this work item, but the machine is actually still around; the machine this failed on actually passed this test around 2020-07-21 12:16 UTC, then failed it in this run at 2020-07-21 14:52, running 32 other work items in between all of which passed and didn't crash.

If there's something super interesting about the machine doing this, ping me ASAP and I can try to keep it around, otherwise I'd chalk this up to an unusual windows state issue.

@v-haren
Copy link

v-haren commented Aug 10, 2020

failed again in job: runtime-coreclr libraries-jitstress2-jitstressregs 20200808.1

failed test: System.Security.Cryptography.X509Certificates.Tests.DynamicChainTests.BuildInvalidSignatureTwice(endEntityErrors: NotSignatureValid, intermediateErrors: NoError, rootErrors: UntrustedRoot)

Error message

Internal.Cryptography.CryptoThrowHelper+WindowsCryptographicException : An internal consistency check failed.


Stack trace
   at System.Security.Cryptography.CngKeyLite.GenerateNewExportableKey(String algorithm, Int32 keySize) in /_/src/libraries/System.Security.Cryptography.Algorithms/src/System/Security/Cryptography/CngKeyLite.cs:line 408
   at System.Security.Cryptography.RSAImplementation.RSACng.GetDuplicatedKeyHandle() in /_/src/libraries/System.Security.Cryptography.Algorithms/src/System/Security/Cryptography/RSACng.cs:line 44
   at System.Security.Cryptography.RSAImplementation.RSACng.ExportKeyBlob(Boolean includePrivateParameters) in /_/src/libraries/System.Security.Cryptography.Algorithms/src/System/Security/Cryptography/RSACng.cs:line 58
   at System.Security.Cryptography.RSAImplementation.RSACng.ExportParameters(Boolean includePrivateParameters) in /_/src/libraries/Common/src/System/Security/Cryptography/RSACng.ImportExport.cs:line 296
   at System.Security.Cryptography.RSA.ExportRSAPublicKey() in /_/src/libraries/System.Security.Cryptography.Algorithms/src/System/Security/Cryptography/RSA.cs:line 303
   at System.Security.Cryptography.X509Certificates.RSAPkcs1X509SignatureGenerator.BuildPublicKey(RSA rsa) in /_/src/libraries/System.Security.Cryptography.X509Certificates/src/System/Security/Cryptography/X509Certificates/RSAPkcs1X509SignatureGenerator.cs:line 36
   at System.Security.Cryptography.X509Certificates.RSAPkcs1X509SignatureGenerator.BuildPublicKey() in /_/src/libraries/System.Security.Cryptography.X509Certificates/src/System/Security/Cryptography/X509Certificates/RSAPkcs1X509SignatureGenerator.cs:line 27
   at System.Security.Cryptography.X509Certificates.CertificateRequest..ctor(String subjectName, RSA key, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding) in /_/src/libraries/System.Security.Cryptography.X509Certificates/src/System/Security/Cryptography/X509Certificates/CertificateRequest.cs:line 139
   at System.Security.Cryptography.X509Certificates.Tests.TestDataGenerator.MakeTestChain(ReadOnlySpan`1 keys, Span`1 certs, IEnumerable`1 endEntityExtensions, IEnumerable`1 intermediateExtensions, IEnumerable`1 rootExtensions) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/TestDataGenerator.cs:line 179
   at System.Security.Cryptography.X509Certificates.Tests.TestDataGenerator.MakeTestChain3(X509Certificate2& endEntityCert, X509Certificate2& intermediateCert, X509Certificate2& rootCert, IEnumerable`1 endEntityExtensions, IEnumerable`1 intermediateExtensions, IEnumerable`1 rootExtensions) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/TestDataGenerator.cs:line 30
   at System.Security.Cryptography.X509Certificates.Tests.DynamicChainTests.BuildInvalidSignatureTwice(X509ChainStatusFlags endEntityErrors, X509ChainStatusFlags intermediateErrors, X509ChainStatusFlags rootErrors) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/DynamicChainTests.cs:line 61

@MattGal
Copy link
Member

MattGal commented Aug 10, 2020

failed again in job: runtime-coreclr libraries-jitstress2-jitstressregs 20200808.1

failed test: System.Security.Cryptography.X509Certificates.Tests.DynamicChainTests.BuildInvalidSignatureTwice(endEntityErrors: NotSignatureValid, intermediateErrors: NoError, rootErrors: UntrustedRoot)

Error message

Internal.Cryptography.CryptoThrowHelper+WindowsCryptographicException : An internal consistency check failed.


Stack trace
   at System.Security.Cryptography.CngKeyLite.GenerateNewExportableKey(String algorithm, Int32 keySize) in /_/src/libraries/System.Security.Cryptography.Algorithms/src/System/Security/Cryptography/CngKeyLite.cs:line 408
   at System.Security.Cryptography.RSAImplementation.RSACng.GetDuplicatedKeyHandle() in /_/src/libraries/System.Security.Cryptography.Algorithms/src/System/Security/Cryptography/RSACng.cs:line 44
   at System.Security.Cryptography.RSAImplementation.RSACng.ExportKeyBlob(Boolean includePrivateParameters) in /_/src/libraries/System.Security.Cryptography.Algorithms/src/System/Security/Cryptography/RSACng.cs:line 58
   at System.Security.Cryptography.RSAImplementation.RSACng.ExportParameters(Boolean includePrivateParameters) in /_/src/libraries/Common/src/System/Security/Cryptography/RSACng.ImportExport.cs:line 296
   at System.Security.Cryptography.RSA.ExportRSAPublicKey() in /_/src/libraries/System.Security.Cryptography.Algorithms/src/System/Security/Cryptography/RSA.cs:line 303
   at System.Security.Cryptography.X509Certificates.RSAPkcs1X509SignatureGenerator.BuildPublicKey(RSA rsa) in /_/src/libraries/System.Security.Cryptography.X509Certificates/src/System/Security/Cryptography/X509Certificates/RSAPkcs1X509SignatureGenerator.cs:line 36
   at System.Security.Cryptography.X509Certificates.RSAPkcs1X509SignatureGenerator.BuildPublicKey() in /_/src/libraries/System.Security.Cryptography.X509Certificates/src/System/Security/Cryptography/X509Certificates/RSAPkcs1X509SignatureGenerator.cs:line 27
   at System.Security.Cryptography.X509Certificates.CertificateRequest..ctor(String subjectName, RSA key, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding) in /_/src/libraries/System.Security.Cryptography.X509Certificates/src/System/Security/Cryptography/X509Certificates/CertificateRequest.cs:line 139
   at System.Security.Cryptography.X509Certificates.Tests.TestDataGenerator.MakeTestChain(ReadOnlySpan`1 keys, Span`1 certs, IEnumerable`1 endEntityExtensions, IEnumerable`1 intermediateExtensions, IEnumerable`1 rootExtensions) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/TestDataGenerator.cs:line 179
   at System.Security.Cryptography.X509Certificates.Tests.TestDataGenerator.MakeTestChain3(X509Certificate2& endEntityCert, X509Certificate2& intermediateCert, X509Certificate2& rootCert, IEnumerable`1 endEntityExtensions, IEnumerable`1 intermediateExtensions, IEnumerable`1 rootExtensions) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/TestDataGenerator.cs:line 30
   at System.Security.Cryptography.X509Certificates.Tests.DynamicChainTests.BuildInvalidSignatureTwice(X509ChainStatusFlags endEntityErrors, X509ChainStatusFlags intermediateErrors, X509ChainStatusFlags rootErrors) in /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/DynamicChainTests.cs:line 61

To quote @bartonjs :

... many other similar test methods with identical codepaths probably succeeded after it. It's something in Windows-land that I can't get them to be interested in investigating; so unless we catch a test like this failing under time travel debugging (which is incredibly expensive in both clock time and disk space) there's nothing to do about it.

@Anipik Anipik added the blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' label Feb 25, 2021
@krwq
Copy link
Member

krwq commented Oct 1, 2021

@bartonjs since this has been open for more than 3 years and we don't have any good path forward to fix or repro this would it make sense to perhaps add retry logic inside GenerateNewExportableKey and observe if that helps? If we find this doesn't help we could revert the change

@danmoseley
Copy link
Member

danmoseley commented Jun 15, 2022

Assuming this query below catches all failures covered by this issue (?) then it's failed 4 times in the last 2 months.

Failures were within either
at System.Security.Cryptography.CngKeyLite.GenerateNewExportableKey(String algorithm, Int32 keySize) in /src/libraries/System.Security.Cryptography.Algorithms/src/System/Security/Cryptography/CngKeyLite.cs:line 411
or
at System.Security.Cryptography.CngKey.Create(CngAlgorithm algorithm, String keyName, CngKeyCreationParameters creationParameters) in /src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/CngKey.Create.cs:line 57

@bartonjs @vcsjones thoughts about some kind of workaround here? without a plan I guess we'll just hit this in a PR periodically, albeit not super often.

https://engsrvprod.kusto.windows.net/engineeringdata?query=H4sIAAAAAAAEAI1RTWvCQBC9F%2fwPQ04GrLb0nEKrCBa0tgn0KOtmNKNxN8xOkEB%2ffDcbbXPowdPOzPvYtzsZOvlEV5fiBnffcLBk4EgmT8gYZPiyfFwInhxY89ss8v%2bob3YbWP7sCOcCGWFOhlyBOTwnYOx5eP%2f0kMd%2fcHc3JAlEc0Vl1BMyocnLZqVOCE4UizuTFBCljfMhxinqmkma8ZSbSuyeVVU0457BEp1TewRtjSgyDiJfOfJioxvQBepjYFdsD6gFBncAa6qwJIOQgFgnTGY%2frBQ73BycNcM12wpZCF08nuHOP03ImjZhPOrLN1NrdrSvWbX4DWa6z%2b%2b83lOv%2b6ixxtY%2fjF5YFzeYKU8j8U%2bqGeNWN5lkfs1emTUVPo66UffzbT267uiCzC45QrtEKewVea2pzG9IEHir%2brRFji8uYReh7i92MgmjUSpKHzNWGtv2B7Hpv%2faVAgAA&web=0

TestResults
| join kind=inner WorkItems on WorkItemId
| join kind=inner Jobs on JobId
| where Finished >= now(-60d)
| where Result == "Fail"
| where FriendlyName startswith "System.Security.Cryptography."
| where Message contains "consistency check"
| project 
  Pipeline = tostring(parse_json(Properties).DefinitionName),
  Pipeline_Configuration = tostring(parse_json(Properties).configuration),
  OS = QueueName,
  Arch = tostring(parse_json(Properties).architecture)
  //Test = Type1,
  //Result
  ,Finished,
  //Duration,
  Method,
  //Build = tostring(parse_json(Properties).BuildNumber),
  Message,
  FriendlyName//,
  ,StackTrace
Pipeline Pipeline_Configuration OS Arch Finished Method Message FriendlyName StackTrace
runtime Debug windows.81.amd64.open.rt x64 2022-05-09 02:48:48.6800000 CreateWithKeysize_BigKeys(keySizeInBits: 1088) Internal.Cryptography.CryptoThrowHelper+WindowsCryptographicException : An internal consistency check failed. System.Security.Cryptography.Tests at System.Security.Cryptography.CngKey.Create(CngAlgorithm algorithm, String keyName, CngKeyCreationParameters creationParameters) in //src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/CngKey.Create.cs:line 57
at System.Security.Cryptography.CngAlgorithmCore.GetOrGenerateKey(Int32 keySize, CngAlgorithm algorithm) in /
/src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/CngAlgorithmCore.cs:line 72
at System.Security.Cryptography.DSACng.get_Key() in //src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/DSACng.Key.cs:line 24
at System.Security.Cryptography.DSACng.ExportKeyBlob(Boolean includePrivateParameters) in /
/src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/DSACng.ImportExport.cs:line 15
at System.Security.Cryptography.DSACng.ExportParameters(Boolean includePrivateParameters) in //src/libraries/Common/src/System/Security/Cryptography/DSACng.ImportExport.cs:line 314
at System.Security.Cryptography.DSAWrapper.ExportParameters(Boolean includePrivateParameters) in /
/src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/DSAWrapper.cs:line 21
at System.Security.Cryptography.Tests.DSACreateTests.CreateWithKeysize_BigKeys(Int32 keySizeInBits) in //src/libraries/System.Security.Cryptography/tests/DSACreateTests.cs:line 54
at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
at System.Reflection.RuntimeMethodInfo.InvokeNonEmitUnsafe(Object obj, IntPtr* arguments, Span`1 argsForTemporaryMonoSupport, BindingFlags invokeAttr) in /
/src/coreclr/System.Private.CoreLib/src/System/Reflection/RuntimeMethodInfo.CoreCLR.cs:line 387
runtime-coreclr libraries-pgo Release windows.10.amd64.open.rt x64 2022-05-28 17:37:32.8230000 TestInvalidAia System.Security.Cryptography.CryptographicException : An internal consistency check failed. System.Security.Cryptography.X509Certificates.Tests at System.Security.Cryptography.CngKey.Create(CngAlgorithm algorithm, String keyName, CngKeyCreationParameters creationParameters) in //src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/CngKey.Create.cs:line 57
at System.Security.Cryptography.CngAlgorithmCore.GetOrGenerateKey(Int32 keySize, CngAlgorithm algorithm) in /
/src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/CngAlgorithmCore.cs:line 76
at System.Security.Cryptography.RSACng.get_Key() in //src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/RSACng.Key.cs:line 24
at System.Security.Cryptography.RSACng.ExportKeyBlob(Boolean includePrivateParameters) in /
/src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/RSACng.ImportExport.cs:line 35
at System.Security.Cryptography.RSACng.ExportParameters(Boolean includePrivateParameters) in //src/libraries/Common/src/System/Security/Cryptography/RSACng.ImportExport.cs:line 282
at System.Security.Cryptography.RSA.WritePkcs1PublicKey() in /
/src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/RSA.cs:line 428
at System.Security.Cryptography.RSA.ExportRSAPublicKey() in //src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/RSA.cs:line 292
at System.Security.Cryptography.RSAWrapper.ExportRSAPublicKey() in /
/src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/RSAWrapper.cs:line 141
at System.Security.Cryptography.X509Certificates.RSAPkcs1X509SignatureGenerator.BuildPublicKey(RSA rsa) in //src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/X509Certificates/RSAPkcs1X509SignatureGenerator.cs:line 36
at System.Security.Cryptography.X509Certificates.RSAPkcs1X509SignatureGenerator.BuildPublicKey() in /
/src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/X509Certificates/RSAPkcs1X509SignatureGenerator.cs:line 27
at System.Security.Cryptography.X509Certificates.X509SignatureGenerator.get_PublicKey() in //src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/X509Certificates/X509SignatureGenerator.cs:line 16
at System.Security.Cryptography.X509Certificates.CertificateRequest..ctor(String subjectName, RSA key, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding) in /
/src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/X509Certificates/CertificateRequest.cs:line 131
at System.Security.Cryptography.X509Certificates.Tests.DynamicChainTests.TestInvalidAia() in //src/libraries/System.Security.Cryptography.X509Certificates/tests/DynamicChainTests.cs:line 378
at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr) in /
/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodInvoker.cs:line 69
runtime-libraries-coreclr outerloop-windows Release windows.81.amd64.open.svc x64 2022-05-30 12:34:18.4950000 RevokeEndEntity Internal.Cryptography.CryptoThrowHelper+WindowsCryptographicException : An internal consistency check failed. System.Security.Cryptography.X509Certificates.Tests at System.Security.Cryptography.CngKeyLite.GenerateNewExportableKey(String algorithm, Int32 keySize) in //src/libraries/System.Security.Cryptography.Algorithms/src/System/Security/Cryptography/CngKeyLite.cs:line 411
at System.Security.Cryptography.RSAImplementation.RSACng.GetDuplicatedKeyHandle() in /
/src/libraries/System.Security.Cryptography.Algorithms/src/System/Security/Cryptography/RSACng.cs:line 53
at System.Security.Cryptography.RSAImplementation.RSACng.ExportKeyBlob(Boolean includePrivateParameters) in //src/libraries/System.Security.Cryptography.Algorithms/src/System/Security/Cryptography/RSACng.cs:line 66
at System.Security.Cryptography.RSAImplementation.RSACng.ExportParameters(Boolean includePrivateParameters) in /
/src/libraries/Common/src/System/Security/Cryptography/RSACng.ImportExport.cs:line 299
at System.Security.Cryptography.RSA.ExportRSAPublicKey() in //src/libraries/System.Security.Cryptography.Algorithms/src/System/Security/Cryptography/RSA.cs:line 305
at System.Security.Cryptography.X509Certificates.RSAPkcs1X509SignatureGenerator.BuildPublicKey(RSA rsa) in /
/src/libraries/System.Security.Cryptography.X509Certificates/src/System/Security/Cryptography/X509Certificates/RSAPkcs1X509SignatureGenerator.cs:line 36
at System.Security.Cryptography.X509Certificates.CertificateRequest..ctor(String subjectName, RSA key, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding) in //src/libraries/System.Security.Cryptography.X509Certificates/src/System/Security/Cryptography/X509Certificates/CertificateRequest.cs:line 130
at System.Security.Cryptography.X509Certificates.Tests.Common.CertificateAuthority.CreateCertificate(String subject, RSA publicKey, TimeSpan nestingBuffer, X509ExtensionCollection extensions, Boolean ocspResponder) in /
/src/libraries/Common/tests/System/Security/Cryptography/X509Certificates/CertificateAuthority.cs:line 258
at System.Security.Cryptography.X509Certificates.Tests.Common.CertificateAuthority.BuildPrivatePki(PkiOptions pkiOptions, RevocationResponder& responder, CertificateAuthority& rootAuthority, CertificateAuthority& intermediateAuthority, X509Certificate2& endEntityCert, String testName, Boolean registerAuthorities, Boolean pkiOptionsInSubject, String subjectName, Int32 keySize, X509ExtensionCollection extensions) in //src/libraries/Common/tests/System/Security/Cryptography/X509Certificates/CertificateAuthority.cs:line 926
at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.BuildPrivatePki(PkiOptions pkiOptions, RevocationResponder& responder, CertificateAuthority& rootAuthority, CertificateAuthority& intermediateAuthority, X509Certificate2& endEntityCert, String testName, Boolean registerAuthorities, Boolean pkiOptionsInSubject) in /
/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 1344
at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.SimpleTest(PkiOptions pkiOptions, RunSimpleTest callback, String callerName, Boolean pkiOptionsInTestName) in //src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 1292
at System.Security.Cryptography.X509Certificates.Tests.RevocationTests.DynamicRevocationTests.RevokeEndEntity(PkiOptions pkiOptions) in /
/src/libraries/System.Security.Cryptography.X509Certificates/tests/RevocationTests/DynamicRevocationTests.cs:line 124
runtime Release windows.10.amd64.open.rt x64 2022-05-31 23:27:00.3330000 NoPlaintextExportAllowsEncryptedPkcs8 System.Security.Cryptography.CryptographicException : An internal consistency check failed. System.Security.Cryptography.Cng.Tests at System.Security.Cryptography.CngKey.Create(CngAlgorithm algorithm, String keyName, CngKeyCreationParameters creationParameters) in //src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/CngKey.Create.cs:line 57
at System.Security.Cryptography.CngAlgorithmCore.GetOrGenerateKey(Int32 keySize, CngAlgorithm algorithm) in /
/src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/CngAlgorithmCore.cs:line 76
at System.Security.Cryptography.DSACng.get_Key() in //src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/DSACng.Key.cs:line 24
at System.Security.Cryptography.Cng.Tests.DSACngPkcs8Tests.CreateKey(CngKey& cngKey) in /
/src/libraries/System.Security.Cryptography.Cng/tests/DSACngPkcs8Tests.cs:line 13
at System.Security.Cryptography.Cng.Tests.CngPkcs8Tests`1.NoPlaintextExportAllowsEncryptedPkcs8(PbeEncryptionAlgorithm algorithm) in //src/libraries/System.Security.Cryptography.Cng/tests/CngPkcs8Tests.cs:line 38
at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr) in /
/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodInvoker.cs:line 69

@bartonjs
Copy link
Member

We added a retry a long while ago, I don't believe it helped. The underlying OS team's response, essentially, was that if we reimaged our test machines more often we'd probably never see the problems (and that they've never seen these sorts of errors on normal user machines).

@MattGal
Copy link
Member

MattGal commented Jun 15, 2022

We added a retry a long while ago, I don't believe it helped. The underlying OS team's response, essentially, was that if we reimaged our test machines more often we'd probably never see the problems (and that they've never seen these sorts of errors on normal user machines).

For every queue listed in the above query we stay up to date with the most recent Azure gallery images for Server 2K12 R2 and 2016, so the images aren't too old, and most of these machines only live a few hours (pick any work item where this failed and I can demonstrate this via Kusto query). Is it possible that the problem isn't "we don't reimage often enough"?

@karelz karelz removed the blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' label Jun 24, 2022
@karelz
Copy link
Member

karelz commented Jun 24, 2022

No hit in last 10 days (Runfo says 3 hits per month), removing blocking-clean-ci label.

@stephentoub
Copy link
Member

Just failed here on Windows 10:
https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-72657-merge-06ecc243922e4a95a0/System.Security.Cryptography.Cng.Tests/1/console.28d88923.log?helixlogtype=result

    System.Security.Cryptography.Rsa.Tests.SignVerify_Array.InvalidKeySize_DoesNotInvalidateKey [FAIL]
      System.Security.Cryptography.CryptographicException : Unknown error (0xc0000001)
      Stack Trace:
        /_/src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/CngCommon.SignVerify.cs(42,0): at System.Security.Cryptography.CngCommon.SignHash(SafeNCryptKeyHandle keyHandle, ReadOnlySpan`1 hash, AsymmetricPaddingMode paddingMode, Void* pPaddingInfo, Int32 estimatedSize)
        /_/src/libraries/Common/src/System/Security/Cryptography/RSACng.SignVerify.cs(67,0): at System.Security.Cryptography.RSACng.SignHash(Byte[] hash, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding)
        /_/src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/RSA.cs(189,0): at System.Security.Cryptography.RSA.SignData(Byte[] data, Int32 offset, Int32 count, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding)
        /_/src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/RSA.cs(170,0): at System.Security.Cryptography.RSA.SignData(Byte[] data, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding)
        /_/src/libraries/Common/tests/System/Security/Cryptography/AlgorithmImplementations/RSA/SignVerify.cs(17,0): at System.Security.Cryptography.Rsa.Tests.SignVerify_Array.SignData(RSA rsa, Byte[] data, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding)
        /_/src/libraries/Common/tests/System/Security/Cryptography/AlgorithmImplementations/RSA/SignVerify.cs(119,0): at System.Security.Cryptography.Rsa.Tests.SignVerify.InvalidKeySize_DoesNotInvalidateKey()
           at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
        /_/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodInvoker.cs(64,0): at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)

@bartonjs
Copy link
Member

There are two things we can do here that might help

  1. Switch from NCrypt.dll to BCrypt.dll for weakly typed ephemeral keys (e.g. RSA.Create()) (a thing the OS team has suggested we do in the past, for unrelated reasons). This will cut down on NCrypt <-> LSASS communication and might help here, at the cost of more code and more tests (but better perf and a big shrug on security).

  2. Make the test libraries use pooled keys, to cut down on object registration and deregistration in the LSA bridge.

The first isn't for 7; the second we can try once higher priority issues run out (which may or may not still be during 7).

@karelz
Copy link
Member

karelz commented Sep 14, 2022

New hit in release/7.0 branch - 9/12 Rolling run 13309:

Platform net7.0-windows-Release-x64-NativeAOT_Release-(Windows.Nano.1809.Amd64.Open)windows.10.amd64.serverrs5.open:

[FAIL] System.Security.Cryptography.Cng.Tests.RSACngPkcs8Tests.NoPlaintextExportAllowsEncryptedPkcs8(algorithm: TripleDes3KeyPkcs12)
System.Security.Cryptography.CryptographicException : Unknown error (0xc0000001)
   at System.Security.Cryptography.CngCommon.SignHash(SafeNCryptKeyHandle, ReadOnlySpan`1, Interop.NCrypt.AsymmetricPaddingMode, Void*, Int32) + 0x21c
   at System.Security.Cryptography.RSACng.SignHash(Byte[], HashAlgorithmName, RSASignaturePadding) + 0x108
   at System.Security.Cryptography.RSA.SignData(Byte[], Int32, Int32, HashAlgorithmName, RSASignaturePadding) + 0x7e
   at System.Security.Cryptography.Cng.Tests.RSACngPkcs8Tests.VerifyMatch(RSACng exported, RSACng imported) + 0x67
   at System.Security.Cryptography.Cng.Tests.CngPkcs8Tests`1.NoPlaintextExportAllowsEncryptedPkcs8(PbeEncryptionAlgorithm algorithm) + 0x27f
   at System.Security.Cryptography.Cng.Tests!<BaseAddress>+0x6fcd7c
   at System.Reflection.DynamicInvokeInfo.Invoke(Object, IntPtr, Object[], BinderBundle, Boolean) + 0x150

@karelz
Copy link
Member

karelz commented Sep 15, 2022

Another hit on different test in release/7.0 branch - 9/13 Rolling run 16085:

Platform: net7.0-windows-Release-x64-CoreCLR_release-(Windows.Nano.1809.Amd64.Open)windows.10.amd64.serverrs5.open

Console log:

    System.Security.Cryptography.Rsa.Tests.EncryptDecrypt_Array.NonPowerOfTwoKeySizeOaepRoundtrip(oaepPaddingMode: OaepSHA1) [FAIL]
      System.Security.Cryptography.CryptographicException : Unknown error (0xc0000001)
      Stack Trace:
        /_/src/libraries/Common/src/System/Security/Cryptography/RSACng.EncryptDecrypt.cs(241,0): at System.Security.Cryptography.RSACng.EncryptOrDecrypt(SafeNCryptKeyHandle key, ReadOnlySpan`1 input, AsymmetricPaddingMode paddingMode, Void* paddingInfo, Boolean encrypt)
        /_/src/libraries/Common/src/System/Security/Cryptography/RSACng.EncryptDecrypt.cs(105,0): at System.Security.Cryptography.RSACng.EncryptOrDecrypt(Byte[] data, RSAEncryptionPadding padding, Boolean encrypt)
        /_/src/libraries/Common/src/System/Security/Cryptography/RSACng.EncryptDecrypt.cs(25,0): at System.Security.Cryptography.RSACng.Decrypt(Byte[] data, RSAEncryptionPadding padding)
        /_/src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/RSAWrapper.cs(53,0): at System.Security.Cryptography.RSAWrapper.Decrypt(Byte[] data, RSAEncryptionPadding padding)
        /_/src/libraries/Common/tests/System/Security/Cryptography/AlgorithmImplementations/RSA/EncryptDecrypt.cs(16,0): at System.Security.Cryptography.Rsa.Tests.EncryptDecrypt_Array.Decrypt(RSA rsa, Byte[] data, RSAEncryptionPadding padding)
        /_/src/libraries/Common/tests/System/Security/Cryptography/AlgorithmImplementations/RSA/EncryptDecrypt.cs(687,0): at System.Security.Cryptography.Rsa.Tests.EncryptDecrypt.NonPowerOfTwoKeySizeOaepRoundtrip(RSAEncryptionPadding oaepPaddingMode)
           at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
        /_/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodInvoker.cs(64,0): at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)

@karelz
Copy link
Member

karelz commented Sep 15, 2022

@bartonjs not sure if we were just lucky to get the failure 2x recently, or if it is more common now from some reason.

@karelz
Copy link
Member

karelz commented Sep 23, 2022

Another hit in release/7.0-rc2 branch - 9/22 Rolling run 26747:

Platform: net7.0-windows-Release-x64-CoreCLR_release-(Windows.Nano.1809.Amd64.Open)windows.10.amd64.serverrs5.open

Console log:

    System.Security.Cryptography.X509Certificates.Tests.CertificateCreation.CertificateRequestLoadTests.VerifySignature_RSA_PSS(hashAlgorithm: "SHA512") [FAIL]
      System.Security.Cryptography.CryptographicException : Unknown error (0xc0000001)
      Stack Trace:
        /_/src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/CngCommon.SignVerify.cs(42,0): at System.Security.Cryptography.CngCommon.SignHash(SafeNCryptKeyHandle keyHandle, ReadOnlySpan`1 hash, AsymmetricPaddingMode paddingMode, Void* pPaddingInfo, Int32 estimatedSize)
        /_/src/libraries/Common/src/System/Security/Cryptography/RSACng.SignVerify.cs(71,0): at System.Security.Cryptography.RSACng.SignHash(Byte[] hash, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding)
        /_/src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/RSA.cs(363,0): at System.Security.Cryptography.RSA.SignData(Byte[] data, Int32 offset, Int32 count, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding)
        /_/src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/RSAWrapper.cs(107,0): at System.Security.Cryptography.RSAWrapper.SignData(Byte[] data, Int32 offset, Int32 count, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding)
        /_/src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/RSA.cs(344,0): at System.Security.Cryptography.RSA.SignData(Byte[] data, HashAlgorithmName hashAlgorithm, RSASignaturePadding padding)
        /_/src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/X509Certificates/RSAPssX509SignatureGenerator.cs(106,0): at System.Security.Cryptography.X509Certificates.RSAPssX509SignatureGenerator.SignData(Byte[] data, HashAlgorithmName hashAlgorithm)
        /_/src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/X509Certificates/Pkcs10CertificationRequestInfo.cs(77,0): at System.Security.Cryptography.X509Certificates.Pkcs10CertificationRequestInfo.ToPkcs10Request(X509SignatureGenerator signatureGenerator, HashAlgorithmName hashAlgorithm)
        /_/src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/X509Certificates/CertificateRequest.cs(377,0): at System.Security.Cryptography.X509Certificates.CertificateRequest.CreateSigningRequest(X509SignatureGenerator signatureGenerator)
        /_/src/libraries/System.Security.Cryptography/src/System/Security/Cryptography/X509Certificates/CertificateRequest.cs(283,0): at System.Security.Cryptography.X509Certificates.CertificateRequest.CreateSigningRequest()
        /_/src/libraries/System.Security.Cryptography.X509Certificates/tests/CertificateCreation/CertificateRequestLoadTests.cs(392,0): at System.Security.Cryptography.X509Certificates.Tests.CertificateCreation.CertificateRequestLoadTests.VerifySignature_RSA_PSS(String hashAlgorithm)
           at InvokeStub_CertificateRequestLoadTests.VerifySignature_RSA_PSS(Object, Object, IntPtr*)
           at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)

@steveisok
Copy link
Member

@jeffhandley this seems to be happening quite a bit lately seemingly on the mono configurations. Can you have someone take a look?

@hoyosjs hoyosjs added the Known Build Error Use this to report build issues in the .NET Helix tab label May 9, 2023
@runfoapp runfoapp bot removed this from the Future milestone May 9, 2023
@runfoapp runfoapp bot closed this as completed May 9, 2023
@ghost ghost locked as resolved and limited conversation to collaborators Jun 9, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-System.Security Known Build Error Use this to report build issues in the .NET Helix tab tracking-external-issue The issue is caused by external problem (e.g. OS) - nothing we can do to fix it directly
Projects
None yet
Development

No branches or pull requests