Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

X509Certificate2/OpenSSL: Unable to open PKCS#12 files with no password and no MAC #18254

Closed
qmfrederik opened this issue Aug 22, 2016 · 17 comments · Fixed by dotnet/corefx#42226
Labels
area-System.Security bug help wanted [up-for-grabs] Good issue for external contributors
Milestone

Comments

@qmfrederik
Copy link
Contributor

If you have a PKCS#12 file which is not protected with a password, and which does not have a MAC entry, opening the file will work on Windows but fails on Linux and Mac (which use OpenSSL).

The following program reproduces the behavior:

using System;
using System.IO;
using System.Security.Cryptography.X509Certificates;

namespace ConsoleApplication
{
    public class Program
    {
        public static void Main(string[] args)
        {
            byte[] rawData = Convert.FromBase64String(
                "MIACAQMwgAYJKoZIhvcNAQcBoIAkgASCA+gwgDCABgkqhkiG9w0BBwGggCSABIID6DCCBRUwggURBgsqhkiG9w0BDAoBAaCCBMAwggS8AgEAMA0GCSqGSIb3DQEBAQUABIIEpjCCBKICAQACggEBAJ9Iw1KuIXuQBnQA3GvlPu2yvXZU8BbM6yBwuypsUdOk4isb8S2+CI/p3Ez8yWMb+HZLQ1QBXDvk2VzZUIJH+xY1oWqu7Hvo9iADkltKbfZ1yjfG5Dy7FatNnVHlGoq7HTK9WlxEVZutrHdpBKHNdHWoXr0xqxS4YtcXnnBZMLlDpVhxUJ6L/X0WRHY52QcLp6ZYfAObU1+l+Ihh7OVP1r8c0HmIH+qL3FaQplhrNjFr6qrCarA42DZjcgShuNOfuvhEJejmgOL7QS0j8h2sT/Fa5oDNwRNJV5vrni8GPbjd4Sbch+90+Oz39tyO8ecvIsKKak67oUbR80EyXNjb0eMCAwEAAQKCAQA8C/AmQSK6NAdav+BYhGl+rj0iWM7RqZqR9i14xrDqOmRQoA4Bknwj1KOKGlnJFQhLf//3sTOWGKWgjQP+uSf8rWcWkq7v31i5pN8NrzdZC/qZoE72XgjDNVUzRE0HM5bERAHGerRTJdu4gEyQuqVGnZxpcknuW7xXHb5K2DS4AiIDfkU9iqkUxa/hEG4HpueBgXv14oRa5z4hPzTbiIcej18f1n3IJA1SsATZ3RjU5B52ni/lYALufdnAJeBycLDU9m7UbqTfXSpb2uWM9yJKCOALHQZzE50/pW3DRQpdfbkoaMucoHqUSsLSbTLkYduHoxb9Bi7A6tgRTQFtOvfhAoGBANC639cEiE4xScrD3s9A/5/8pRoN8ojLEVDk44igGMHz+usHfxxp8YQvmYd5US1P6qPAX1scHKqHDbokd7aDh/5L7TKlwxo+iv7aOPQWJbpEmtsXASsMYfKneifUUxA47GtEYLHQ9Yy3kAratTaHH9K3a6x3wbsk815p77hlRygbAoGB" +
                "AMNbRSmmx/53b1YV6LxOuxzRSyCP5qYswlG2i7H6A+MLmUnxYurFvSptk+vEngmbOJW9rd49bHrK6PUtKBNYLxRWjRUtkgK2Z9PQguc8GD0W7SjHZafpFPETVdKxIVP6a18RB5jNvIuOPA8Z09/Kh80dw9dEPXs1EFQubxEFQ6nZAoGAOZsJgcb/c00JB4vNJzfSFK5eRnWI9RXOHpw864z7qDOUkV7NRuM6Q3f7kDb8H1xJ7o1+A6AbjTieojvESju8wYLk4LB8yvZt1+4T/9FI8kJS1ppfuSi+s4BjJzDjB7weC3CgmxKHYiGbAFPh5T2fm8EBV2Tps6N8AxeLkEFrBIID6ET3AoGAWmveYV7+5rtlXxUY+j/+v2HoBIIBMUIUGRAFW5PyyEoCjNYEQllFTyGXkO0YdwUDppqPq+szNkzNZW6YiKci1Y/Om0vwm7CXvSNgRkJ2GoDpAdcUy4S6dkT3z2eeKXUx41k5aYVBHqENaR23IfljXPwShDTeeA0lWseyUfKE44efRihRAoGAYn+PAVLDWmSKMm6rJDbJne12NYviqOa6n3ZJg0N9I8o0C/xFX1o/IzJF0EPdyVlCj/laIYoUX3O1P5I8YeubcR/JvY/LnCMutgsWlkdbz5c9gfbFVaEXXQPFyKuSaMea6pn+kv9EYtVfgiQ4rZ4aZwe0CWkzfDvIE6AGYgMICMIxPjAXBgkqhkiG9w0BCRQxCh4IAGMAZQByAHQwIwYJKoZIhvcNAQkVMRYEFO3z0SLPYjzwz8nNImJh6EFag+YwAAAAAAAAMIAGCSqGSIb3DQEHAaCAJIAEggPoMIIGETCCBg0GCyqGSIb3DQEMCgEDoIIFvDCCBbgGCiqGSIb3DQEJFgGgggWoBIIFpDCCBaAwggSIoAMCAQICCBulrjADvm1lMA0GCSqGSIb3DQEBBQUAMIGWMQswCQYDVQQGEwJVUzETMBEGA1UECgwKQXBwbGUgSW5jLjEsMCoGA1UECwwjQXBwbGUgV29ybGR3a" +
                "WRlIERldmVsb3BlciBSZWxhdGlvbnMxRDBCBgNVBAMMO0FwcGxlIFdvcmxkd2lkZSBEZXZlbG9wZXIgUmVsYXRpb25zIENlcnRpZmljYXRpb24gQXV0aG9yaXR5MB4XDTE1MDExMjEyMTIxMFoXDTE2MDExMjEyMTIxMFowgZMxGjAYBgoJkiaJk/IsZAEBDApSSlhWVEU4NjUzMTgwNgYDVQQDDC9pUGhvbmUgRGV2ZWxvcGVyOiBGcmVkZXJpayBDYXJsaWVyICg4VDlVS1VCR1k5KTETMBEGA1UECwwKVENESzVFTEFINzEZMBcGA1UECgwQRnJlZGVyaWsgQ2FybGllcjELMAkGA1UEBhMCVVMwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCfSMNSriF7kAZ0ANxr5T7tsr12VPAWzOsgcLsqbFHTpOIrG/EtvgiP6dxM/MljG/h2S0NUAVw75Nlc2VCCR/sWNaFqrux76PYgA5JbSm32dco3xuQ8uxWrTZ1R5RqKux0yvVpcRFWbrax3aQShzXR1qF69MasUuGLXF55wWTC5Q6VYcVCei/19FkR2OdkHC6emWHwDm1NfpfiIYezlT9a/HNB5iB/qi9xWkKZYazYxa+qqwmqwONg2Y3IEggOgBKG405+6+EQl6OaA4vtBLSPyHaxP8VrmgM3BE0lXm+ueLwY9uN3hJtyH73T47Pf23I7x5y8iwopqTruhRtHzQTJc2NvR4wIDAQABo4IB8TCCAe0wHQYDVR0OBBYEFO3z0SLPYjzwz8nNImJh6EFag+YwMAwGA1UdEwEB/wQCMAAwHwYDVR0jBBgwFoAUiCcXCam2GGCL7Ou69kdZxVJUo7cwggEPBgNVHSAEggEGMIIBAjCB/wYJKoZIhvdjZAUBMIHxMIHDBggrBgEFBQcCAjCBtgyBs1JlbGlhbmNlIG9uIHRoaXMgY2VydGlmaWNhdGUgYnkgYW55IHBhcnR5IGFzc3VtZXMgYWNjZXB0YW" +
                "5jZSBvZiB0aGUgdGhlbiBhcHBsaWNhYmxlIHN0YW5kYXJkIHRlcm1zIGFuZCBjb25kaXRpb25zIG9mIHVzZSwgY2VydGlmaWNhdGUgcG9saWN5IGFuZCAEggItY2VydGlmaWNhdGlvbiBwcmFjdGljZSBzdGF0ZW1lbnRzLjApBggrBgEFBQcCARYdaHR0cDovL3d3dy5hcHBsZS5jb20vYXBwbGVjYS8wTQYDVR0fBEYwRDBCoECgPoY8aHR0cDovL2RldmVsb3Blci5hcHBsZS5jb20vY2VydGlmaWNhdGlvbmF1dGhvcml0eS93d2RyY2EuY3JsMA4GA1UdDwEB/wQEAwIHgDAWBgNVHSUBAf8EDDAKBggrBgEFBQcDAzATBgoqhkiG92NkBgECAQH/BAIFADANBgkqhkiG9w0BAQUFAAOCAQEAaIuMydHST4l1fMsaXr51Ejqa00fB3PNY9Rw6oG1cWkSS6RgxAz7AJ7dJCO0tZdqPCX6VKnTHhgMlQrI8tIfU2WcG+sV3tvnAysqRtDPqhHWiR4judlp0ETzoLHJFDbbnEph3NpOYVfUwyCthYL/xv3cF9ohcHvfS02O4MsQl1QKhUzLXEOnjUGgPIb7xmGIP54+TNePhEdOi05ijKr1AO4BgCKjeu7tHYvIuyY9HGOfGyuXsoDrP6F+Jj3VRvYCCZuKIvDnocGHsi9AgaxuOSdp5GQOD6OQvXaOPeJNLf8+1Z1S4h/9lS6VubqH+tp9nvgUuq+zbmHv5iqRadMmz4TE+MBcGCSqGSIb3DQEJFDEKHggAYwBlAHIAdDAjBgkqhkiG9w0BCRUxFgQU7fPRIs9iPPDPyc0iYmHoQVqD5jAAAAAAAAAAAAAAAAAAAAAA");

            File.WriteAllBytes(@"rawData.bin", rawData);

            var certificate = new X509Certificate2(
                rawData: rawData,
                password: null,
                keyStorageFlags: X509KeyStorageFlags.Exportable);

            Console.WriteLine(certificate.SerialNumber);
        }
    }
}

The output on Windows:

1BA5AE3003BE6D65

and on Linux:

Unhandled Exception: Interop+Crypto+OpenSslCryptographicException: error:23076071:PKCS12 routines:PKCS12_parse:mac verify failure
   at Internal.Cryptography.Pal.OpenSslPkcs12Reader.Decrypt(String password)
   at Internal.Cryptography.Pal.PkcsFormatReader.TryReadPkcs12(OpenSslPkcs12Reader pfx, String password, Boolean single, ICertificatePal& readPal, List`1& readCerts)
   at Internal.Cryptography.Pal.PkcsFormatReader.TryReadPkcs12(Byte[] rawData, String password, Boolean single, ICertificatePal& readPal, List`1& readCerts)
   at Internal.Cryptography.Pal.CertificatePal.FromBlob(Byte[] rawData, String password, X509KeyStorageFlags keyStorageFlags)
   at System.Security.Cryptography.X509Certificates.X509Certificate..ctor(Byte[] rawData, String password, X509KeyStorageFlags keyStorageFlags)
   at System.Security.Cryptography.X509Certificates.X509Certificate2..ctor(Byte[] rawData, String password, X509KeyStorageFlags keyStorageFlags)
   at ConsoleApplication.Program.Main(String[] args)

You can get the same error message by running OpenSSL directly on the rawData.bin file (don't provide a password):

openssl.exe pkcs12 -info -in rawData.bin
WARNING: can't open config file: /usr/local/ssl/openssl.cnf
Enter Import Password:
Mac verify error: invalid password?
20876:error:2307E06C:PKCS12 routines:PKCS12_verify_mac:mac absent:.\crypto\pkcs12\p12_mutl.c:119:
20876:error:2307E06C:PKCS12 routines:PKCS12_verify_mac:mac absent:.\crypto\pkcs12\p12_mutl.c:119:

However, OpenSSL will correctly inspect the file if you pass the -nomacver option:

openssl.exe pkcs12 -info -in rawData.bin -nomacver
WARNING: can't open config file: /usr/local/ssl/openssl.cnf
Enter Import Password:
PKCS7 Data
Key bag
Bag Attributes
    friendlyName: cert
    localKeyID: ED F3 D1 22 CF 62 3C F0 CF C9 CD 22 62 61 E8 41 5A 83 E6 30
Key Attributes: <No Attributes>
Enter PEM pass phrase:
PKCS7 Data
Certificate bag
Bag Attributes
    friendlyName: cert
(...)
-----END CERTIFICATE-----

The issue seems to be caused by the absence of a MAC entry in the file. Based on RFC7292, I believe that's valid, and BouncyCastle creates PKCS#12 files without MAC entries if no password is provided.

So net, because

  • This file was created by Bouncy Castle, which emits the MAC entry if it is not password protected
  • That seems to be compliant with the RFC
  • The file loads correctly on Windows
  • The file loads correctly using OpenSSL if you specify the -nomacver flag

I would expect .NET Core on Linux & Mac to also be able to read this file.

@bartonjs bartonjs self-assigned this Aug 22, 2016
@bartonjs
Copy link
Member

OpenSSL and Windows both use the null password as the empty password when creating PFX files. Does bouncy castle generate a usable file if you specify "" instead of null?

@qmfrederik
Copy link
Contributor Author

@bartonjs If I use string.Empty as a password when saving the file using Bouncy Castle, and specify string.Empty when opening the file using X509Certificate2 on Linux, everything works because Bouncy Castle creates a MAC entry.

So that's a workaround for me now.

I'd still argue the behavior where X509Certificate2 requires a MAC entry is incorrect (at least, different from X509Certificate2 on Windows), but it's good to have a workaround :)

@bartonjs bartonjs removed their assignment Sep 30, 2016
@vaindil
Copy link

vaindil commented Oct 18, 2016

Not sure if this is related or if I should open a new issue, but I'm having a similar problem. I'm using certs with a password. They work on Windows but fail on Linux. The specific exception message is this:

error:23076071:PKCS12 routines:PKCS12_parse:mac verify failure

I created the certificates using OpenSSL on Windows Bash, so I don't know if a MAC entry was included. I can't find any results when trying to search for this subject, all I can get are results about Mac OS. I don't know how to generate a cert that will work on Linux as well as Windows.

I used this command to create the cert:

openssl req -x509 -newkey rsa:4096 -keyout mykey.pem -out certificate.pem -days 3650

Then this to convert to PFX:

openssl pkcs12 -export -out mycert.pfx -inkey mykey.pem -in certificate.pem

@bartonjs
Copy link
Member

@vaindil Yeah, it seems like a separate issue. But, in order to identify if we'd be able to do anything about it you should send that PFX to a Linux machine with the openssl utility and try

openssl pkcs12 -in mycert.pfx -nokeys

If that can't read it then it's an OpenSSL library problem, and we won't be able to, either. If it can, then there's a fallback path we're missing.

If the command line utility test passes, it'd be great if you could give us a sample PFX that fails.

@vaindil
Copy link

vaindil commented Oct 18, 2016

@bartonjs Just tested, the openssl command does work correctly. I created a new test certificate, all three parts are attached here.

Key passphrase (testkey.pem): 123456
PFX password (badfinal.pfx): 789789

Generate the key and certificate:

openssl req -x509 -newkey rsa:4096 -keyout testkey.pem -out testcert.pem

Convert to PFX:

openssl pkcs12 -export -out badfinal.pfx -inkey testkey.pem -in testcert.pem

@karelz
Copy link
Member

karelz commented Nov 16, 2016

We need a test and likely add one more fallback call.

@sphiecoh
Copy link

Any update on this issue , hitting the same error with pfx that is password protected on Ubuntu 16.04.

@bartonjs
Copy link
Member

@sphiecoh If your PFX has a password that'd be a different problem, since this is specifically about a PFX with a null password (which is different than "the empty password"). Please open a new issue if you're having a different problem.

@sphiecoh
Copy link

nvm I found my issue , password was cleared out of env.

@Condorello
Copy link

the mac verify fail also with my Comodo SSL cert, which works fine with other a service. There is something to pass online web server because with "bought" SSL certificate there's not possibility to change the cert-key generation string..

@jeremyVignelles
Copy link

Hi, any update on this?

@bartonjs
Copy link
Member

bartonjs commented Dec 7, 2018

@jeremyVignelles Nope, sorry. I'm hoping to get the time to rewrite the PFX loader entirely to work around platform-specific quirks like this. The new Pkcs12Info type in .NET Core 3.0 should be capable of exploring (and extracting data from) a PFX in this state, so you might be able to write a loader for your particular needs.

https://apisof.net/catalog/System.Security.Cryptography.Pkcs.Pkcs12Info

@ghost
Copy link

ghost commented Dec 14, 2018

I'm seeing the same issue when creating a PFX file from a key and cert with an empty password (enter twice when prompted) or by passing -password pass: when using openssl. If I pass -nomac to openssl I get a format error when trying to load the PFX.

@inpicksys
Copy link

This is annoying, & blocks all my work. Do you have any workaround for this? While .NET Core 3.0 not released yet?

@jeremyVignelles
Copy link

jeremyVignelles commented Dec 15, 2018

The workaround would be to set a password I guess...

EDIT : you can also try with BouncyCastle (the .net core-compatible version)

@ghost
Copy link

ghost commented Dec 17, 2018

@inpicksys What I did is just used a password "password" and stored the credential in the source code with a comment explaining that it's a workaround for a bug and included a link to this thread. It's my intent to use no password protection at all for the certificate and I wanted to capture that intent in there as to not raise any alarms when someone eventually notices it.

@stephentoub
Copy link
Member

@bartonjs, there's still mention of this issue in the code:

// [ActiveIssue("https://github.com/dotnet/corefx/issues/11046", TestPlatforms.AnyUnix)]

// [ActiveIssue("https://github.com/dotnet/corefx/issues/11046", TestPlatforms.AnyUnix)]

@msftgits msftgits transferred this issue from dotnet/corefx Jan 31, 2020
@msftgits msftgits added this to the 5.0 milestone Jan 31, 2020
@ghost ghost locked as resolved and limited conversation to collaborators Dec 29, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-System.Security bug help wanted [up-for-grabs] Good issue for external contributors
Projects
None yet
Development

Successfully merging a pull request may close this issue.

10 participants