Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Generate a self-signed placeholder certificate for use when real certificate not available #32

Closed
jsha opened this issue Aug 19, 2017 · 23 comments
Closed

Generate a self-signed placeholder certificate for use when real certificate not available #32

jsha opened this issue Aug 19, 2017 · 23 comments

Comments

@jsha
Copy link
Contributor

jsha commented Aug 19, 2017

Steps to reproduce:

Run Apache with a failing MD config (e.g. see #31), and try to connect via SSL (e.g. curl -k https://172.17.0.2:443).

Expected result: Bogus certificate

Actual result:

curl: (35) gnutls_handshake() failed: Error in the pull function.

In the Apache error logs, I see:

[Sat Aug 19 15:49:00.906790 2017] [ssl:error] [pid 9] [client 172.17.0.1:58708] AH01962: Unable to create a new SSL connection from the SSL context
[Sat Aug 19 15:49:00.907038 2017] [ssl:error] [pid 9] SSL Library Error: error:02001002:system library:fopen:No such file or directory (fopen('/apache/md/domains/powerdns.crud.net/pkey.pem','r'))
[Sat Aug 19 15:49:00.907101 2017] [ssl:error] [pid 9] SSL Library Error: error:2006D080:BIO routines:BIO_new_file:no such file
[Sat Aug 19 15:49:00.907110 2017] [ssl:error] [pid 9] SSL Library Error: error:02001002:system library:fopen:No such file or directory (fopen('/apache/md/domains/powerdns.crud.net/pkey.pem','r'))
[Sat Aug 19 15:49:00.907134 2017] [ssl:error] [pid 9] SSL Library Error: error:2006D080:BIO routines:BIO_new_file:no such file
[Sat Aug 19 15:49:00.907143 2017] [ssl:error] [pid 9] SSL Library Error: error:02001002:system library:fopen:No such file or directory (fopen('/apache/md/domains/powerdns.crud.net/pkey.pem','r'))
[Sat Aug 19 15:49:00.907151 2017] [ssl:error] [pid 9] SSL Library Error: error:2006D080:BIO routines:BIO_new_file:no such file
[Sat Aug 19 15:49:00.907204 2017] [ssl:error] [pid 9] SSL Library Error: error:140BA0C3:SSL routines:SSL_new:null ssl ctx

Ideally mod_md should generate a placeholder self-signed certificate for use when there is no real cert available, so handshakes can be completed. This will also avoid spamming the error logs in such a situation.
@icing
Copy link
Owner

icing commented Aug 31, 2017

I like this proposal. However I need to touch the mod_md/mod_ssl interface for this one. So, I prefer to delay any change here until I touch that part again.

@icing
Copy link
Owner

icing commented Sep 1, 2017

@oerdnj I would like to make changes to the mod_md <-> mod_ssl interface next week. Do you have some spare cycles there to push an updated PPA?

@oerdnj
Copy link

oerdnj commented Sep 1, 2017

Sure, when exactly?

@icing
Copy link
Owner

icing commented Sep 1, 2017 via email

@oerdnj
Copy link

oerdnj commented Sep 4, 2017

JFTR I finally got to creating libapache2-mod-md package available from ppa:ondrej/apache2.

icing pushed a commit that referenced this issue Sep 4, 2017
* Improved interface to mod_ssl for fallback handling. Backward comp…
46c3cd8 
…atible to previous mod_ssl

   patch, but fallbacks will not work.
 * Provide a temporary, self-signed certificate with a speaking command and domain name if we
   have no other cert for a Managed Domain, yet. Refs github issue #32
 * Continue to provide expired or not-completely matching, existing certificate for a Managed
   Domain until the renewal was successful. This is helpful when one adds a DNS name to
   a MD, so the previous domains can be served while a new cert is requested.
@icing
Copy link
Owner

icing commented Sep 4, 2017

Available in v0.9.0 together with new mod_ssl patch.

@oerdnj : the interface between mod_ssl and mod_md was kept backward compatible, so the pressure on updates is not high. It's just this fallback certificate mechanism that will not work with old mod_ssl versions.

@oerdnj
Copy link

oerdnj commented Sep 5, 2017

Updated the mod_ssl patch and packaged 0.9.0 in the ppa:ondrej/apache2 repository.

@icing
Copy link
Owner

icing commented Sep 5, 2017

Thanks! Wonderful!

@icing
Copy link
Owner

icing commented Sep 5, 2017

@jsha whenever you find time to test this, a short ping here would be nice.

@jsha
Copy link
Contributor Author

jsha commented Sep 6, 2017

I checked out v0.9.2 and did make install, then restarted Apache. At first I got "connection refused" on port 443. Then I enabled mod_ssl and mod_socache_shmcb. I now get "an unexpected TLS packet was received," and no entries in the error log on connection.

# cd /etc/apache2/mods-enabled/
# ln -s ../mods-available/socache_shmcb.load .
# ln -s ../mods-available/ssl.* .
# curl -vvv -k https://localhost
* Rebuilt URL to: https://localhost/
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 443 (#0)
* found 173 certificates in /etc/ssl/certs/ca-certificates.crt
* found 694 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* gnutls_handshake() failed: An unexpected TLS packet was received.
* Closing connection 0
curl: (35) gnutls_handshake() failed: An unexpected TLS packet was received.

000-default.conf.txt

@icing
Copy link
Owner

icing commented Sep 7, 2017

Thanks for including the conf. There is no directive that tells mod_ssl that it should handle *:443 (I know, this is kind of what one would expect it to do by itself...but...)

How does the following behave?

<VirtualHost *:443>
	ServerName powerdns.crud.net

        SSLEngine on
</VirtualHost>

@jsha
Copy link
Contributor Author

jsha commented Sep 7, 2017

Ah, of course! If fixed that and did service apache2 restart, and now I get a different error from curl, "Error in the pull function." From Chrome I get ERR_CONNECTION_RESET.

$ curl -vvv https://powerdns.crud.net
* Rebuilt URL to: https://powerdns.crud.net/
*   Trying 45.55.14.235...
* Connected to powerdns.crud.net (45.55.14.235) port 443 (#0)
* found 173 certificates in /etc/ssl/certs/ca-certificates.crt
* found 704 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* gnutls_handshake() failed: Error in the pull function.
* Closing connection 0
curl: (35) gnutls_handshake() failed: Error in the pull function.

000-default.conf.txt

@icing
Copy link
Owner

icing commented Sep 8, 2017

Do you have any other SSL* directives set? Try something more complete, such as:

    SSLProtocol             all -SSLv3 -TLSv1 -TLSv1.1
    SSLCipherSuite          ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256
    SSLHonorCipherOrder     on
    SSLCompression          off
    SSLSessionTickets       off
    SSLPassPhraseDialog  builtin
    SSLSessionCache        "shmcb:logs/ssl_scache(512000)"
    SSLSessionCacheTimeout  300

What is the OpenSSL version your server is linked with? In the error_log you see something like

[DateTime] [mpm_event:notice] [pid NNN:tid mmm] AH00489: Apache/2.4.28-dev (Unix) OpenSSL/1.1.0e configured -- resuming normal operations

@icing
Copy link
Owner

icing commented Sep 12, 2017

Ok, in v0.9.4 @michael-koeller added some nice test cases. We needed fixes in mod_md and a change in mod_ssl to get it working (v5 of the patch). But now the fallback certificate is in place on a new Managed Domain.

@jsha
Copy link
Contributor Author

jsha commented Sep 12, 2017

Sorry I forgot to reply to this. This is what I have as part of my default apache config (I symlinked in ssl.conf into mods-enabled from mods-available).

# grep -v '^\s*#\|^$' /etc/apache2/mods-enabled/ssl.conf
<IfModule mod_ssl.c>
        SSLRandomSeed startup builtin
        SSLRandomSeed startup file:/dev/urandom 512
        SSLRandomSeed connect builtin
        SSLRandomSeed connect file:/dev/urandom 512
        AddType application/x-x509-ca-cert .crt
        AddType application/x-pkcs7-crl .crl
        SSLPassPhraseDialog  exec:/usr/share/apache2/ask-for-passphrase
        SSLSessionCache         shmcb:${APACHE_RUN_DIR}/ssl_scache(512000)
        SSLSessionCacheTimeout  300
        SSLCipherSuite HIGH:!aNULL
        SSLProtocol all -SSLv3
</IfModule>

Here's another log: md.error.log.txt

I downloaded and installed v0.9.4 but get the same error. Note: I did an apt update && apt -y upgrade to get any available updates from @oerdnj's PPA, but it seems there was no new version available, so I may be missing the change in mod_ssl you mentioned.

@icing
Copy link
Owner

icing commented Sep 13, 2017

@oerdnj ping! It would be nice if you could push the lastest mod_ssl patch and the new v0.9.5 - if you find the time! Thanks!

@oerdnj
Copy link

oerdnj commented Sep 18, 2017 

Uploading libapache2-mod-md_0.9.6-1+ubuntu17.10.1+deb.sury.org+1.dsc
Uploading libapache2-mod-md_0.9.6.orig.tar.gz
Uploading libapache2-mod-md_0.9.6-1+ubuntu17.10.1+deb.sury.org+1.debian.tar.xz
Uploading libapache2-mod-md_0.9.6-1+ubuntu17.10.1+deb.sury.org+1_source.buildinfo
Uploading libapache2-mod-md_0.9.6-1+ubuntu17.10.1+deb.sury.org+1_source.changes

Uploading apache2_2.4.27-5.1+ubuntu17.10.1+deb.sury.org+1.dsc
Uploading apache2_2.4.27.orig.tar.bz2
Uploading apache2_2.4.27-5.1+ubuntu17.10.1+deb.sury.org+1.debian.tar.xz
Uploading apache2_2.4.27-5.1+ubuntu17.10.1+deb.sury.org+1_source.buildinfo
Uploading apache2_2.4.27-5.1+ubuntu17.10.1+deb.sury.org+1_source.changes

@jsha
Copy link
Contributor Author

jsha commented Sep 18, 2017

Alright, upgraded to latest Apache2 now it works great, thanks! One quibble: For some reason the placeholder certificate causes curl to say "Error in the certificate," while an example self-signed certificate from badssl.com produces a more useful error. I'm guessing there's something curl doesn't like, possible the lack of a hostname. It's a super minor thing, but it would be nice to figure out. :-)

$ curl https://powerdns.crud.net
curl: (35) gnutls_handshake() failed: Error in the certificate.

$ curl https://self-signed.badssl.com/
curl: (60) server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none
More details here: http://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a "bundle"
 of Certificate Authority (CA) public keys (CA certs). If the default
 bundle file isn't adequate, you can specify an alternate file
 using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
 the bundle, the certificate verification probably failed due to a
 problem with the certificate (it might be expired, or the name might
 not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
 the -k (or --insecure) option.

@michael-koeller
Copy link
Contributor

Using version 0.9.6:
I just tried to reproduce above error message, but on my local environment curl delivers exactly the same message as given above when targeting badssl.com.

$> curl --resolve www.testdomain.org:5001:127.0.0.1 https://www.testdomain.org:5001/
curl: (60) SSL certificate problem: self signed certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a "bundle"
 of Certificate Authority (CA) public keys (CA certs). If the default
 bundle file isn't adequate, you can specify an alternate file
 using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
 the bundle, the certificate verification probably failed due to a
 problem with the certificate (it might be expired, or the name might
 not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
 the -k (or --insecure) option.
HTTPS-proxy has similar options --proxy-cacert and --proxy-insecure.

If the error still persists: Does your local curl give more information about the different error causes if your turn on verbose mode?

@jsha
Copy link
Contributor Author

jsha commented Sep 20, 2017 •
edited
Loading

Still repros on mod_md v0.9.6.

$ curl -vvv https://powerdns.crud.net
* Rebuilt URL to: https://powerdns.crud.net/
*   Trying 45.55.14.235...
* Connected to powerdns.crud.net (45.55.14.235) port 443 (#0)
* found 173 certificates in /etc/ssl/certs/ca-certificates.crt
* found 704 certificates in /etc/ssl/certs
* ALPN, offering http/1.1
* gnutls_handshake() failed: Error in the certificate.
* Closing connection 0
curl: (35) gnutls_handshake() failed: Error in the certificate.
$ curl --version
curl 7.47.0 (x86_64-pc-linux-gnu) libcurl/7.47.0 GnuTLS/3.4.10 zlib/1.2.8 libidn/1.32 librtmp/2.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp smb smbs smtp smtps telnet tftp 
Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz TLS-SRP UnixSockets

@jsha
Copy link
Contributor Author

jsha commented Sep 20, 2017

Aha, gnutls-cli provides more info. Specifically, "extensions present in certificate with version 1." I think you need to set the certificate version to 3 (as in x509v3). Also, I'd recommend setting CA:FALSE instead of TRUE, just to avoid any potential confusion or trouble.

$ gnutls-cli  -p 443 powerdns.crud.net -d 2
Processed 173 CA certificate(s).
Resolving 'powerdns.crud.net'...
Connecting to '45.55.14.235:443'...
|<2>| error: extensions present in certificate with version 1
*** Fatal error: Error in the certificate.
*** Handshake has failed
GnuTLS error: Error in the certificate.

$ gnutls-cli --version
gnutls-cli 3.4.10
Copyright (C) 2000-2016 Free Software Foundation, and others, all rights reserved.
This is free software. It is licensed for use, modification and
redistribution under the terms of the GNU General Public License,
version 3 or later <http://gnu.org/licenses/gpl.html>


Please send bug reports to:  <bugs@gnutls.org>

icing pushed a commit that referenced this issue Sep 26, 2017
* When building against 2.4.x, one probably needs to configure witho…
3df3228 
…ut ```--enable-werror```,

   since there were some warnings fixed in Apache httpd trunk.
 * Removed obsolete function from interface to mod_ssl. Module now requires at least mod_ssl
   patch v4 in place.
 * Fallback certificates has version set and no longer claims to be a CA. (re issue #32)
 * ```MDRequireHttps``` now happens before any ```Redirect```.
@icing
Copy link
Owner

icing commented Sep 26, 2017

Thanks, this should be fixed in v0.9.7 now.

@jsha
Copy link
Contributor Author

jsha commented Sep 27, 2017

Confirmed, this now successfully fetches with curl for me. Thanks for the fix.

@jsha jsha closed this as completed Sep 27, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@icing @oerdnj @jsha @michael-koeller