error querying OCSP responder

[drwetter]

PR #1055 brought OCSP revocation checks (merged in crl_ocsp branch), see also #254. I added a host header to the query but still the feature doesn't seem to be production ready yet :-(

The first 1169 nodes (Alexa ranking) with TLS replies on port 443 TCP:

• 336 still return error querying OCSP responder (305 with Code=400, 31 without any code returned).
• 152 of 336 are showing a Google responder (pki.goog)
• 833 are fine.

This issue is just informational.

[dcooper16]

Hi Dirk,

Thanks for the improvements that you made to the code that I submitted. I thought it was in fairly good shape, but that it could use some improvements, which is why I submitted it for the crl_ocsp branch rather than the 2.9dev branch. I didn't test as many sites as you did, but it seemed to at least pass the initial test of not reporting any incorrect results. It seemed that the code would either report the correct result (revoked or not revoked) or it wouldn't print anything at all.

My testing produced mixed results. I tested the code against several sites using parallel mass testing, and then ran individual tests in debug mode for any server that didn't print a result during mass testing. It seemed that in every case, however, that the individual test resulted in a response of "not revoked."

To some degree we are going to have to accept less than perfection on these tests. There is a reason that browsers have basically given up on using CRLs or OCSP to check server certificates, and that even when they did try to check CRLs or OCSP, the browsers would be configured to accept a certificate as valid if no revocation information could be obtained. It is my understanding that the sources of revocation information just weren't reliable enough to treat the inability to obtain revocation information as a hard failure.

[drwetter]

I would at least to understand the 400s which is probably the HTTP server status. The logic that it's "our problem" seems to me too far away yet. Why would a certificate issue operate a server which is broken and put this URI into the certificates.

Thanks for the improvements that you made to the code that I submitted. I thought it was in fairly good shape, but that it could use some improvements, which is why I submitted it for the crl_ocsp branch rather than the 2.9dev branch.

In fact I did realize this (with a smile) and I appreciate you seem always to pay a lot of attention what you're doing 👍

My testing produced mixed results. I tested the code against several sites using parallel mass testing, and then ran individual tests in debug mode for any server that didn't print a result during mass testing. It seemed that in every case, however, that the individual test resulted in a response of "not revoked."

I need to pass to you tomorrow the servers I tested. Two servers I remembered off the top of my head: ca.gov, sourceforge.net .

As far as the currently technology is concerned, I fully agree. CRL is dead since a long time for browsers -- wasn't handy anyway. OCSP revocation checking seems still enabled in FF 60 though. For other clients, also non-browsers, I can't tell. In any case as long as this is in widespread use, it makes sense to support this feature.

[drwetter]

Fixed the major error, see commit message.

More later.

[drwetter]

I merged the crl_ocsp branch. Despite my commit message there are still a few errors (used the wrong grep pattern) checking with the OCSP.

Notably some errors were expedia.com and mozilla.com, probably because of chain issues. The latter is not reproducible anymore (has a new certificate). The first is.

More later.

[dcooper16]

For sites such as expedia.com the only solution is to not run the test, or at least not print out any results. When sending an OCSP query, you need to provide the following information for each certificate that you are asking about:

   CertID          ::=     SEQUENCE {
       hashAlgorithm       AlgorithmIdentifier,
       issuerNameHash      OCTET STRING, -- Hash of Issuer's DN
       issuerKeyHash       OCTET STRING, -- Hash of Issuers public key
       serialNumber        CertificateSerialNumber }

In the case of expedia.com, the server only sends its own certificate. Without any other certificates, we cannot provide the ocsp command with a copy of the issuer's public key. Without that, the ocsp command cannot generate the issuerKeyHash and so cannot even create an OCSP request.

The way the code is written, it will only work if the server provided more than one certificate:

     $OPENSSL ocsp -no_nonce -header Host ${host_header} -url "$uri" \
          -issuer $TEMPDIR/hostcert_issuer.pem -verify_other $TEMPDIR/intermediatecerts.pem \
          -CAfile $TEMPDIR/intermediatecerts.pem -cert $HOSTCERT -text &> "$tmpfile"

$TEMPDIR/intermediatecerts.pem contains the certificates (other than the server's certificate) that were sent by the server, and $TEMPDIR/hostcert_issuer.pem contains the first certificate from $TEMPDIR/intermediatecerts.pem. If $TEMPDIR/intermediatecerts.pem is empty, then $TEMPDIR/hostcert_issuer.pem won't be available, and the ocsp command needs that file in order to create issuerKeyHash.

In theory, if the server didn't provide any intermediate certificates, but the server's certificate was issued by one of the CAs in etc/Apple.com, etc/Linux.pem, etc/Microsoft.pem, or etc/Mozilla.pem, we could get the certificate needed for the -issuer option from one of those files, but trying to do that would be infeasible. Besides, in the case of expedia.com the chain of trust is "NOT ok", so even that wouldn't work.

[drwetter]

Expedia is another exception for the rest see #1057 (comment). Thus closing it.

[egberts]

Mmmmm, is it me or is it LetsEncrypt?

$ cd github/testssl.sh

$ OSSL_SHORTCUT=true ./testssl.sh --openssl=/usr/bin/openssl egbert.net

 Certificate Revocation List  --
 OCSP URI                     http://ocsp.int-x3.letsencrypt.org
 OCSP stapling                offered, error querying OCSP responder (ERROR: No Status found)
 OCSP must staple extension   --

Running the "latest" testing OpenSSL.

###########################################################
    testssl.sh       3.1dev from https://testssl.sh/dev/
    (4af9016 2020-10-20 21:00:02 -- )
 Using "OpenSSL 1.1.1d  10 Sep 2019" [~98 ciphers]
 on ns1:/usr/bin/openssl
 (built: "Apr 20 20:23:01 2020", platform: "debian-amd64")

Note: You have permission to run TLS-based tools against this site.

[drwetter]

can't tell, works for me.

In general: Thx for the permission but I assume when hostnames are not redacted I have the permission to scan. Also be aware this is a public space in the internet, so others probably will do that too if they are bored and have no pet or child to play with ;-)

[egberts]

It’s OSINT-blockable for those who abuse.