CNAME lookups

[koter84]

i suddenly have an error on one of my domains where the authorative DNS-server lookup fails, i'm looking into it and trying to debug it, and now i'm wondering why the workflow is the way it is....
(i'm thinking my DNS-provider changed something on their servers causing this to suddenly fail)

what i don't understand is that it is looking for a CNAME and then follows the CNAME, but the CNAME could point to a different domain, which has a different authorative nameserver.
I'd think that you should only go up the domain ladder if there isn't a NS record for the domain, and don't follow the CNAME.

for instance if cname.example.com is a CNAME record for google.com the authorative server should be the nameserver for example.com and not google.com

i think it should be that if there aren't any NS records for cname.example.com the script should check example.com for the NS records.
for test.cname.example.com it should first lookup the NS for test.cname.example.com, then cname.example.com and last example.com

it seems to me that it doesn't make sense to follow the CNAME since even if cname.example.com points to test.example.com and test.example.com has it's own authorative nameservers still the nameservers for example.com should be checked used for cname.example.com

[koter84]

i did a test with a CNAME to verify the Let's Encrypt validation.

i made a DNS entry cnametest.domain1.com pointing to cnametest.domain2.com
then i created a getssl config for cnametest.domain1.com and setup my webserver to host this.

when requesting a certificate the script made _acme-challenge.cnametest under domain1.com
and Let's Encrypt validated the DNS-01 challenge.

so it seems to me that the CNAME should not be followed to the pointed hostname to find an authoritative server. Though in my test both domain1.com and domain2.com have the same authoritative nameserver.

so i did a second test, with cnamegoogle.domain1.com pointing to google.com since google does not have the same authoritative nameservers as my domains.
this gave an error...
getssl: cnamegoogle.domain1.com:Verify error:CAA record for cnamegoogle.domain1.com prevents issuance
searching for CAA i found that it is a system to prevent CA's to issue certificates for that domain, so google is not a good domain for the thing i'm testing since they have to much security...

so i pointed cnamegoogle to marktplaats.nl after checking if they have CAA records....
this did work, and i got the certificate from Let's Encrypt

so it seems to me that CNAME records should not be followed when determining the authoritative nameservers. unless it is to check the CAA records, but i think that should be out-of-scope for getssl.

i have changed the code to be able to test these scenarios, and i'll create a pull-request in a couple of days, right now the code is a bit messy and should be cleaned up a bit before sending the PR.

[QuingKhaos]

I know this ist old. But the verify error, could this come from the ACME server itself?

[donmurphycanada]

In summary I am using a recent copy of getssl (getssl ver. 2.30) and have used it to obtain Let's Encrypt certificates for several domains successfully however when trying to use the cname method with references to another domain the getssl script is unable to verify the domain.

Details:

1. • Using the method "The Advantages of a CNAME" with two domains as presented in https://letsencrypt.org/2019/10/09/onboarding-your-customers-with-lets-encrypt-and-acme.html
2. • I have created a cname in DomainA which refers to txt records in DomainB
3. • I configure getssl (via DNS_ADD_COMMAND and DNS_DEL_COMMAND) to use dns_add_manual and dns_del_manual so I am in full control of the DNS record creation process and I set the TTL to 60 seconds to have full control over these records. This allows me to create a txt record in domainB
4. • when testing with "dig -t txt DomainA.DomainB +short" I noticed the challenge value is returned correctly
5. • when testing with "dig @NameServersOfDomainA -t txt DomainA.DomainB +short" I noticed the challenge value is not returned and blank

If I take (getssl ver. 2.30 line 1174)
check_result=$($DNS_CHECK_FUNC TXT "_acme-challenge.${d}" "@${ns}" \

and drop the "@${ns}"
check_result=$($DNS_CHECK_FUNC TXT "_acme-challenge.${d}" \

The check passes and the validation is performed.

Line 1174 effectively translates from:
dig TXT _acme-challenge.DomainA @NameserversOfDomainA
to
dig TXT _acme-challenge.DomainA

What I suggested above simply allowed the domain validation to be performed in a hack sort of way. I think this needs to be fixed and it appears as if this has already been addressed? #381

Why am I seeing this behaviour?

[timkimber]

Hi @donmurphycanada

Thanks for the bug report, there have been a lot of changes to code which checks the DNS records have been updated since #381 so I'll take a look at the CNAME handling.

[timkimber]

I've just tested this using a domain I control, Namecheap DNS and the Letsencrypt staging server and everything works as expected:

adding dns via command: /getssl/dns_scripts/dns_add_manual www.getssl.uk.test asP-N_pdCWItxiE02YmgYdUF7bQjFPUVZuvEvgIyjxQ
In the DNS, a new TXT record needs to be created for;
_acme-challenge.www.getssl.uk.test
containing the following value
asP-N_pdCWItxiE02YmgYdUF7bQjFPUVZuvEvgIyjxQ
Press any key to obtain the certificate once the records have been updated...
 
Using dig SOA +trace +nocomments www.getssl.uk.test @ to find primary nameserver
 
Checking for CNAME using dig CNAME www.getssl.uk.test @
 
Domain is a CNAME, actual domain is getssl.uk.test

Using dig NS getssl.uk.test @ to find primary nameserver
 
primary_ns dns2.namecheaphosting.com

loading DNSfile: /root/.getssl/www.getssl.uk.test/tmp/dns_verify/www.getssl.uk.test

checking dns at dns2.namecheaphosting.com

dig TXT _acme-challenge.www.getssl.uk.test @dns2.namecheaphosting.com
 
expecting  asP-N_pdCWItxiE02YmgYdUF7bQjFPUVZuvEvgIyjxQ

dns2.namecheaphosting.com gave ...
checking DNS at dns2.namecheaphosting.com for www.getssl.uk.test. Attempt 1/100 gave wrong result,  waiting 10 secs before checking again
 
dig TXT _acme-challenge.www.getssl.uk.test @dns2.namecheaphosting.com
 
expecting  asP-N_pdCWItxiE02YmgYdUF7bQjFPUVZuvEvgIyjxQ

dns2.namecheaphosting.com gave ...
asP-N_pdCWItxiE02YmgYdUF7bQjFPUVZuvEvgIyjxQ
sleeping 60 seconds before asking the ACME-server to check the dns

Can you attach debug logs for your domain so I can see which check isn't working as expected.

[donmurphycanada]

Sorry I would love to help unfortunately I can't post the logs with the names etc.

As I said this is the code that is not returning the correct value:
dig TXT _acme-challenge.www.getssl.uk.test @dns2.namecheaphosting.com

I run the command manually and it is the same result? I moved the name servers and the records to a different dns server and it worked straight away. As I leave off the @dns2.namecheaphosting.com I get the result and I get validated.

Very odd indeed

Thanks for looking into it but I don't think the software is buggy.

[donmurphycanada]

Nice to see someone with the same issue. So much confusion surrounding this
https://superuser.com/questions/1300671/recursive-dns-server-answers-with-cname-but-does-not-return-the-a-record

[donmurphycanada]

The other thing I have observed / determined is this is simply a preliminary test condition and will not prevent certificates from being issued because if you can get past this issue in some fashion (I simply drop the @nameserver from the dig command) then the domain validation is deemed valid by getssl.

Let’s Encrypt on the other hand does not use this check directly, it performs it’s own dns check to ensure you are the owner of the domain and Let’s Encrypt has been validating my domain and issuing all the certificates I have requested even though getssl fails.

[timkimber]

Hi @donmurphycanada

Thanks for the updates. If you set PUBLIC_DNS_SERVER to "" then getssl will have the behaviour you need (i.e. it won't add @ NS to the dig query).

The duckdns.org servers have a similar behaviour (i.e. dig ubuntu-staging.duckdns.org @ ns1.duckdns.org returns nothing so I'm just writing some unit tests to make sure that we handle this scenario.

[timkimber]

Hi @donmurphycanada

Can you test in the multiple-ns branch - I've just pushed some changes to how the CNAME lookups work so would be interested to know if it fixes the problem you're seeing

Thanks

[NicoP-S]

Hi @timkimber i just updated from 2.30 to 2.31 and CNAME lookups does not work anymore.

Does not work:

elif [[ "$DNS_CHECK_FUNC" == "drill" ]] || [[ "$DNS_CHECK_FUNC" == "dig" ]]; then
        debug "$DNS_CHECK_FUNC" TXT "_acme-challenge.${lower_d}" "@${ns}"
        check_result=$($DNS_CHECK_FUNC TXT "_acme-challenge.${lower_d}" "@${ns}" \
                      | grep -i "^_acme-challenge.${lower_d}" \
                      | grep 'IN\WTXT'|awk -F'"' '{ print $2}')

Before the first grep was not there. I think this breaks the check.

[timkimber]

Hi @NicoP-S

Thanks for reporting this. I remember I needed to add that line, but can't remember why.

I'm happy to remove it and release as a new version if that will fix the problem for you.

I'm going to add some test cases for CNAMEs as I think that's one of the problems using acme-dns (#600). Can you confirm that you have something like the following as your dns entry (example taken from a public letsencrypt forum post):
_acme-challenge.sites.karotte.org. IN CNAME sites.karotte.org._acme_challenges.challenges.karotte.org.

[sidrew]

So... this worked in earlier versions, such as v2.27 ... here's an example of a CNAME configuration:

dig TXT _acme-challenge.banxe-appdev.wichita.edu @156.26.1.30

;; ANSWER SECTION:
_acme-challenge.banxe-appdev.wichita.edu. 28800 IN CNAME banxe-appdev._acme-challenge.wichita.edu.
banxe-appdev._acme-challenge.wichita.edu. 600 IN TXT "<something>"

In 2.27, et al... the drill/dig check looked like this:

            check_result=$($DNS_CHECK_FUNC TXT "_acme-challenge.${d}" "@${ns}" \
                           | grep 'IN\WTXT'|awk -F'"' '{ print $2}')

... this worked beautifully... but for some reason in 2.31 ... this was changed to this...

        check_result=$($DNS_CHECK_FUNC TXT "_acme-challenge.${lower_d}" "@${ns}" \
                      | grep -i "^_acme-challenge.${lower_d}" \
                      | grep 'IN\WTXT'|awk -F'"' '{ print $2}')

... which of course in this case completely prevents check_result from returning the actual TXT record. So, I'm trying to figure out why this was added... at this point, we're just staying at v2.27 for now...

[NicoP-S]

Can you confirm that you have something like the following as your dns entry (example taken from a public letsencrypt forum post):
_acme-challenge.sites.karotte.org. IN CNAME sites.karotte.org._acme_challenges.challenges.karotte.org.

@timkimber Yes our CNAME records look like this.

[sidrew]

This one is still an issue...

[sideeffect42]

Does #654 help?

[github-actions]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[timkimber]

I'm moving the staging server tests to use acme-dns and have managed to reproduce this issue and will fix.

[sidrew]

I'm moving the staging server tests to use acme-dns and have managed to reproduce this issue and will fix.

Excellent... we've stuck with v2.27 as a result of this bug. Glad to see it getting some love.

[sidrew]

I'm moving the staging server tests to use acme-dns and have managed to reproduce this issue and will fix.

Any update on this issue?

[timkimber]

Sorry, I've just not had time to progress this, although it's high on my list of things I want to fix. I'll do the quick bug fixes next then start on this

[sidrew]

Sorry, I've just not had time to progress this, although it's high on my list of things I want to fix. I'll do the quick bug fixes next then start on this

Quick bump on this... curious if you've made any progress.

[timkimber]

@sidrew I got 80% of the way to making it work, then ran out of time. Now that I'm sort-of on top of bugs/issues I'll see if I can find what I did last time and complete it.

[timkimber]

@sidrew can you try the version in the acme-dns branch - it works for the tests I wrote which use acme-dns but want to check that it works for you too before releasing

[sidrew]

@sidrew can you try the version in the acme-dns branch - it works for the tests I wrote which use acme-dns but want to check that it works for you too before releasing

@timkimber I see what you've done with the script... looks like the CNAME check is happening in get_auth_dns() ... which should work. However, this is only called if the user isn't specifying a DNS server... as we are. So... the code to check and handle a CNAME isn't being called.

[timkimber]

Thanks @sidrew I've managed to reproduce the CNAME problem by setting a DNS_SERVER, I'll work on a better fix

[sidrew]

Thanks @sidrew I've managed to reproduce the CNAME problem by setting a DNS_SERVER, I'll work on a better fix

Awesome, @timkimber ... it looks like that CNAME discovery block could be jammed in a separate function... yeah?

[timkimber]

Hi @sidrew

I've updated the version in the acme-dns with a better CNAME fix (and added tests and unit tests). If you have time to test that would be great, otherwise I'll release it in a day or two.

[sidrew]

Hi @sidrew

I've updated the version in the acme-dns with a better CNAME fix (and added tests and unit tests). If you have time to test that would be great, otherwise I'll release it in a day or two.

@timkimber ... there we go... now she's working. Thanks for the effort!

[sidrew]

@timkimber ... I think this one is ready to merge and close... yeah?

[timkimber]

@sidrew it is, I'm just fixing one broken test and then I can merge the changes