Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug: Cannot get Record ID, Cloudflare DNS #243

Closed
SuperJakish opened this issue Sep 12, 2021 · 21 comments
Closed

Bug: Cannot get Record ID, Cloudflare DNS #243

SuperJakish opened this issue Sep 12, 2021 · 21 comments
Assignees

Comments

@SuperJakish
Copy link

SuperJakish commented Sep 12, 2021

TLDR: Cannot get Record ID, Cloudflare DNS

  1. Is this urgent: No

  2. DNS provider(s) you use: Cloudflare

  3. Program version: Running version latest built on 2021-09-10T22:10:33Z (commit 8b327f8)

  4. What are you using to run the container: docker-compose

  ddns_updater:
    image: qmcgaw/ddns-updater:latest
    container_name: ddns_updater
    restart: always
    volumes:
      - $DOCKERDIR/ddns_updater:/updater/data
    ports:
      - "<EXTENRNAL PORT1>:443/tcp"
      - "<EXTENRNAL PORT2>:53/udp"
      - "<EXTENRNAL PORT3>:8000/tcp"
    networks:
      - default
  1. Extra information (optional)
    I've been trying a few different config.json versions starting with the one in the example here, but it looks like Cloudflare has changed the way they authenticate from when this was written. Either that or I'm just not smart enough to put in the right keys/tokens. I was able to pull my ID using instructions from Cloudflare's API page which now uses Bearer to authenticate rather than X-Auth-Email and X-Auth-Key as in this documentation. The new call looks like this:
curl -X GET "https://api.cloudflare.com/client/v4/zones/<ZONE ID>/dns_records" \>
    -H "Authorization: Bearer <DNS API TOKEN>" \
    -H "Content-Type:application/json"

Logs:

INFO Updating record [domain: <DOMAIN> | host: @ | provider: cloudflare | ip: ipv4] to use <IP ADDRESS>
ERROR cannot get record ID: Get "https://api.cloudflare.com/client/v4/zones/<ZONE ID>/dns_records?name=<DOMAIN>&page=1&per_page=1&type=A": net/http: request canceled (Client.Timeout exceeded while awaiting headers)

Configuration file (remove your credentials!):

{
  "settings": [
    {
      "provider": "cloudflare",
      "zone_identifier": "<ZONE ID>",
      "identifier": "<ID>",
      "domain": "<DOMAIN>",
      "host": "@",
      "ttl": 600,
      "token": "<DNS API TOKEN>",
      "ip_version": "ipv4",
      "proxied": true
    }
  ]
}

Host OS: Ubuntu 20.04

@qdm12
Copy link
Owner

qdm12 commented Sep 12, 2021

Hi there! Thanks for the detailed issue.

I checked their official documentation and it looks it's the same as before. From their curl example:

curl -X GET "https://api.cloudflare.com/client/v4/zones/023e105f4ecef8ad9ca31a8372d0c353/dns_records?type=A&name=example.com&content=127.0.0.1&proxied=undefined&page=1&per_page=20&order=type&direction=desc&match=all" \
     -H "X-Auth-Email: user@example.com" \
     -H "X-Auth-Key: c2547eb745079dac9320b638f5e225cf483cc5cfdda41" \
     -H "Content-Type: application/json"

And that's how they are set in the code of this program:

headers.SetUserAgent(request)
headers.SetContentType(request, "application/json")
headers.SetAccept(request, "application/json")
switch {
case len(p.token) > 0:
headers.SetAuthBearer(request, p.token)
case len(p.userServiceKey) > 0:
request.Header.Set("X-Auth-User-Service-Key", p.userServiceKey)
case len(p.email) > 0 && len(p.key) > 0:
request.Header.Set("X-Auth-Email", p.email)
request.Header.Set("X-Auth-Key", p.key)
}

I doubt Cloudflare would break this behavior without releasing a v5 version of their API too.

In addition, the error you get:

net/http: request canceled (Client.Timeout exceeded while awaiting headers)

Looks like Cloudflare doesn't reply at all. If it would be an authentication error, you would get a 401 or 403 http status code as a response.

What are your thoughts? 🤔

@SuperJakish
Copy link
Author

SuperJakish commented Sep 12, 2021

Okay, so when I started writing this bug report, I had my config host set to "host": "*", which gave me a 403 authentication error:

INFO Updating record [domain: <MY DOMAIN> | host: * | provider: cloudflare | ip: ipv4] to use <IP ADDRESS>
ERROR cannot get record ID: bad HTTP status: 403: {"success":false,"errors":[{"code":10000,"message":"Authentication error"}]}

When I switch host to "host": "@",, I get the timeout error as above:

INFO Updating record [domain: <DOMAIN> | host: @ | provider: cloudflare | ip: ipv4] to use <IP ADDRESS>
ERROR cannot get record ID: Get "https://api.cloudflare.com/client/v4/zones/<ZONE ID>/dns_records?name=<DOMAIN>&page=1&per_page=1&type=A": net/http: request canceled (Client.Timeout exceeded while awaiting headers)

I'm not certain which I should be using, but my configuration uses a series of subdomains (sub1.mydomain.com; sub2.mydomain.com; etc).

@qdm12
Copy link
Owner

qdm12 commented Sep 12, 2021

Ok from Cloudflare's docs, you can use a name url parameter which ddns updater uses to narrow down the record id result to a single one. Now the problem is that it builds that name parameter as *.domain.com and host.domain.com for wildcard and subdomain records respectively I think. The Cloudflare doc doesn't specify if this is meant to be or not. My guess is that it can only be domain.com. That would explain the 403 for wildcards at least, and maybe their api crashes or something with two dots who knows 😄

To confirm that, would you be able to try curling the api as you did but using the url query parameter ?name=*.yourdomain.com and another with ?name=yourhost.yourdomain.com see if it works or not?
EDIT: also can you try with the @ host on your domain and the program as it is perhaps to see if it works or not?

If it doesn't, I'll change the code to use only the domain.tld as the name query parameter and filter the multiple results obtained from cloudflare.

@SuperJakish
Copy link
Author

Okay - you've crashed through the boundary of my understanding, but here is my attempt at following through with something useful. I curled the API using the following commands with the corresponding results:

First with just my domain (mydomain.com):

curl -X GET "https://api.cloudflare.com/client/v4/zones?name=<MY DOMAIN>&status=active&account.id=<IDENTIFIER>&page=1&per_page=20&order=status&direction=desc&match=all" \
>      -H "X-Auth-Email: <EMAIL>" \
>      -H "X-Auth-Key: <GLOBAL API KEY>" \
>      -H "Content-Type: application/json"
{"result":[],"result_info":{"page":1,"per_page":20,"total_pages":0,"count":0,"total_count":0},"success":true,"errors":[],"messages":[]}

Next with *.mydomain.com:

curl -X GET "https://api.cloudflare.com/client/v4/zones?name=*.<MY DOMAIN>&status=active&account.id=<IDENTIFIER>&page=1&per_page=20&order=status&direction=desc&match=all" \
>      -H "X-Auth-Email: <EMAIL>" \
>      -H "X-Auth-Key: <GLOBAL API KEY>" \
>      -H "Content-Type: application/json"
{"result":[],"result_info":{"page":1,"per_page":20,"total_pages":0,"count":0,"total_count":0},"success":true,"errors":[],"messages":[]}

and finally with @.mydomain.com:

curl -X GET "https://api.cloudflare.com/client/v4/zones?name=@.<MY DOMAIN>&status=active&account.id=<IDENTIFIER>&page=1&per_page=20&order=status&direction=desc&match=all" \
>      -H "X-Auth-Email: <EMAIL>" \
>      -H "X-Auth-Key: <GLOBAL API KEY>" \
>      -H "Content-Type: application/json"
{"result":[],"result_info":{"page":1,"per_page":20,"total_pages":0,"count":0,"total_count":0},"success":true,"errors":[],"messages":[]}

If this isn't what you're looking for, would you kindly please give the complete command you'd like me to execute?

@qdm12
Copy link
Owner

qdm12 commented Sep 18, 2021

Hi there sorry I lost track of the issue.

Try with:

curl -X GET "https://api.cloudflare.com/client/v4/zones/023e105f4ecef8ad9ca31a8372d0c353/dns_records?type=A&name=example.com" \
     -H "X-Auth-Email: user@example.com" \
     -H "X-Auth-Key: c2547eb745079dac9320b638f5e225cf483cc5cfdda41" \
     -H "Content-Type: application/json"

Replacing your zone id, domain name, auth email and auth key.

You should get at least one record, ideally multiple including the wildcard one 🤔

If you do I'll adapt the code to extract the right record id.

@SuperJakish
Copy link
Author

That seemed to work. Here is the result:

curl -X GET "https://api.cloudflare.com/client/v4/zones/<IDENTIFIER>/dns_records?type=A&name=<DOMAIN>" \
>      -H "X-Auth-Email: <EMAIL>" \
>      -H "X-Auth-Key: <GLOBAL API KEY>" \
>      -H "Content-Type: application/json"
{"result":[{"id":"<IDENTIFIER>","zone_id":"<ZONE ID>","zone_name":"<ZONE NAME>","name":"<DOMAIN>","type":"A","content":"<IP ADDRESS>","proxiable":true,"proxied":true,"ttl":1,"locked":false,"meta":{"auto_added":false,"managed_by_apps":false,"managed_by_argo_tunnel":false,"source":"primary"},"created_on":"2020-11-28T18:33:52.355102Z","modified_on":"2021-09-19T15:16:13.386835Z"}],"success":true,"errors":[],"messages":[],"result_info":{"page":1,"per_page":20,"count":1,"total_count":1,"total_pages":1}}

@qdm12
Copy link
Owner

qdm12 commented Sep 20, 2021

Ok interesting. Try pulling image qmcgaw/ddns-updater:243, it now lists records from cloudflare using "name": "domain.com" instead of "name": "sub.domain.com" (or "*.domain.com"), so that will solve at least the get records part. I'm not sure the update with "name": "*.domain.com" would work since Cloudflare doesn't precise how wildcards should be handled.

Actually, if it doesn't work, try with curl now that you have your single record ID ("id":"<IDENTIFIER>") using the command you used above:

curl -X PUT -d '{"type": "A", "name": "*.domain.com", "content": "127.0.0.1", "ttl": 3600}' 
          "https://api.cloudflare.com/client/v4/zones/<ZONE-ID>/dns_records/<RECORD-ID>" \
         -H "X-Auth-Email: <EMAIL>" \
        -H "X-Auth-Key: <GLOBAL API KEY>" \
        -H "Content-Type: application/json"

And fiddle with the "name": "*.domain.com" portion to see how this command succeeds. Note that if this works, it updates your wildcard record to 127.0.0.1. Thanks!!

PS: Also alternatively, Cloudflare supports DNS-O-Matic so you could use that instead since ddns-updater supports dns-o-matic.

@SuperJakish
Copy link
Author

Alrighty - tried to pull :243 with the following failure:

Pulling ddns_updater (qmcgaw/ddns-updater:243)...
ERROR: manifest for qmcgaw/ddns-updater:243 not found: manifest unknown: manifest unknown

Also tried the curl command

curl -X PUT -d '{"type": "A", "name": "*.<DOMAIN>", "content": "127.0.0.1", "ttl": 3600}' "https://api.cloudflare.com/client/v4/zones/<ZONE-ID>/dns_records/<RECORD-ID>" \
>         -H "X-Auth-Email: <EMAIL>" \
>         -H "X-Auth-Key: <GLOBAL API KEY>" \
>         -H "Content-Type: application/json"

With the following failure:

{
  "code": 1001,
  "error": "method_not_allowed"
}

FWIW, I tried with *.<DOMAIN> and just <DOMAIN> for the "name" entry with the same results.

I'll check out DNS-O-Matic this weekend to see if I can get that working. Thanks for the tip there!

@qdm12
Copy link
Owner

qdm12 commented Sep 23, 2021

Sorry for the dumb question, but did you replace <ZONE-ID> and <RECORD-ID> in the url? If you check their API docs at the Update DNS Record section, it should accept a PUT request as you wrote that curl command... 🤔

tried to pull :243 with the following failure:

What's your machine / cpu architecture? I just pushed it for amd64 that's probably why.

@SuperJakish
Copy link
Author

SuperJakish commented Sep 25, 2021

I did replace them, but apparently not with the correct thing. Re-checking the API doc, I found that I had used the wrong thing in the <RECORD-ID> section. That is actually the <IDENTIFIER>. So, now with the correct entries, here's what it looked like for <DOMAIN>:

curl -X PUT -d '{"type": "A", "name": "<DOMAIN>", "content": "<IP ADDRESS>", "ttl": 3600}' "https://api.cloudflare.com/client/v4/zones/<ZONE-ID>/dns_records/<IDENTIFIER>" \
>         -H "X-Auth-Email: <EMAIL>" \
>         -H "X-Auth-Key: <GLOBAL API KEY>" \
>         -H "Content-Type: application/json"
{"result":{"id":"<IDENTIFIER>","zone_id":"<ZONE-ID>","zone_name":"<ZONE NAME>","name":"<DOMAIN>","type":"A","content":"<IP ADDRESS>","proxiable":true,"proxied":false,"ttl":3600,"locked":false,"meta":{"auto_added":false,"managed_by_apps":false,"managed_by_argo_tunnel":false,"source":"primary"},"created_on":"2020-11-28T18:33:52.355102Z","modified_on":"2021-09-25T14:47:24.464278Z"},"success":true,"errors":[],"messages":[]}

and for *.<DOMAIN>:

curl -X PUT -d '{"type": "A", "name": "*.<DOMAIN>", "content": "<IP ADDRESS>", "ttl": 3600}' "https://api.cloudflare.com/client/v4/zones/<ZONE-ID>/dns_records/<IDENTIFIER>" \
>         -H "X-Auth-Email: <EMAIL>" \
>         -H "X-Auth-Key: <GLOBAL API KEY>" \
>         -H "Content-Type: application/json"
{"result":{"id":"<IDENTIFIER>","zone_id":"<ZONE-ID>","zone_name":"<ZONE NAME>","name":"*.<DOMAIN>","type":"A","content":"<IP ADDRESS>","proxiable":false,"proxied":false,"ttl":3600,"locked":false,"meta":{"auto_added":false,"managed_by_apps":false,"managed_by_argo_tunnel":false,"source":"primary"},"created_on":"2020-11-28T18:33:52.355102Z","modified_on":"2021-09-25T14:44:36.609432Z"},"success":true,"errors":[],"messages":[]}

There was a problem with the *.<DOMAIN> because it correctly showed *.<DOMAIN> in the output "name" field, but the name on the Cloudflare portal was only * and none of the subdomain CNAME entries routed properly. I think that means just <DOMAIN> is preferred.

The second thing here is that I run my domain proxied through Cloudflare and this script reverts the domain to DNS Only (not proxied). I recall you have a setting in the config for the proxy setting and this is probably covered - just wanted to mention it if some other functionality was expected.

Otherwise, I was able to successfully update the A record with an IP address.

What's your machine / cpu architecture?

I'm runing on x86_64.

@qdm12
Copy link
Owner

qdm12 commented Sep 26, 2021

and none of the subdomain CNAME entries routed properly. I think that means just is preferred.

So does that mean you cannot update *.<DOMAIN> alone and need to update all A records for <DOMAIN> together? Or that each subdomain hosts (including wildcard *) need to be CNAME records and cannot be A/AAAA records? If that's the case, for now the program doesn't support CNAME records but I can change that if it's actually required for Cloudflare and subdomains/wildcard.

Re-checking the API doc, I found that I had used the wrong thing

Oops sorry, maybe it was also me misleading you!

The second thing here is that I run my domain proxied

If you want to test, you can just add "proxied": true to the JSON body (after that -d flag in the curl command) to update it and keep it proxied 😉 And yes it's covered in this program.

I'm runing on x86_64.

Oh Ok yes, I forgot to push it oops! 🤔 It's pushed now!

@SuperJakish
Copy link
Author

In Cloudflare, all the subdomains are CNAME records that point to the A record. The A record is the only one that needs to be updated, so <DOMAIN> looks to be sufficient. I may have led us on a wild goose chase with the subdomain thing. That subplot can be dropped when this is turned into a movie.

Okay, I pulled down 243 and everything looks good. It successfully updated and proxied the A record as expected. I tried with both * and @ host options with no difference in the result, though the log did have a difference.

For Host @:

INFO Last IPv4 address stored for <DOMAIN> is <nil> and your IPv4 address is <nil>
INFO Updating record [domain: <DOMAIN> | host: @ | provider: cloudflare | ip: ipv4] to use <IP-ADDRESS>

And for Host *:

INFO Last IPv4 address stored for any.<DOMAIN> is <nil> and your IPv4 address is <nil>
INFO Updating record [domain: <DOMAIN> | host: * | provider: cloudflare | ip: ipv4] to use <IP-ADDRESS>

Here is a look at my settings.json:

{
  "settings": [
    {
      "provider": "cloudflare",
      "zone_identifier": "<ZONE-ID>",
      "identifier": "<IDENTIFIER>",
      "domain": "<DOMAIN>",
      "host": "*",
      "ttl": 600,
      "token": "<API-TOKEN>",
      "ip_version": "ipv4",
      "proxied": true
    }
  ]
}

Thanks very much for your patience in working through this with me! I think it's working!

@qdm12
Copy link
Owner

qdm12 commented Sep 27, 2021

I may have led us on a wild goose chase with the subdomain thing. That subplot can be dropped when this is turned into a movie.

That made me laugh quite a bit 😄

Okay, I pulled down 243 and everything looks good

Good I'll merge it in the latest image before closing this issue.

Thanks very much for your patience in working through this with me! I think it's working!

Likewise, thanks for the long debugging!

One last question, so are you telling me you cannot have different subdomains with different ip addresses with Cloudflare? If that's the case, removing subdomain and wildcard options would be good for this program.

@SuperJakish
Copy link
Author

One last question, so are you telling me you cannot have different subdomains with different ip addresses with Cloudflare? If that's the case, removing subdomain and wildcard options would be good for this program.

I'm not exactly sure, but here is what I found when making the API token. I had to create the token and then could restrict it to either a single specific zone, all zones, or zones only affiliated with my account (email address). This was configured at the token level. So if I had 2 zones that each pointed to different IP addresses, I'd only want to update the DNS of the specific zone (based on the Zone ID). The token would only have permission to edit that zone for added security.

I'm not sure how other people use this updater, but I've only got the one zone I'm worrying about, so all the subdomains point to the same A record. Based on this, if I had different servers with different IPs, I'd think DDNS Updater would need to be running locally on each of them, so it wouldn't be worth supporting the wildcards as each server would update it's own zone.

Your mileage may vary.

@qdm12
Copy link
Owner

qdm12 commented Oct 2, 2021

Alright, I think I'll leave it as it is just in case it's actually used somehow by someone and to avoid breaking compatibility.

87f06ee adds documentation on that host parameter for Cloudflare and links up to your comment for context, thanks for that. Let's close the issue for now 😉 Thanks again for your patience and debugging!

@qdm12 qdm12 closed this as completed Oct 2, 2021
@TheVooDooDaddy
Copy link

TheVooDooDaddy commented Apr 28, 2022

I was able to get this to work with two separate domains and www subdomains. Leave the "host" as "@" and then add the subdomain to the "domain". My config.json file is below:

{
  "settings": [
    {
      "provider": "cloudflare",
      "zone_identifier": "<ZONE-ID>",
      "domain": "<DOMAIN>",
      "host": "@",
      "ttl": 600,
      "token": "<API-TOKEN>",
      "ip_version": "ipv4"
    },
    {
      "provider": "cloudflare",
      "zone_identifier": "<ZONE-ID>",
      "domain": "www.<DOMAIN>",
      "host": "@",
      "ttl": 600,
      "token": "<API-TOKEN>",
      "ip_version": "ipv4"
    },
    {
      "provider": "cloudflare",
      "zone_identifier": "<ZONE-ID>",
      "domain": "<DOMAIN>",
      "host": "@",
      "ttl": 600,
      "token": "<API-TOKEN>",
      "ip_version": "ipv4"
    },
    {
      "provider": "cloudflare",
      "zone_identifier": "<ZONE-ID>",
      "domain": "www.<DOMAIN>",
      "host": "@",
      "ttl": 600,
      "token": "<API-TOKEN>",
      "ip_version": "ipv4"
    }
  ]
}

@jk-andersen
Copy link

jk-andersen commented Jun 9, 2023

Just to add on that. This works now with the API.

My config:

	{
      "provider": "cloudflare",
      "zone_identifier": "<ID>",
      "domain": "domain.xyz",
      "host": "*",
      "ttl": 600,
      "token": "<API_KEY>",
      "ip_version": "ipv4"
    }

https://i.imgur.com/OLMjwH2.png

@qdm12
Copy link
Owner

qdm12 commented Jun 12, 2023

@jk-andersen Does it work with any subdomain such as abc for the "host" parameter?

@jk-andersen
Copy link

After a fast smoke-test I would say yes:
https://i.imgur.com/chyNw1T.png

{
   "provider": "cloudflare",
   "zone_identifier": "ZONE_IDENTIFIER",
   "domain": "DOMAIN",
   "host": "abc",
   "ttl": 600,
   "token": "TOKEN",
   "ip_version": "ipv4"
 },

@qdm12
Copy link
Owner

qdm12 commented Jun 12, 2023

Awesome @jk-andersen I documented it in the docs/Cloudflare.md document 👍

@baaa
Copy link

baaa commented Jul 19, 2023

I'll drop this line of comment here for people like me and landed on this page.

I was getting this same error message and while debugging I noticed you are querying only for "A" records.
I had a CNAME configured for wildcard where * was pointing to domain.com, so your query naturally found nothing. Only after deleting the CNAME on cloudflare and adding an A record for wildcard this started working.

honestly this is an oversight/error on my side, as it literally makes no sense to "update the ip address of a CNAME" since a CNAME is a resolution from name to name.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants