Account for wildcard DNS resolution

[h-m-f-t]

https://yourefired.usajobs.gov is not a host that OPM should be held accountable for, and resolves only because of a wildcard DNS record. trustymail presumes that "no NXDOMAIN response" means there's an individual host there, but this is too simplistic.

TODOs:

• See how domain-scan tackled this problem
• Fix it

[jsf9k]

What do we want to do here? Just add another field in the output for each non-second-level domain indicating whether it's a wildcard match?

[konklone]

https://yourefired.usajobs.gov is not a host that OPM should be held accountable for

So, it depends. What circumstances in trustymail make it reasonable to treat this differently from a deliberately set record? For DMARC, you're requiring that base domains apply policy to all subdomains. For other record types, you're requiring that email-sending hostnames do some work and have STARTTLS enabled, and presumably yourefired.usajobs.gov is not being used to send email.

Also, when wildcard DNS is being used for legitimate purposes (such as *.app.cloud.gov being used for all cloud.gov sandbox apps), you don't want to necessarily drop all of those records from what you're scanning. And scanning app.cloud.gov won't necessarily give you the same results as [anything].app.cloud.gov.

See how domain-scan tackled this problem

Just a note that we've never used that in our actual scans. That code was written for a one-time analysis of DNS log data from the .gov TLD, to try to get a better signal-to-noise ratio.

I could see it potentially being broadly useful to have this information available during some kinds of scans. But it's tricky to avoid dropping legitimate services when wildcard DNS is in use.