implement wildcard DNS

[Cottand]

Implement simple wildcard DNS matching. For a server with

  example.com.   3600 IN A 2.2.2.2
*.example.com.   3600 IN A 1.1.1.1

You will now get 1.1.1.1 when querying for www.example.com as per https://datatracker.ietf.org/doc/html/rfc4592.

Note that after the PR, this lib is still not rfc4592 compliant because for:

    example.com.   3600 IN A 1.1.1.1
*.*.example.com.   3600 IN A 2.2.2.2

foo.bar.example.com would still match example.com rather than the wildcards.

But still, this PR is small and has no backwards-incompatible changes (assuming you were not using wildcards, which were not compliant anyway).

So I do think it's worth getting closer to compliance until we get the full implementation

[miekg]

If we are touching this code, it should be done in a RFC compliant matter. This current PR is a step in the right direction, but should take it all the way IMO

[Cottand]

I was hoping to get this out until you managed to find some time to pull the complexity from CoreDNS.
I can try to implement it all the way but it will take me longer.

[Cottand]

the common case of www.example.com matching *.example.com should be fast as it is always the first iteration of the for loop after the continue

[Cottand]

@miekg I have tried to implement full compliance. Please see the tests for examples

[janik-cloudflare]

The *.*.example.com. example doesn't seem right. A single wildcard should be enough to match any number of labels. In the case of *.*.example.com., the second * should be interpreted as a literal * character.

[Cottand]

Thanks @janik-cloudflare - Looking at your profile, there's a chance you understand DNS better than I do!

I gave the RFC another read, and realised that foo.bar.example.com should not match *.*.example.com like you said. I will fix the PR.

On the other hand, I am not sure that "A single wildcard should be enough to match any number of labels". See this section of the RFC. I will paste below the bits I deemed relevant (emphasis also mine):

[..] consider this complete zone:

 $ORIGIN example.
 example.                 3600 IN  SOA   <SOA RDATA>
 example.                 3600     NS    ns.example.com.
 example.                 3600     NS    ns.example.net.
 *.example.               3600     TXT   "this is a wildcard"
 *.example.               3600     MX    10 host1.example.

[The following response] would not be synthesized from any of the
wildcards in the zone:

QNAME=ghost.*.example., QTYPE=MX, QCLASS=IN
          because *.example. exists

The final example highlights one common misconception about
wildcards. A wildcard "blocks itself" in the sense that a wildcard
does not match its own subdomains. That is, *.example. does not
match all names in the "example." zone; it fails to match the names
below *.example..
To cover names under *.example., another
wildcard domain name is needed-- *.*.example. --which covers all but
its own subdomains.

Prompted by your comment, I also set up a *.dcotta.eu TXT record with Cloudflare, and I get no response when I dig TXT hello.there.dcotta.eu.

Please let me know if this sounds right to you too, and thanks again!

[janik-cloudflare]

Wildcards are, unfortunately, very messy and complicated. (Continue reading at your own risk; it may make you never want to touch DNS again.) "A single wildcard should be enough to match any number of labels" is generally true, but perhaps a bit of an oversimplification. Let's say you have a wildcard record *.example.com (type A) and a non-wildcard record www.example.com (type TXT).

1. Because www.example.com exists directly, queries for that name would never match the wildcard record, regardless of the query type (i.e., even if querying for A records, which the www.example.com record doesn't provide but the wildcard would).
2. The record on www.example.com would also block subdomains such as abc.www.example.com (or sub-subdomains, etc.) from matching the wildcard, again regardless of query type.
3. As a consequence, wildcards don't apply across zone boundaries (because the existence of an NS record blocks it).
4. However, in general, if there is nothing "blocking" the wildcard, it will work across an arbitrary subdomain label depth.
5. The example you quoted from RFC 4592 section 2.2.1 seems to describe the behavior when querying for something like sub.*.example.com (that's a literal * in the query name). In that case, the RFC states that the wildcard record *.example.com does not apply because the existence of *.example.com blocks the wildcard from applying to subdomains of that name (just like the existence of www.example.com blocked abc.www.example.com in my example from item no. 2). People wouldn't normally query for a name containing a literal * so this is a bit of an obscure (but nevertheless important) edge case.

I recommend https://www.ietf.org/slides/slides-edu-dns-00.pdf slides 14-16 as a good overview of these behaviors. Implementing all this efficiently gets quite challenging, especially when empty non-terminals are involved, and it's something we've been having lots of fun with too. For empty non-terminals you'd also need to make sure to return the correct response code (i.e., if sub.www.example.com exists then even if there are no records for www.example.com, queries for www.example.com should return NOERROR instead of NXDOMAIN, because the existence of a subdomain implies that the parent exists, even if it is empty).

This works for me, by the way:

~ % dig +short TXT hello.there.dcotta.eu @1.1.1.1
"matched single wildcard"

[renanbastos93]

Have we ever had any updates yet?

[Cottand]

Hi, sorry for the inactivity. Quite frankly, Janik's post was perscient

Continue reading at your own risk; it may make you never want to touch DNS again

I just own a project that makes use of this library and would love to see #1534 fixed. But I am not a DNS expert (and to be honest I do not have the energy to try to become one) so it is hard for me to fix this myself.

Ideally I would love @miekg to implement this properly, as he is much more knowledgeable than me. I appreciate he might also not have the bandwidth (and that is fair enough) but in that case I think it would be reasonable to remove RFC-4592 (DNS wildcards) from the README's 'supported RFCs'.