I was changing name servers for one of my domains. When testing the mail setup, I kept getting mail sent to the web server rather than the mail server. It turns out that if the mail server can’t find mx records, it falls back to A (or CNAME) records. So why was the mail server falling back to using the A/CNAME record when I had a perfectly good MX record assigned to the domain? After many hours of debugging and comparing working domain with non-working ones, the only difference I found was that I has a CNAME for the domain eg.
domain.com. 3600 IN CNAME www.domain.com
Turns out that if you have a CNAME for the domain, it redirects the lookup not just for A but for every record. So from dig, it looks fine if you query the authorative name servers directly – you see the mx and cname record – but it falls flat when queried on the recursive nameserver. Indeed after looking at RFC 1034, it states that “If a CNAME RR is present at a node, no other data should be present”. So setting the CNAME on the domain also consequently redirected requests for NS and SOA records, which made the domain quite invalid, although the subdomain records still resolved. Bottom line, don’t ever put CNAME on the root domain.
I was trying to be smart and save on retyping the IP by using CNAME… but turns out I was too smart.