The myfamily.com DNS Poisoning Problem Understood

Several people have observed a problem on their networks where various web sites, apparently at random, would be replaced by www.myfamily.com. The problem comes and goes without obvious cause, and affects different web sites at different times. I started encountering this problem a few days ago and tracked down the cause.

Something funny started happening with my web browser…

I clicked on a link to, say, www.imdb.com, and found myself looking at the home page for www.myfamily.com. My browser’s URL bar indicated http://www.imdb.com/ but the contents of the page were obviously not from that site. At first I wondered whether imdb had gone out of business and given up their domain name (hard to believe) or if someone had hijacked their domain name (unlikely but it has happened to other companies). Or had someone or something infected my browser, my computer, or my network infrastructure, taking control of what domains I would access? I had to understand the real cause.
I used nslookup to find the numeric IP address for www.imdb.com and it reported:

$ nslookup www.imdb.com.
Server:         209.157.144.2
Address:        209.157.144.2#53

Name:   www.imdb.com
Address: 66.43.25.130

I looked up this IP address in whois:

$ whois 66.43.25.130
[Querying whois.arin.net]
[whois.arin.net]

OrgName:    Myfamily.com, Inc.
OrgID:      MYFAMI-1
...

NetRange:   66.43.16.0 - 66.43.31.255
CIDR:       66.43.16.0/20
NetName:    MYFAMILY
...

Strange. The DNS (at least the DNS server on my network) is giving a bogus IP address when I look up this particular domain name. The bogus address seems to belong to a legitimate, and probably not malicious, organization. About this time, the problem disappeared (www.imdb.com started giving me the real imdb site) and I stopped investigating it for the moment.

A little while later, I typed in a URL to a different web site and got redirected to www.myfamily.com again. Actually, I didn’t even notice this at first, and stupidly typed in my username and password from the other site into myfamily.com’s login form! When I noticed what had happend, I was starting to realize the severe security implications of this problem. As far as I could tell, this was an unintentional misconfiguration or bug somewhere in the Internet’s DNS, but if someone started taking advantage of this error, they could presumably do a lot of harm. I started searching the web to see if anyone else had seen this problem. I found about 3 or 4 sites on the web that mentioned the exact same symptom, even including www.myfamily.com as the site being substituted.

Investigation

I observed that the immediate cause of the problem was that my local DNS server was returning the same bogus result about 10% of the time, apparently at random, for domains ending in .com. Once this bogus result gets into my DNS server for a given domain, it stays there for about 15 minutes and then gets deleted, allowing that domain to begin normal operation again. I could tell by dumping out the state of my DNS server’s cache that the bogus data had been placed there by the name server at [66.43.24.8]. But there shouldn’t be any reason that my name server would ask [66.43.24.8] for data about, e.g. www.imdb.com. It should be going to one of the .com TLD servers for that kind of data. Then I noticed a few more interesting facts:

The name server at [66.43.24.8] is configured in such a way that any address query sent to it always returns the same answer: 66.43.25.130 – the IP address of www.myfamily.com. That’s right, if you ask this name server what is the IP address for www.imdb.com or anything else, it will give you the myfamily.com web server’s address instead. And for any MX (mail exchanger) query sent to it, it always sends you to the mta.myfamily.com mail server.

My name server believed that [66.43.24.8] and [66.43.24.8] (a.k.a. mfns1.myfamily.NET and mfns2.myfamily.NET) were two of the .com TLD servers:

$ nslookup -type=ns com.
Server:         209.157.144.2
Address:        209.157.144.2#53

Non-authoritative answer:
com     nameserver = G.GTLD-SERVERS.NET.
com     nameserver = H.GTLD-SERVERS.NET.
com     nameserver = I.GTLD-SERVERS.NET.
com     nameserver = J.GTLD-SERVERS.NET.
com     nameserver = K.GTLD-SERVERS.NET.
com     nameserver = L.GTLD-SERVERS.NET.
com     nameserver = M.GTLD-SERVERS.NET.
com     nameserver = A.GTLD-SERVERS.NET.
com     nameserver = B.GTLD-SERVERS.NET.
com     nameserver = C.GTLD-SERVERS.NET.
com     nameserver = D.GTLD-SERVERS.NET.
com     nameserver = E.GTLD-SERVERS.NET.
com     nameserver = mfns2.myfamily.NET.
com     nameserver = mfns1.myfamily.NET.
com     nameserver = F.GTLD-SERVERS.NET.

That explained why it was (sometimes) going to the evil name servers [66.43.24.8] and [66.43.24.9] and asking them to resolve things. But how did these two servers manage to get themselves listed in such an important list in my name server? I had heard of DNS cache poisoning attacks, and I even suspected that the old version of the BIND software I’m running is probably vulnerable to them, but this really didn’t seem like any kind of intentional attack. Who would intentionally try to redirect my web browser to something as (apparently) innocuous as www.myfamily.com?

Restarting my name server would always fix the problem, but it would always recur within an hour or so. Using tcpdump, I started capturing and logging all packets going to or from my name server to try to isolate the event that poisoned the .com name server list. I eventually spotted it:

11:57:16.860000 myns.domain > 66.43.24.8.domain: 37457+ MX? calvin.bellahs.com. (36)
11:57:16.960000 66.43.24.8.domain > myns.domain: 37457* 1/2/3 MX mta.myfamily.com. 10 (165) (DF)
11:57:57.510000 myns.domain > 66.43.24.9.domain: 37461 A? www.mastergrip.com. (36)
11:57:57.610000 66.43.24.9.domain > myns.domain: 37461* 1/2/2 A myfamily.com (136) (DF)

Something asked my name server to find the mail exchanger for the domain calvin.bellahs.com, and my name server passed on this query to [66.43.24.8], which responded as it always does (”mta.myfamily.com”). (Incidentally, the “something” was my mail server trying to reject some spam that had been forged with a from address @calvin.bellahs.com.) Up until that moment, my name server was operating correctly, and after that moment, it was corrupt. So was this calvin.bellahs.com query and response the cause of the corruption or just the first effect of it? A simple nslookup suggested that it was the former:

$ nslookup -type=ns bellahs.com.
Server:         209.157.144.2
Address:        209.157.144.2#53

Non-authoritative answer:
bellahs.com     nameserver = mfns1.myfamily.net.
bellahs.com     nameserver = mfns2.myfamily.net.

Authoritative answers can be found from:
mfns1.myfamily.net      internet address = 66.43.24.8
mfns2.myfamily.net      internet address = 66.43.24.9

So my name server was behaving properly when it contacted [66.43.24.8] for the first time, since that is an authoritative name server for bellahs.com. But something in the response to that query permanently poisoned my name server with the belief that [66.43.24.8] and [66.43.24.9] can be consulted for future queries about any .com domains, not just bellahs.com and myfamily.com. The right nslookup invocation reveals the poison explicitly:

$ nslookup -debug -type=mx calvin.bellahs.com. 66.43.24.8
Server:         66.43.24.8
Address:        66.43.24.8#53

------------
    QUESTIONS:
        calvin.bellahs.com, type = MX, class = IN
    ANSWERS:
    ->  calvin.bellahs.com
        mail exchanger = 10 mta.myfamily.com.
    AUTHORITY RECORDS:
    ->  com
        nameserver = mfns2.myfamily.net.
    ->  com
        nameserver = mfns1.myfamily.net.
    ADDITIONAL RECORDS:
    ->  mta.myfamily.com
        internet address = 66.43.25.130
    ->  mfns1.myfamily.net
        internet address = 66.43.25.130
    ->  mfns2.myfamily.net
        internet address = 66.43.25.130
------------
calvin.bellahs.com      mail exchanger = 10 mta.myfamily.com.

Here we can finally see the gross misconfiguration that causes all this trouble. Somehow, the (official, legitimate) DNS server for myfamily.com and bellahs.com has been configured to claim, once it has been contacted, that it is an authoritative name server for the entire .com top level domain itself. Normally, this wouldn’t be so terrible, as long as it did a decent job of answering the misdirected queries for other .com domains, but it’s also misconfigured in another terrible way: It gives bogus answers to queries for domains other than myfamily.com.

But the blame does not lie entirely with the sloppy administration of myfamily.com’s name server. There are two other culprits:

  1. The DNS software running on my name server should not have accepted the foreign name server’s claim that it is authoritative for .com. This is poor software design. This is probably why so few people on the Internet have encountered this problem – you have to be running your own name server, using software that has this flaw. In my case, it was BIND 4.9.3, but there may be other DNS implementations (for MS-Windows?) that have the same flaw.
  2. The administrator of my name server (that’s me) should not be running such ancient (10 years old) DNS software on a name server. This problem is most likely fixed in later versions of the BIND software and I should be keeping up with current releases for just this kind of reason.

So, the reason that the problem appeared suddenly one day is that a piece of spam caused my name server to contact the misconfigured myfamily.com name server and thereby become poisoned. Once poisoned, the name server will behave improperly until it is restarted or the cache flushed somehow. The reason that the problem kept coming back shortly after the name server was restarted is that there was an undeliverable mail message to someone @calvin.bellahs.com kicking around my mail server, causing the poisonous name server to be contacted again every hour or so.

Fixes

Of course the administrator of the myfamily.com domain should fix their DNS data and/or server configuration. Until that happens, there are a few workarounds.

Once the problem occurs, it can be cleared up by restarting the local name server. As long as nobody tries to contact myfamily.com or any other domain served by their name server, the problem won’t come back. But as happened in my case, sometimes events outside of your control, like a stray spam message, can cause a poisionous domain to be queried.

To avoid the chance of a stray email or web link sending your name server off into poisonous territory, you can block access to the poisonous name server at your firewall. This is what I did as an interim protection, although of course it only protects against myfamily.com, not other negligently managed domains.

I suspected that updating to newer, more secure DNS software on my name server would eliminate the vulnerability to being poisoned by legitimate name servers; and in fact testing showed that the vulnerability was gone after I upgraded to BIND 9.

Update – August 2009

More than 4 years later, myfamily.com’s name server is still serving incorrect data. In the mean time, several people have contacted me saying they have been affected by this problem, and this article is the only explanation they have found. In most cases, they have been customers or guest users of a shared network such as a wifi access point, and are at the mercy of the network operator’s DNS implementation.

References

Tags: , , , ,

One Response to “The myfamily.com DNS Poisoning Problem Understood”

  1. Dan says:

    Thanks for the info. I used to use mf.com but stuff like this (ineptitude) motivated me to create an improved tool to help me keep in touch with my family. Check it out at spokt.com – we don’t poison dns :)