|
NeuStar’s UltraDNS faced attack on two fronts on Tuesday, March 31. One of the attacks was technical—a massive denial-of-service attack. The second was a rather surprising opening strike from competitor Dynamic Network Services (DynDNS), which launched a full-scale (and in T1R‘s opinion, misguided) public relations broadside.
First, to the actual denial of service attack. Contrary to many early reports, UltraDNS was not ‘down’ on Tuesday—instead, it suffered partial outages in specific geographies for a subset of its DNS hosting business. Because of their use of IP Anycast techniques, denial of service impacts tend to be significantly more local—it’s tough to bring their entire infrastructure crashing down. Also wrong in many early reports: the Conficker worm, seen as a major threat by security experts, was not involved—this was an entirely separate attacker. It also appears that NeuStar wasn’t actually the target of the attack—rather, it was collateral damage, as the actual target was a small group of NeuStar’s customers, probably the victims of a highly sophisticated extortion attempt.
Interestingly, the actual attack was a work of art—NeuStar was hit by a huge volume of completely legitimate-looking DNS queries, which all appeared to come from legitimate DNS servers, all asking for data on the true attack targets. NeuStar couldn’t block the apparent source without causing an entirely different sort of outage. Accordingly, it took NeuStar’s staff a few hours to identify unique signatures in the attacking queries and block them. This speaks of a highly sophisticated attacker, motivated by significant financial incentives—likely organized crime.
What of the other attack? At the same time this outage was occurring, Dynamic Network Services, a much smaller challenger to NeuStar, decided to blog and twitter about the attack. It’s always a challenge when one sees a competitor having an outage—sometimes the temptation to take advantage of the situation is simply too great to ignore. The question is, as always, how far do you take it? There are accusations that DynDNS took things a bit too far. T1R has seen what has and hasn’t worked across the hosting industry and offers these suggestions to those whose competitors are having an outage:
In this case, DynDNS may just have put a target on its collective back—it had largely been under Ultra’s radar before, a situation that is certainly over now. Furthermore, its infrastructure is actually quite a bit less rugged when it comes to these sorts of attacks—it just isn’t big enough yet to attract one.
One other group that deserves a bit of a drubbing here is the tech press, from bloggers to more established outfits, who relied on DynDNS’s blog entry to say that Ultra ‘was down,’ when in fact it was actually a reasonably small outage. This was easily testable through technical means—if the bloggers and tech press don’t know how to perform those sorts of analyses, they shouldn’t write about the subject.
T1R Take
DNS hosting is a tricky business because the DNS protocol, not to put too fine a point on it, stinks. The best bet for assurance against future attacks is more robust infrastructure—the only question is, can firms like NeuStar stay ahead of the bad guys? We had better hope so, because the Internet doesn’t exist as a useful medium without DNS. Carriers and service providers can help by becoming part of NeuStar’s Cache Defender program.
Sponsored byVerisign
Sponsored byCSC
Sponsored byWhoisXML API
Sponsored byVerisign
Sponsored byIPv4.Global
Sponsored byRadix
Sponsored byDNIB.com
Hi Daniel,
I could not resist replying to this post because it is seemingly so filled with hot air. Perhaps I missed some fundamental fact in my reading, but what leads you to the conclusion that this was an attempted extortion by organized crime?
The fact the queries looked legitimate certainly doesn’t lead me to believe the people responsible were anything like a highly sophisticated attacker, motivated by significant financial incentives—likely organized crime. More likely, someone who read up on the basics of DNS and perhaps a text file or two from Packet Storm. This technical excellence leads me more to the belief that the attacker was an upset nerd recently banned from IRC, than organized crime, who tend to be at least a generation behind in technology.
David,
While I hesitate to respond to a comment that begins with “lolwut”, I will enter the fray regardless. The involvement of organized crime in the botnet business is well known and thoroughly documented. I suggest you do some research. IRC is indeed involved - its usually the control channel for the botnets.
The actual attack itself was somewhat more sophisticated that I can easily describe in a research note meant for a general audience. There is also the matter of the targets - there were specific targets, of a profile likely meant for extortion (my apologies, I am under NDA as to certain details). While this attack could have been executed by a single person, acting alone, most botnets are constructed and then sold by teams, and then passed on to other teams who separate the targets from their money. Again, lots of written research on this.
Neustar is usually pretty open about this sort of thing. I expect we’ll see a presentation on the details of the attack at the next NANOG or major security conference.
Hi Daniel,
Thanks for the clarification; it was unclear your assumptions were based on unpublished information.
David.
One of the things I absolutely love about the DNS industry are my competitors, because for the most part I’ve never been in competition with a nicer, genuinely helpful and clueful bunch. Yes, Steve at DNSmadeeasy and David at EveryDNS, NO-IP, DTdns, (and formerly Erik over Zoneedit) and with apologies to anybody I missed. You all rock.
We’ve had two outages precipitated by DOS attacks over our 11 years in business. During those outages, (and the myriad other attacks that we’ve weathered), all of the aforementioned competitors were on the phone with us, trading emails with us, exchanging data, packet captures, and offering help. Because as we all know, DOS attacks suck, and when you’re on the receiving end of one, you can feel pretty alone.
So when your competitors pick up the phone and send you email, connect you with security researchers, law enforcement and work the back channels for you, it makes an impression and you remember it, always.
Then you also remember the exceptions. Most notably UltraDNS. During those dark periods in our history ultraDNS had no hesitations about cold-calling our customers and telling them we were down hard and that they better switch over them before we went out of business.
That sounds to me like you’re saying that it’s ok when UltraDNS does this (since they do it all the time), but it’s not ok when somebody else blogs or twitters about a DOS attack on UltraDNS. You criticize the bloggers for embellishing what you call “a reasonably small outage”. Well guess what? Both of the outages in our history were also “reasonably small” but you can bet that wasn’t how the UltraDNS sales drones were describing it to our customers.
If Ultra made a habit to extend a hand out to the other DNS providers when they were getting hit instead of trying to feast on their carcasses this might not have happened. I don’t know why the DynDNS guys did what they did on this one, but I wouldn’t be surprised at all if they were settling an old grudge for getting exactly this sort of treatment in the past during a DOS attack of their own.
So, while your article is very sympathetic to CircleID sponsors UltraDNS, just keep in mind that from where some of us sit, what goes around comes around.
Quick clarification here: While CircleID uses UltraDNS for its DNS management and has featured Dynamic Network Services on DNS related postings, neither companies are sponsors of this site. All parties are equally welcome to participate on CircleID.
“There is nothing wrong with having your sales force call into the competitor’s customers, but be aware that it may look predatory.”
That sounds to me like you’re saying that it’s ok when UltraDNS does this (since they do it all the time), but it’s not ok when somebody else blogs or twitters about a DOS attack on UltraDNS. You criticize the bloggers for embellishing what you call “a reasonably small outage”. Well guess what? Both of the outages in our history were also “reasonably small” but you can bet that wasn’t how the UltraDNS sales drones were describing it to our customers.
Calling into your competitors is an obvious fact of life, all across the hosting industry, DNS or not. This is not a gentleman’s game, its business. Salespeople will call into competitor’s customers - if you don’t like that, you may in the wrong line of work. The situation with bloggers and the tech press is different. I don’t assume that what a sales person is telling me is correct - in fact, I usually assume a significant degree of marketing hyperbole. The assumption, however, is that bloggers and the tech press are accurate on issues like this. Sales people have a motive to distort. The tech press has no motive - its simply cluelessness, not malice.
Please note, the three points I made concerning what do to during a customer outage were generalized to the entire hosting industry and were not meant to comment on the behavior of any one provider - which is what I said. “That sounds to me like you’re saying…” is much more about your feelings (obviously strong) than mine.
I have no idea if Neustar is a CircleID sponsor. I don’t work for CIrcleID and I don’t work for Neustar. This was originally a private research note to my customers that someone decided to post publicly, which is fine.
As far as Ultra reaching out or not - While their sales people are certainly aggressive, I’m also aware of cases where Ultra’s technical staff has assisted other providers - Rodney Joffe, in particular. The two issues are quite separate.