|
This post follows an earlier post about DNS amplification attacks being observed around the world. DNS Amplification Attacks are occurring regularly and even though they aren’t generating headlines targets have to deal with floods of traffic and ISP infrastructure is needlessly stressed—load balancers fail, network links get saturated, and servers get overloaded. And far more intense attacks can be launched at any time.
The latest twist on amplification attacks is to induct home gateways, tens of millions of them are scattered throughout the world and they can act as willing accomplice as shown in the diagram below. Attackers simply send packets with spoofed source addresses to gateways, which proxy them to the resolver they’re configured to use, typically an ISPs resolver. When ISP resolvers see DNS queries coming from IP addresses within their network they rightfully prepare responses and send them back to the home gateway. From there the home gateway forwards the response to the target using the spoofed source address. It’s a pretty neat trick, except its causing lots of problems for ISPs including those who go to great lengths to protect their networks.
In this case Best Practices like Source Address Validation/ingress filtering (BCP 38) and “closing” resolvers (restricting resolver access to IP ranges within a providers network) don’t work. The open DNS proxies on home gateways subvert resolver protections and since the attack packets are sourced from outside the network internal address spoofing protections aren’t effective. This presents a major problem for service providers: how to mitigate the attacks?
“Upgrade home gateways!” is a seemingly obvious response but even if patches can be developed to resolve the issue getting consumers to upgrade software on a device many probably forgot about shortly after they installed it is an unlikely proposition. Recall last year despite massive (and very costly) communications about DNSChanger and the prospect of losing Internet connectivity altogether many consumers were unwilling or unable to take action. In this case consumers don’t really suffer any inconvenience at all; most will never notice their home gateway is being used for an attack so there’s essentially no motivation to take action. Other approaches are needed.
Resource Rate Limiting (RRL) was devised to address DNS amplification but is currently only implemented on authoritative DNS servers so it doesn’t help with attacks that use resolvers. What’s needed is more intelligence in the resolver. The previous post in this series talked about the importance of logging DNS data. Logs will reveal critical details necessary to identifying and characterizing an attack: unusual spikes in the query rate (QPS) or unusual query activity—uncommon domain names among the top domains being queried, atypical query types (like ANY), or an uptick in requests for DNSSEC. Some kind of operational dashboard fed by DNS data is useful to track baseline (“normal”) operating conditions and to provide the basis for comparison when an attack is suspected.
Depending on the sophistication of the attack simple forensics can reveal the source, although it may also be necessary to drill down and visualize the data—by name, query type, etc. to investigate more deeply. No matter what having access to the data is critical, it should become a best practice to capture it all the time. In fact when data is logged as part of standard operating procedure proactive monitoring becomes possible. Setting alerts based on typical operating thresholds plus a variance factor can provide valuable lead time.
Once an attack has been identified it’s necessary to filter (drop) incoming queries to mitigate it. Filtering incoming traffic (versus filtering responses) is advantageous because the server has to do less work (it doesn’t have to look up the answer first). For simple attacks it might only be necessary to filter based on Query Type. For instance, ANY queries have few legitimate uses and it might be appropriate to deal with large spikes with a simple filtering policy. Since DNS is an essential network service it’s also important to ensure legitimate queries are always answered. Finer grained filters can be used for better targeting to eliminate collateral damage. For instance a slightly more sophisticated policy can filter based on Query Type and name. Tighter specification of drop criteria minimizes the possibility valid queries are impacted.
Amplification attacks will continue to evolve so methods for identifying and mitigating them will have to evolve too. The next post will discuss some of the latest insights from the wild.
Join Nominum for a 30 minute webinar on this topic on September 24. Sign up here.
Sponsored byWhoisXML API
Sponsored byIPv4.Global
Sponsored byVerisign
Sponsored byCSC
Sponsored byDNIB.com
Sponsored byRadix
Sponsored byVerisign
Most home routers seem to have firmware based on DD-WRT, so they already have firewall capability in place. I have a standard configuration that most routers also follow: the DNS server listens on port 53 and originates queries from random unprivileged ports. That allows for a simple firewall rule: block all packets with a destination port of 53 arriving on the WAN interface, since such packets aren’t legal (we’re not a DNS server for the outside world, and any responses to our legitimate queries will be to the random unprivileged port we used to send the query). That can be done with a simple configuration-file change to add a static interface-based firewall rule, no major firmware upgrade required for the majority of routers.
My standard firewall rules don’t even require that much. The default rule is to drop all packets destined for privileged ports arriving on the WAN interface, I’m not running servers so there should never be any such traffic. Then I make exceptions for specific ports as required.
There are several hundred million home routers out there.. It's a bit of a stretch to say that MOST routers have firmware based on DD-WRT. Have a look at the Open resolvers project. It lists the number of open resolvers per ISP (by ASN#). http://dns.measurement-factory.com/surveys/openresolvers/ASN-reports/latest.html There has recently been an effort to clamp down on these open resolvers yet some ISP's have hundreds or thousands of these open resolver. You can punch in some IP and check for open resolvers. Most likely many of those listed are mis-configured home routers that are listening and responding to DNS queries on the WAN interface. So clearly its an issue that was brought to the center with the recent attacks. Implementing BCP-38 (ingress filtering) will prevent these DNS queries with the spoofed source address of the victim in DDOS attacks from leaving the ISP's network. Yet many ISP's have not implemented this yet.
I was quite surprised to find open resolvers just within the /24 I share with other customers of my ISP. I even wrote about it here.
I was responsible for the development of a DNS monitoring product for my previous employer, as I could see the value that monitoring query traffic and certain query types could provide. I know of several large companies in the UK that are using this now and find it very valuable.
I note that some of the commercial vendors are coming out with monitoring/reporting servers that do the same thing, so it does appear that monitoring/reporting and alerting is becoming a necessity rather than an option.
There is still a piece missing though and that is automated mitigation. Before I left my previous employer we were discussing ways to do this, i.e. in the event of a query storm could we manipulate iptables to temporarily drop the traffic from particular sources before it even reached the DNS server, or find some other way to dynamically drop the bad query traffic? Unfortunately I left before we really got to grips with this piece of the puzzle, but I’m sure it’s something you are going to address in your next post! :-)