Eircom have been having problems with internet connectivity. It’s hard to get information about exactly what they’re seeing, but there seem to be 2 aspects to it:
- Eircom are getting hit with a lot of packets
- Customers have sometimes been directed to strange sites by Eircom’s DNS servers
Justin Mason has a good overview of the news coverage. There some points of his worth correcting though:
- The Kaminsky attack takes less than seconds against servers which do not randomise the QID.
- No DNS server is immune to being poisoned by spoofed QID – even with randomised QIDs, DNS servers can still be poisoned in just 10 hours, at GigE rates of spoofing.
I.e. DDoS levels of incoming DNS packets are consistent with a poisoning attack on up-to-date DNS servers, which randomise QID.
The moral of the story here is that using recursive, caching DNS servers that are shared on any significant scale, like ISP nameservers or (even worse) OpenDNS, is just unhygienic. They’re just fundamentally flawed in todays internet environment, as they’re juicy targets for poisoning, until DNSSec is widely deployed. When finally DNSSec is deployed, shared, recursive nameservers remain a bad idea as they terminate the chain of the trust – the connection between the NS and client can still be spoofed.
In short:
- Technical users and systems admins should install local, recursive nameservers. Preferably on a per-system basis.
- Operating system vendors should provide easily-installed recursive nameservers and should consider installing and configuring their use by default. (Fedora provides a convenient ‘caching-nameserver’ package, and also a new dnssec-conf package with F11, though not enabled by default).
- Consumer router vendors already ship with recursive servers, but tend to forward queries to the ISP – they should stop doing that and just provide full recursive service (hosts already do caching of lookup results anyway).
Widely shared, recursive nameservers are a major security risk and really should be considered an anachronism. It’s time to start gettting rid of them…
Paul Jakma's Blog 
Niall said
How well do you reckon the root servers would hold up to everyone having their own local recursive servers?
David Conrad said
“Crunch all you want, we’ll make more.”
Or, you could mirror the root zone in your local resolver.
pjakma said
They’d hold up just fine, I’m pretty certain.
Stéphane Bortzmeyer said
Yes, this approach seems interesting. However, there is a big risk that many ISP will sooner or later block port 53 oubound (to force their consumers to go through their lying DNS resolvers, those which rewrite NXDOMAIN into an ad Web site address). This would be a big threat to your idea.
bill manning said
its not a novel idea. in last years attacks, we ran this little exercise and wrote it up here:
http://www.cisco.com/web/about/ac123/ac147/archived_issues/ipj_12-2/122_dns.html
sure, the ideal is a validating resolver at each end point, but we can’t get there right now.
instead of goofy protocol hacks, its just easier all around to reduce the attack surface.
Frank said
Most ISPs operate their own DNS servers which they hand out via DHCP to their residential subscribers. As has been Eircom’s experience, and many others ISPs, this can be the point of attack by DDoS.
Does this mean ISPs have to start taking a different approach (i.e. advising customers to purchase SOHO routers that have a recursive resolver; distribute caching DNS servers at every node or DSLAM), or is there a way to architect their DNS services such that they are not exposed?
Why not isolate the caching nameservers for customer use from the ISP’s upstreams, with forwarders on their caching nameservers to a distributed network (around the world) of “external” DNS servers that are not (as) vulnerable to a DDoS attack (i.e. like what Akamai apparently does)? That way the “internal” customer-facing nameservers can’t be affected by a DDoS attack, and the external forwarders are too distributed to make a DDoS attack feasible.
Frank
Douglas Otis said
Local recursive name servers within natted corporate networks might be individually less secure than shared recursive name servers having direct access. Protocols like SPF within MUA filters can cause transactions without consuming attacker’s resources (beyond that needed to spam). This approach enables distributed kaminsky style attacks for individual recursive name servers.
EDNS0/DNSSEC creates a significant exposure to DDoS attacks, where again SPF within MUA filters can also instigate targeted attacks.
Running DNS on SCTP offers a complete remedy that can be phased into operation without impacting existing DNS services.
Frank said
I don’t follow your first point. How does receiving spam allow for distributed Kaminsky style attacks? I thought if one was running a patched version of DNS it would take hours at 1 Gbps speed to poison a cache. Unless you have a unique and unannounced exploit in mind, I don’t see how its practical to poison a cache via spam — it would take weeks if not months of spam to generate a volume of DNS requests by the MTA to match what’s required in a Kaminsky attack.
Perhaps you can elaborate.
Frank
pjakma said
I think he means that Dr Evil can get the target NS to query his evil.com nameserver, even if the target NS is not accessible to Dr Evil, so as to observe the QID. Alternatively, he means that Dr. Evil can somehow enlist many unwitting, innocent nameservers to send attack packets – but I don’t see how that could work.
So I’m confused as you are Frank. 🙂
Perhaps he means just a plain DDoS, enlisting unwitting NSes by having them convert and amplify a spam. However, surely that’s already a potential problem today with spam and HTTP DDoS? (I.e. not a huge problem, least so far that I’ve noticed – and not a kaminsky attack).
Frank said
Generally speaking, a “reflective” DDoS attack is most economical (for the attacker) if the packet initiator is smaller in size than the packet directed to the target. To say it in reverse, you want the reflector to amplify the packet. But the problem is that the size of an e-mail message is usually larger than a DNS query.
Frank
pjakma said
There’s some interesting discussion of this on the DNS-Ops list, both on the attack and the above suggestions.
Root operators confirm it would not cause problems
there’s some suggestion that DNSSec in fact can work through recursive nameservers and still allow stub resolvers to confirm the DNSSec chain of trust. As far as I know (and I’ve had a quick read of the RFCs) this would require the stub resolver to be able to query the whole chain of delegations and validate the chain itself. Basically, it’d have to be a fully capable DNSSec-aware, recursive resolver.
The only way to allow DNSSec to work in the context of a limited query-ability stub resolver is to configure transport security (DNS TSIG, IPSec). The chances of this being done are often remote, particularly between ISPs and their customers.