Forgot your password?
typodupeerror
Networking Security Technology

Misconfigured Open DNS Resolvers Key To Massive DDoS Attacks 179

Posted by Unknown Lamer
from the check-your-sources dept.
msm1267 writes with an excerpt From Threat Post: "While the big traffic numbers and the spat between Spamhaus and illicit webhost Cyberbunker are grabbing big headlines, the underlying and percolating issue at play here has to do with the open DNS resolvers being used to DDoS the spam-fighters from Switzerland. Open resolvers do not authenticate a packet-sender's IP address before a DNS reply is sent back. Therefore, an attacker that is able to spoof a victim's IP address can have a DNS request bombard the victim with a 100-to-1 ratio of traffic coming back to them versus what was requested. DNS amplification attacks such as these have been used lately by hacktivists, extortionists and blacklisted webhosts to great success." Running an open DNS resolver isn't itself always a problem, but it looks like people are enabling neither source address verification nor rate limiting.
This discussion has been archived. No new comments can be posted.

Misconfigured Open DNS Resolvers Key To Massive DDoS Attacks

Comments Filter:
  • First post (Score:5, Funny)

    by Anonymous Coward on Thursday March 28, 2013 @04:32PM (#43306485)

    In before the fight between those two guys and their walls of text...

    • by noh8rz10 (2716597) on Thursday March 28, 2013 @05:09PM (#43306801)
      maybe if the routers had been configured with HOSTS FILES then all of this could have been avoided...
      • You are IGNORANT and EVIL for ignore Earth 4 day simultaneous Time Cube rotation and preaching Academic Religious Singularity... wait.

        Bloody hell, I'm on the wrong thread.
    • Re: (Score:2, Offtopic)

      by Tynin (634655)

      In before the fight between those two guys and their walls of text...

      I've begun to think it is actually just one guy just trolling (poorly) for all they are worth. Either that or it has turned into a meme that encourages the the likes of 4chan /b/tards to, in their own way, declare I am Spartacus(APK), just for the lolz...

      • by gl4ss (559668)

        it's a meme.
        the sign of a meme is that discussion about it fills first few posts even if the actual subject of the meme is nowhere to be seen.
        and spamhauscraft confirms: gnaa is dead.

        but why is such source address spoofing still such a problem? ironically this ties in to dozens of post-packages signed apk.

        and the hosts file approach isn't that bad way to filter bunch of ads. it works...

      • by Githaron (2462596)

        ... Either that or it has turned into a meme that encourages the the likes of 4chan /b/tards to, in their own way, declare I am Spartacus(APK), just for the lolz...

        More like Spamtacus.

  • by six025 (714064) on Thursday March 28, 2013 @04:34PM (#43306509)

    Running an open DNS resolver isn't itself always a problem, but it looks like people are enabling neither source address verification nor rate limiting.

    One has to wonder if this is caused by negligence, or if it's more a case of "oopsie, we left this door open, oh well" - which would be a great way to set up nodes around the 'net specifically to allow these types of attacks to occur.

    Not saying that is right or wrong - asking a genuine question.

    Peace,
    Andy.

    • by h4rr4r (612664)

      It might also be an old DNS server no one is using or remembers they have up.

      • by RockDoctor (15477)

        It might also be an old DNS server no one is using or remembers they have up.

        Which is just a different form of administrative incompetence.

        Proposals have been forwarded to use open DNS resolvers to DDoS open DNS resolvers to force their admins to fix their problems. Which is probably not going to help much where the actual problem is simple ignorance. If the person doing the "administration" of a system doesn't even know what DNS resolution is, then blowing them away for running an open resolver isn't actu

    • by Anonymous Coward on Thursday March 28, 2013 @04:52PM (#43306661)

      Isn't the real problem the originating ISPs for allowing spoofed packets to be sent in the first place? Is it really correct to be blaming the DNS resolver that it's responding to packets it has no way to authenticate? If the original ISP dropped a packet it shouldn't be routing, the whole problem would go awa.

      • by Anonymous Coward on Thursday March 28, 2013 @05:08PM (#43306781)

        Yep. Had BCP38 (Best Current Practice No. 38) [ietf.org] been in effect at those ISP's, this attack would not have occurred.

    • That would seem to be a matter of what the default configuration is. Do these DNS servers have these protections enabled by default, and are then disabled? Is it that they left it off by default on older versions? Do they still leave it off by default?
    • Running an open DNS resolver isn't itself always a problem, but it looks like people are enabling neither source address verification nor rate limiting.

      One has to wonder if this is caused by negligence, ...

      Also, one has to wonder if it's negligence by the person installing the resolver, or by the person distributing the resolver.

      What are the default values for source address verification and rate limiting? If having them both disabled is a problem, at least ONE of them should be on by default, requiring it

    • by Yaa 101 (664725)

      This has to do with not good thought out default settings of the various DNS servers out there, but also has to do with people running too old DNS servers.
      You can restrict recursing resolve based on source IP, groups of IP or subnets.
      First I do is define an internal mode and an external mode where external mode only sees an authoritive server and the internal mode can do recursed resolving.

      • by sjames (1099)

        Many deliberately don't restrict recursive lookup as a service to anyone who needs it. The fault lies with ISPs that allow their customers to source packets with addresses they haven't been assigned.

    • by jon3k (691256)
      It's the default BIND configuration, unless they've changed it recently. If you take a (at least CentOS/RHEL) linux machine, install BIND and turn it on, you're now an open recursive resolver. I don't know if it's the distro's default bind configuration or if it's the one that ships from ISC when the package maintainers built the RPM, or what. It's your responsibility to write a bind ACL to restrict recursive resolution to particular subnets or secure it in some other way (rate limiting, etc).
    • DNS used to not be a threat; that's been changing. Rate limiting wasn't an issue. Source address verification was a problem for ISP routers (to prevent address spoofing), but it wasn't a problem for recursive DNS servers (who were willing to serve anybody, not just their own customers), and it especially wasn't a problem for authoritative DNS servers, because they're supposed to tell anybody the address for www.yourdomain.com, and aren't in the right part of the network to verify whether a UDP DNS request

  • by h4rr4r (612664) on Thursday March 28, 2013 @04:35PM (#43306517)

    Why are they not sending out emails to the people running these things.

    Check which domains these servers are authoritative for and send them a damn email.

    • Why are they not sending out emails to the people running these things.

      Check which domains these servers are authoritative for and send them a damn email.

      I agree, something proactive needs to be done. The question I have is: whose job is it to do something proactive in these instances? Does anyone do these sorts of things?

      According to the article, there are ~27 million open DNS resolvers. That might take some time. I suppose it could be automated, though, with a "Dear [admin and/or technical contact], your DNS located at [ip address] is breaking the internet. Love, Some people you have never heard of. Click here for more information." I wonder how many of th

      • by DrProton (79239)

        That might take some time. I suppose it could be automated, though, with a "Dear [admin and/or technical contact],

        Are you joking? What if they don't read English? The most open-server dense netblock is .tw. Pakistan is also high on the list of offender IPs.

    • by pavon (30274)

      There are over 25 million known open DNS resolvers [openresolverproject.org] that can be used in DNS amplification attacks. Directly contacting the administrators of all the servers used in the attack is not a tractable problem. The same is true of pretty much any DDOS attach vector; there are too many broken machines to deal with them directly.

    • by LordLimecat (1103839) on Thursday March 28, 2013 @06:06PM (#43307245)

      Because the DNS servers are doing nothing wrong.

      The problem is that people can spoof source addresses (because ISPs arent stopping it). Fix this issue, and youll still have to worry about any of a million other scenarios where a small request gets a lot of data back.

      All you have to do is make sure source addresses are filtered when they hit the ISP, and the huge majority of these issues (as well as being able to cloak where an attack came from) go away.

      • by Shoten (260439) on Thursday March 28, 2013 @06:38PM (#43307469)

        Because the DNS servers are doing nothing wrong.

        The problem is that people can spoof source addresses (because ISPs arent stopping it). Fix this issue, and youll still have to worry about any of a million other scenarios where a small request gets a lot of data back.

        All you have to do is make sure source addresses are filtered when they hit the ISP, and the huge majority of these issues (as well as being able to cloak where an attack came from) go away.

        Actually, they are. The feature being leveraged here is that the servers are performing recursive lookups for domains that they do not control for the open Internet; BIND turns this off, by default, starting with version 9.4. The problem is that a lot of 9.3.X and older DNS servers are still out there, as well as a lot of bad network architecture jobs. The servers should only handle recursion for IP addresses that are on the inside. And as for the spoofing? Well, ingress filtering is trivial to do at the border. And these two things in concert shut this problem down entirely.

        • by sjames (1099)

          Actually, they are not, or this wouldn't be happening. The attack involves sending DNS requests to an open resolver with your target's source address. If your ISP was doing the right thing, those packets would never get to the open net at all.

          I know that any time I've set up a dual homed network, the upstream providers have wanted evidence that I had a right to use the addresses I told them I would be originating. Until I provided that and a complete list of those addresses, the packets would not go out. Th

        • by Maow (620678)

          Because the DNS servers are doing nothing wrong.

          The problem is that people can spoof source addresses (because ISPs arent stopping it). Fix this issue, and youll still have to worry about any of a million other scenarios where a small request gets a lot of data back.

          All you have to do is make sure source addresses are filtered when they hit the ISP, and the huge majority of these issues (as well as being able to cloak where an attack came from) go away.

          Actually, they are. The feature being leveraged here is that the servers are performing recursive lookups for domains that they do not control for the open Internet; BIND turns this off, by default, starting with version 9.4. The problem is that a lot of 9.3.X and older DNS servers are still out there, as well as a lot of bad network architecture jobs. The servers should only handle recursion for IP addresses that are on the inside. And as for the spoofing? Well, ingress filtering is trivial to do at the border. And these two things in concert shut this problem down entirely.

          If my DNS is doing recursive lookups on spoofed packets, and I turned off recursion, wouldn't the attackers then send spoofed packets to, say, Google's DNS, causing the same havok?

          Sure it would be easier for Google to implement rate limiting through IP tables or whatever, but they wouldn't be able to recognise spoofed packets either.

          So I don't see how recursion is the issue versus ISPs blocking spoofed packets leaving their own networks.

          Am I wrong here, and if so, how?

      • by Tom (822)

        The problem is that people can spoof source addresses (because ISPs arent stopping it).

        Bingo.

        I wrote a paper about a similar attack (called a reflection DDoS attack back in the days) in 2002. Mine didn't use DNS, but a different service, but same principle - massive amplification.

        I'm honestly surprised that IP spoofing is still possible today. Have ISPs been asleep for the past decade? The number of legitimate use cases for source spoofing across a WAN (there are a number in LAN) can probably be enumerated in a very short list. I personally can think of... one: Testing if your ISP is stupid.

    • by Shoten (260439)

      Trust me...it's not that simple. For one, there usually isn't an email to send to...there may be one listed somewhere, but it may not be real or nobody may read it. But on top of that, who is this "they" you are referring to...Spamhaus? So, Spamhaus should do a lookup on every single DNS server that is hammering them during the largest DDoS in history, find the abuse email address for each of them, and send them an email? All while getting hit with the biggest DDoS ever?

      To get a sense of why abuse email

    • Why are they not sending out emails to the people running these things.

      Check which domains these servers are authoritative for and send them a damn email.

      Cause "fixing" them solves nothing?

    • by jon3k (691256)
      Well, there's a couple million of them. It's really an uphill battle. For every one you shut down today another will popup tomorrow. Plus this is just one attack vector. What we need to do is get ISPs to start filtering egress traffic and we can solve an entire range of these attacks by being able to track down the source.
  • Article is garbage (Score:5, Insightful)

    by Anonymous Coward on Thursday March 28, 2013 @04:40PM (#43306559)

    It claims that the problem is DNS resolvers that don't authenticate the sender's IP address using BCP38 [ietf.org]. It is comparing chalk and cheese. Filtering out spoofed IP addresses is something that needs to happen at the edge of the network. It's not something that a single server on the network can do.

    • by asc4 (413110)

      This. Rate-limiting can help...I'm quite sure Google has rate-limiting in place on 8.8.8.8 for instance. But for those who don't have Google's budget, it's a challenge. There are not currently any sufficiently tested and stable DNS rate-limiting features in any of the top 3 resolvers out there. The problem here is networks letting spoofed packets out of their nets, not DNS servers performing correctly.

      • by mukund (163654)

        There is a rate limiting patch for BIND. The BIND package in RHEL/CentOS has it now:
        https://bugzilla.redhat.com/show_bug.cgi?id=873624 [redhat.com]

        A person has also attached a graph showing its effect on that bug page. I know of some other places which are using this patch.

        (Disclaimer: I work for the organization that produces BIND.)

        • There is a rate limiting patch for BIND. The BIND package in RHEL/CentOS has it now:

          If you really want to solve the problem then implement DNS cookies.

          Please think about what your doing. Rate limiting solves nothing. Once this is deployed the attacker will simply alter the contents of their queries rather than resending the same tired request such that it now becomes indistinguishable from background with much the same results. At that point any additional rate limiting hurts legitimate users just as much as attackers.

          This is your typical spam fighting downward spiral. You see a proble

  • Hoax? (Score:5, Interesting)

    by Ubi_NL (313657) <joris@@@ideeel...nl> on Thursday March 28, 2013 @04:43PM (#43306581) Journal

    I know Its not the primary topic here,, but gizmodo has some evidence that the whole cyberbunker thing is a fake

    http://gizmodo.com/5992652/that-internet-war-apocalypse-is-a-lie [gizmodo.com]

    • by Anonymous Coward

      I know Its not the primary topic here,, but gizmodo has some evidence that the whole cyberbunker thing is a fake

      http://gizmodo.com/5992652/that-internet-war-apocalypse-is-a-lie [gizmodo.com]

      WHOA WHOA WHOA. No.

      This 'thing' is definitely real. It's happening. That's not in question. What ISN'T real, however, is CloudFlare's assertion that it's "jamming crucial infrastructure around the world".

      • by gl4ss (559668)

        dunno, it is real if you define spamhaus as crucial infrastructure.

        it isn't though..

  • Maybe this is over my head. But how would one rung a "safe" DNS server then? My interpretation of the article basically says to let only specific people use your DNS server, but then how would a company run a public resolver?

    For example, Google runs open public name servers on 8.8.8.8 and 8.8.4.4, same with OpenDNS, and many, many more. What is to stop those servers from being used in this sort of attack? Is this article really advocating a situation where you MUST use only your own ISP's resolver and trust

    • by nairnr (314138)

      Maybe this is over my head. But how would one rung a "safe" DNS server then? My interpretation of the article basically says to let only specific people use your DNS server, but then how would a company run a public resolver?

      For example, Google runs open public name servers on 8.8.8.8 and 8.8.4.4, same with OpenDNS, and many, many more. What is to stop those servers from being used in this sort of attack? Is this article really advocating a situation where you MUST use only your own ISP's resolver and trust them not to do what so many of them consistently do and mess with the results?

      Or am I completely missing the point to this article?

      Two different things. If you are running a DNS server yourself, for your own domain then you should only respond to requests for your domain from the outside. IE - Non-recursive. The only answers you serve are for those queries you are authoritative for. You only accept recursive queries from inside your own network. Those are the recursive ones.

      Public servers would use rate-limiting to to protect against being effective in spoofed attacks.

      • by t4ng* (1092951)

        Two other different things...

        1) ISPs could drop out-going tcp and udp packets on port 53 from all their IP address except their own DNS servers. That would stop their customers from using public DNS server outside their networks. But it would also stop this kind of attack.

        2) Drop all outgoing traffic that has a spoofed source IP address. This is a very simple bit mask operation. Yes, it requires more compute power than not doing it, but not very much. The ISPs know what IP addresses they own, they can

        • #2 is the right answer, be responsible for the traffic on your network.

          #1 is the wrong answer. Too many ISPs fuck with DNS by returning IP addresses to advertizing domains instead of NXDOMAIN.

        • by green1 (322787)

          1) would be REALLY bad, and I hate anyone who would even consider such a solution.

          2) I can't imagine why every ISP and transit provider doesn't already do this. This has been a known problem for over a decade, deal with it already!

          • by Trolan (42526)

            Actually, transit providers are one of the groups that can't reliably apply BCP38 or RPF. BCP38 and RPF is very easily applied at the edge, where you know specifically the IPs involved, since they're either connected or statically routed. Now, when you get into things over BGP, it gets dicey. You may see traffic over a BGP-managed link from an IP that isn't involved in the received prefixes, but yet still belong to the specific peer. Is this an error? No. Is dropping the bits on the floor because you'

        • Two other different things...

          1) ISPs could drop out-going tcp and udp packets on port 53 from all their IP address except their own DNS servers. That would stop their customers from using public DNS server outside their networks. But it would also stop this kind of attack.

          It would also have a high collateral cost: diagnosing many DNS issues becomes impossible when you can only work with one recursive resolver (which may be what is causing the DNS issues!) It is necessary to access legitimate open resolvers and authoritative servers on any kind of Internet connection, even residential broadband (don't think of grandma but think of the tech helping grandma).

          In short, we *need* TCP and UDP port 53 traffic unfiltered.

          2) Drop all outgoing traffic that has a spoofed source IP address. This is a very simple bit mask operation. Yes, it requires more compute power than not doing it, but not very much. The ISPs know what IP addresses they own, they can very easily prevent spoofed traffic from leaving their networks, effectively stopping this kind of attack, as well as other types of hacking. At the same time, it would still allow legitimate use of public DNS servers.

          This is what we need more of. Provided, of course, that it isn

      • by mcrbids (148650)

        You simply configure your DNS server properly, including setting the networks it's allowed to resolve for. A nameserver can be both authoritative for certain domains globally, and also be recursive for specific hosts.

        Of course, there's also the problem of DNS amplification using source address spoofing by requesting authoritative DNS records, but simply doing the above greatly mitigates the effectiveness of the attack.

    • Or am I completely missing the point to this article?

      Yes.

      It's talking about spoofed requests - much like if someone sent a request for more information to a Scientology center, and they put your return address on the form. Suddenly you're getting very creepy mail from the Scientologists and you have no idea where it came from. If they do it enough times to enough organizations, and your mailbox is full, and your Netflix Blu-ray of Tootsie is deferred until you can clean out your mailbox.

      • by green1 (322787)

        You're confirming that I understood the article perfectly. The problem is in their choice of solution.

        It seems there are 2 possible solutions.

        1) get ISPs and transit providers to actually start blocking IP spoofing (something they all should have been doing years ago)

        2) break the internet by banning all public resolvers.

        Unfortunately the article seems to me to be advocating for number 2, which would harm many people, and just cause the attackers to continue to use IP spoofing on different services or protoc

    • by gparent (1242548)

      The problem is that almost no one actually needs to run a public resolver.

      Your ISP provides a DNS server to you that is recursive (usually), so they can use ACLs to make sure only their clients are using them.

      Domain owners provide DNS servers that are authoritative, but only for their own domain, so it limits the scope of the problem as well.

      The problem is when domain owners provide DNS servers authoritative for their domain, but -also- allowing other people to use them as public recursive servers. There's

    • Maybe this is over my head. But how would one rung a "safe" DNS server then? My interpretation of the article basically says to let only specific people use your DNS server, but then how would a company run a public resolver?

      The problem is one of degree. The theory is if you don't offer a recursive resolver to the public amount of amplified output you get for your input is diminished over running a resolver that would respond to just whatever your authoratitive for.

      Personally I don't buy much into this theory. There are enough ways to request an earfull from enough properly configured servers we can find much more effective things to be doing with our Internet fixing time.

    • by jon3k (691256)
      You're compounding two things. One is providing RECURSIVE RESOLUTION to clients and the other is providing an AUTHORITATIVE NAMESERVICE for particular zones. I'll explain the two briefly:

      1. Recursive resolver - this is a nameserver that you can ask to resolve any domain name and it'll do just that. If it isn't authoritative for it (meaning you RUN the zone and it's in a config file locally) it will go through the recursive resolution process for you. It will look at it's root hints, talk to the root
      • by green1 (322787)

        Actaully, I wasn't confusing the 2 at all. But I don't think that we should be banning public resolvers and forcing people to use the resolver operated by their ISP when those resolvers have frequently been messed with for profit by the companies running them. The article would propose killing OpenDNS, Google DNS, and many, many more.

        The correct solution is not to break the internet by banning a harmless and very useful feature, but to fix the internet by blocking IP spoofing. Why on earth are there any ISP

        • by jon3k (691256)
          If you read my later posts in this thread, you'll see that I agree that source address filtering should be our #1 concern.

          With that said, there are ways to provide recursive DNS resolution to clients openly, but without being used as an attack vector. Specifically by rate limiting the requests.

          The reason ISPs are allowing spoofed traffic is simply one of two things: ignorance or laziness. It's very easy to write an ACL that mirrors your BGP announcements then apply that as an outbound filter at your
  • than the users that get their computers infected with botnets and spew spam. These people are supposed to know what they're doing.

    Take away their Geek Card, and then suspend their internet license ;)

  • The ARegister also has coverage of this story...

    http://www.theregister.co.uk/2013/03/28/spamhaus_mega_ddos_little_collateral_damage/ [theregister.co.uk]

  • The kind of traffic it generated could practically disconnect entire countries from internet, and is still open to whatever with the right resources to use it, What kind of measures can be taken to prevent it? To have as DNS mirrors several with really big bandwidth?
  • Let's say I want to run a public DNS recursive server, that is, I want to allow anyone to issue a handful of queries for any arbitrary DNS records, and in addition to just serving up my own, I want to also service requests for whatever arbitrary thing they requested. Is there an easy way to rate limit these queries based on source IP address, to prevent abuse of this service I've chosen to offer?

    How should one set that up?

  • by WaffleMonster (969671) on Friday March 29, 2013 @12:34AM (#43309377)

    For sake of argument assume you are able to snap your fingers and miraculously all open resolvers have been locked down. What has been accomplished?

    Will anyone still be able to issue legitimate DNS queries using forged source address with impunity for which response is several times larger than request? YES.

    Will DNSSEC with egregiously enormous amplification when configured entire as recommended simply go away? A man can dream. I doubt this will come to fruition.

    The way I see it there are two solutions to this problem. BOTH need to be implemented.

    1. Ingres filtering (AKA tools.ietf.org/html/bcp38) as TFA and many others here point out needs to be implemented with enough specificity to meaningfully raise the bar for successful source address spoofing.

    2. All UDP protocols allowing amplification or resource exhaustion from spoofed source addresses need to be beaten with a clue stick for making the Internet worse than need be. There is NO EXCUSE.

    It does not need to be perfect it only needs to not suck more than the underlying network.

    We know how to do this. There are production protocols which get it right. The answer is stateless cookies. It might require an extra round trip once in a blue moon or a few extra CPU cycles to calculate HMACs... we can easily afford it.

    In return we get UDP protocols at least as trustworthy as underlying transport. Protocols which can no longer be turned into weapons of mass deluge.

    For DNS we have had reasonable solutions for years...yet we sit on our hands and nothing gets done...
    http://tools.ietf.org/html/draft-eastlake-dnsext-cookies-03 [ietf.org]

    This can easily be phased in conjunction with DNS query rate limiting applicable for requests without cookies.

    It seems to me all the money and political interest follow fools errands like DNSSEC which paradoxically makes the Internet we actually have right now less safe from denial of service.

The bomb will never go off. I speak as an expert in explosives. -- Admiral William Leahy, U.S. Atomic Bomb Project

Working...