|
||
A few hours ago, Pakistan Telecom (AS 17557) began advertising a small part of YouTube’s (AS 36561) assigned network. This story is almost as old as BGP. Old hands will recognize this as, fundamentally, the same problem as the infamous AS 7007 from 1997, a more recent ConEd mistake of early 2006 and even TTNet’s Christmas Eve gift 2005.
Just before 18:48 UTC, Pakistan Telecom, in response to government order to block access to YouTube (see news item), started advertising a route for 208.65.153.0/24 to its provider, PCCW (AS 3491). For those unfamiliar with BGP, this is a more specific route than the ones used by YouTube (208.65.152.0/22), and therefore most routers would choose to send traffic to Pakistan Telecom for this slice of YouTube’s network.
I became interested in this immediately as I was concerned that I wouldn’t be able to spend my evening watching imbecilic videos of cats doing foolish things (even for a cat). Then, I started to examine our mountains of BGP data and quickly noticed that the correct AS path (“Will the real YouTube please stand up?”) was getting restored to most of our peers.
The data points identified below are culled from over 250 peering sessions with 170 unique ASNs. While it is hard to describe exactly how widely this hijacked prefix was seen, we estimate that it was seen by a bit more than two-thirds of the Internet.
This table shows the timing of the event and how quickly the route propagated (this is actually a fairly normal propagation pattern). The ASNs seeing the prefix were mostly transit ASNs below, so this means that these routes were distributed broadly across the Internet. Almost all of the default free zone (DFZ) carried the hijacked route at least briefly.
| 18:47:00 | Uninterrupted videos of Exploding jello |
| 18:47:45 | First evidence of hijacked route propagating in Asia, AS path 3491 17557 |
| 18:48:00 | Several big trans-Pacific providers carrying hijacked route (9 ASNs) |
| 18:48:30 | Several DFZ providers now carrying the bad route (and 47 ASNs) |
| 18:49:00 | Most of the DFZ now carrying the bad route (and 93 ASNs) |
| 18:49:30 | All providers who will carry the hijacked route have it (total 97 ASNs) |
| 20:07:25 | YouTube, AS 36561 advertises the /24 that has been hijacked to its providers |
| 20:07:30 | Several DFZ providers stop carrying the erroneous route |
| 20:08:00 | Many downstream providers also drop the bad route |
| 20:08:30 | And a total of 40 some-odd providers have stopped using the hijacked route |
| 20:18:43 | And now, two more specific /25 routes are first seen from 36561 |
| 20:19:37 | 25 more providers prefer the /25 routes from 36561 |
| 20:28:12 | Peers of 36561 start seeing the routes that were advertised to transit at 20:07 |
| 20:50:59 | Evidence of attempted prepending, AS path was 3491 17557 17557 |
| 20:59:39 | Hijacked prefix is withdrawn by 3491, who disconnect 17557 |
| 21:00:00 | The world rejoices; Leeroy Jenkins online again. |
Since BGP relies on a transitive trust model, validation between customer and provider is important. In this case, PCCW (3491) did not validate Pakistan Telecom’s (17557) advertisement for 208.65.153.0/24. By accepting this advertisement and readvertising to its peers and providers PCCW was propagating the wrong route. Those who saw this route from PCCW selected it since it was a more specific route. YouTube was advertising 208.65.152.0/22 before the event started and the /24 was a smaller (and more specific) advertisement. According to usual BGP route selection process, the /24 was then chosen, effectively completing the hijack.
Because of the fast detection and reaction of the YouTube staff and cooperation with other providers, service for their (sub-) prefix was interrupted for only thirty minutes for some lucky customers and, at most, a bit more than two hours. The exact duration of the outage depends on your vantage point on the Internet.
When these sorts of events occur, there is renewed interest in a variety of solutions to this problem. BGP is fundamental to provider relationships and will not be going away anytime soon. Cryptographic extensions to BGP have been suggested (Pretty Good BGP, Secure Origin BGP and SBGP). These may be too taxing for router CPUs. Of course, after any sort of hijacking event (whether inadvertent or malicious) prefix and AS monitoring is suggested (e.g., the Internet Alert Registry, the Prefix Hijack Alert System, RIPE’s MyASN and Renesys’ Routing Intelligence).
Ultimately, though, the problem remains one of transitive trust. A provider can and should limit the advertisements it will accept from a customer. The mechanics can be arranged manually or can be configured using Routing Policy Specification Language (RPSL) to communicate the policy and drive configuration. In the case of Pakistan Telecom, they originate or transit fewer than 1000 prefixes.
So, it’s heartwarming to know that two things are still true. It is still trivially possible to hijack prefixes (whether maliciously or inadvertently). I can go to sleep knowing that my neighbors are happily watching their LOLCATS.
This post reproduced here with kind permission from Renesys. To visit the blog maintained by Renesys click here.
Sponsored byVerisign
Sponsored byCSC
Sponsored byDNIB.com
Sponsored byIPv4.Global
Sponsored byWhoisXML API
Sponsored byRadix
Sponsored byVerisign
A World-Renowned Source for Internet Developments. Serving Since 2002.
I’m actually surprised this doesn’t happen more often, if not inadvertently then as a form of attack. So much of the Internet indeed depends on trust, it’s a wonder it doesn’t break down more often. Smaller ISPs don’t have the resources necessarily to check out every route to make sure the advertiser is also the owner or otherwise authorized to advertise it if they are the owner’s ISP, and their policies may not be implemented strictly enough to prevent leaks. Regardless of whether you are doing AS path or prefix-based filters, mistakes can easily be made. More often then not, mistakes usually lead to the inadvertent filtering of a route rather than the advertisement of an incorrect route. But I’m betting there are a whole bunch of misconfigured filters sitting out there vulnerable to the announcement of an incorrect route, just waiting to let it through, surviving only on the correct configuration of downstream customers’ routers. The trust model mostly works, but is definitely vulnerable.