Posted by: Eric Siegel
OK, yesterday there was a major disruption of the communications links in the San Francisco Bay Area. Ten fiber optic cables, belonging to AT&T and to Sprint, were cut at four separate locations by saboteurs. (See, for example, The San Francisco Chronicle, "Sabotage attacks knock out phone service" August 10, 2009,and The Mercury News, "Phone service fully restored; AT&T offers 100K reward" August 10, 2009)
All Hell broke loose; telephones were down, ATMs stopped working; 911 calls went nowhere.
There were the usual articles talking about how this should "remind" everyone about link failures, et cetera, and about how everyone was so surprised at this.
There really shouldn't be any surprise. THIS is a major reason for having a truly remote data center! In any geographic area, there are only a few paths for communications links. They're typically next to railroads or other utilities that have a long, narrow right-of-way. It's just too difficult to negotiate with hundreds of landowners for a new right-of-way for hundreds of miles. The result is that most communications links, even from different carriers, end up in the same manholes and tunnels. (Remember the July 2001 CSX train derailment in a Baltimore railroad tunnel? It started a fire and melted the fiber optic cables in that tunnel, causing major problems for at least five ISPs. Or how about the December 2008 cut of a major undersea cable in the Mediterranean, one of many cuts, often caused by boat anchors? [See WIRED.com "Undersea Cables Cut; 14 Countries Lose Web -- Updated" December 19, 2008 ] There are hundreds of examples.)
The rules are:
If you have "redundant" access links to the service provider, using the same or different carriers, you must inquire very carefully about the routing of those links. They will usually share facilities at some point! Be sure each access link arrives at a major service provider switching node, with many physically divergent routes from that point, before it encounters any shared facilities with any other access link. Physical diversity will guard against a single physical event killing both access links into the massive service provider meshes.
Strongly consider using alternative media, such as satellite or radio links (see the Burton Group reports "Fixed Broadband Wireless Access Technology: Unwiring the Last Mile" and "The Wild World of WISPs" ), to connect to a service provider who will be outside the radius of any probable disruptions.
Ensure that your "backup" link is actually working. Share traffic across it, or test it very frequently.
And remember that loss of communications is a major reason for having a redundant data center!
Comments