By FRANK BAJAK and MATT O’BRIEN
AP Technology Writers
BOSTON (AP) — An outage at a little-known firm that speeds up access to websites knocked a lot of top internet destinations offline on Tuesday, disrupting business and leisure for untold millions globally. The problem was quickly resolved. The company, Fastly, blamed a configuration error in its technology.
But the incident — Fastly’s traffic dropped 75% for about an hour just as the U.S. East Coast was beginning to stir —raises questions about how vulnerable the global internet is to more serious disruption.
WHAT’S A FASTLY? WHY DID ITS TECHNICAL PROBLEMS LEAD TO SO MANY OUTAGES?
San Francisco-based Fastly isn’t a household name, but its “edge server” computing technology is used by many of the world’s most popular websites, such as The New York Times, Shopify, the Guardian, Ticketmaster, Pinterest, Etsy, Wayfair and Stripe. The British government is among its clients.
The company provides what’s called a content delivery network — an arrangement that allows customer websites to store data such as images and videos on various mirror servers across 26 countries so that the data is closer to users, and thus shows up faster. Many of Fastly’s customers are news sites that use its technology to update their websites with breaking news. Buzzfeed, for example, used Fastly to cut the time its users took to reach the site by half. Fastly had $290.9 million in revenues last year.
WAS THERE ANY BACKUP? COULD OTHER COMPANIES HAVE STEPPED IN IF THE PROBLEM HAD BEEN MORE SEVERE?
Customers rely on Fastly and its rivals to host and protect their website data from denial-of-service attacks and disruption from spikes in traffic. Had this outage been more serious, customers could have moved to competitors such as Cloudflare or Akamai. But that’s not simple; many businesses would have had to scramble and might have suffered losses.
“You can’t switch quickly to another service unless you had it set up ahead of time,” said Doug Madory, an internet infrastructure expert with the traffic-measuring company Kentik. “If Fastly were down for a day, that would be pretty bad.”
Even if they do have an alternative provider, engineering a smooth switchover from one to another is not for the faint of heart, said Ben April, chief technical officer of Farsight Security.
Madory and other experts said Fastly and its competitors spend heavily and devote major engineering resources to reducing the possibilities of such outages and ensuring they can recover as quickly as Fastly did on Tuesday.
Such outages are not new — but not at all common. “There may be years between when a company has an outage like this,” Madory added. “I think we are going to have these very rare but probably impactful short outages for the foreseeable future.”
ARE OTHER PARTS OF THE INTERNET SIMILARLY VULNERABLE?
Like the content distribution network world, cloud computing — when computing services are entrusted to a remote provider — is dominated by just a few major players led by Amazon Web Services, Google and Microsoft. Amazon, the biggest cloud provider, periodically has brief outages, which are a big deal for customers.
“And if it became a major outage of, say, more than six, eight hours — but days — it could put companies out of business,” said Josh Chessman, an analyst with the tech market researcher Gartner Inc.
The question is: What could cause such a serious outage that might destroy customer data? A major cyberattack is one possibility. Another is fire or catastrophic natural disaster. These businesses, after all, are based in datacenters. In March, a fire at a datacenter in Strasbourg, France, owned by a major cloud computing firm knocked out service to millions of websites.
SHOULD THE GOVERNMENT REGULATE THESE FIRMS? WHAT CAN COMPANIES AND INDIVIDUALS DO TO PROTECT THEMSELVES?
“I don’t know that we need regulation,” Chessman said. Suppose Congress proposed to mandate additional cloud providers to increase competition. “How do you do that?” he asked.
Of course, the federal government can set new standards for security at companies that control vast data resources online. It’s already beginning to tighten up cybersecurity requirements for critical infrastructure in the energy sector following last month’s cyberattack on the Colonial Pipeline, he said.
In a regulatory filing last year, Fastly said it had been subject to “cyber-attacks from third parties — including parties who we believe are sponsored by government actors.” Those attacks “have strained our network” and could harm it in the future, it said.
Businesses and consumers, meanwhile, should be thinking seriously about how much they should rely on the cloud for their most valuable data. “If there’s an outage, what’s the impact on our business?” Chessman asked. Perhaps it makes sense not to rely on a cloud-based service for your company’s email if you’d go bankrupt without it during a two-week outage.
But running your own email and backup services is complicated and costly — one reason companies turned to the cloud in the first place.
David Vaskevitch, a former Microsoft chief technical officer and CEO of the photo management app Mylio, said people have grown so accustomed to the always-on internet —- everywhere we go, we carry a pocket computer with us — that we wrongly assume it will be available 24/7/365.
“It’s not very realistic and it’s not a good way to live,” said Vaskevitch, who at 67 grew up in a pre-digital world. “The internet is always there — until it isn’t.”
Despite the vast interconnectedness of the world, it can still be wise to store some data locally, said Vaskevitch. Instead of streaming all our music, we should think about saving some locally. Same goes for email — for instance, in an arrangement where you store it on the computing device you use most.
“Your device is both the best friend of the internet and the best insurance policy,” he said. “When the internet goes down, if you arrange things carefully, you can still do most of the things you need to do.”
