The smoke was visible from 3 miles away on that cold Saturday four years ago, a sure sign that the fire would consume everything — including Bob Dorsey’s consulting and information technology business, R. Dorsey & Co.
But thanks to good preparation, the data that was absolutely crucial to Dorsey’s business were secure, even if all the computers and furnishings in his Grandview Heights office were a complete loss.
“We had a safe off-site data center,” Dorsey said. “So by Sunday, we were in (borrowed) offices. On Monday, we were back in operation, and we were able to put invoices out on Tuesday morning as usual.”
As storm season roars in again, tech-disaster preparations such as those that Dorsey made before that epic fire are at the top of businesses’ to-do list — or should be.
The losses can be utterly devastating if they aren’t.
Most companies that experience a computer outage lasting for more than 10 days will never fully recover financially, and at least 50
percent of those companies will be out of business within five years, according to a Pepperdine University study. Fewer than 10 percent of businesses completely recover from such lengthy outages.
“It’s really frightening,” said Anil Vadhi, the owner of CMIT Solutions on the Northwest Side, an IT service provider for small to midsize businesses. “People don’t realize the effect of a massive data loss. They think, if it ain’t broken, why bother? Then data crashes, and they have no backup and no way of recovering the data. It really leaves a business at risk.”
Even businesses with backup plans sometimes don’t think through their disaster preparations, said C. Matthew Curtin, founder of Interhack, a Columbus-based forensic computing firm.
“I’ve seen people who keep backups in basements, then the backups get completely flooded,” Curtin said. “Another one I’ve seen, there was a fire in their server room and it literally burned up a critical server, and it burned the backup tape in the drive.”
After that company got a replacement server, there was still trouble. “They loaded all the software from their vendors and then they loaded some tapes they had off-site,” Curtin said. “But it turned out the last two years of backup were blank. They had been doing weekly backups for two years, but nobody checked to find out if the backup was working.
“Yeah. That was bad.”
Small businesses tend to be most vulnerable, because the few people on the staff wear so many hats.
“There’s usually one guy who handles the system, so what happens if that guy is on vacation or sick?” Curtin said. “Or what happens if he gets hostile? I know of a small company where a guy held the passwords hostage. Weather is one of the things that can trigger a contingency plan, but there are plenty of other things that can have a similar effect.”
Even larger businesses and government agencies can slip up on basic data protection plans, said Michael Mata of Netwave Corp., which manages and secures data for businesses.
“It’s kind of scary when you look inside people’s networks,” Mata said. “It’s one thing if it’s a simple business. That impacts employees, customers. But it’s another thing if it’s a hospital or a government agency, like police or fire.”
But government agencies, hospitals and other critical organizations tend to be less-vulnerable to massive data losses from disasters than businesses, said Larry Mead, principal at Worthington Partners, a management-consulting firm.
“Government has been way ahead of the private sector on this,” Mead said. “The first World Trade Center bombing (in 1993) really galvanized the government. Business was slower to pick up on it. Everyone really started to look at the problem more closely in the ramp-up to Y2K.”
The rapid proliferation of companies that specialize in data protection is starting to improve the percentage of businesses that survive disasters. “But there’s no real silver-bullet design that will fit everyone’s needs,” said Rick Vanover, product strategy specialist at Veeam Software, a Columbus-based information-technology and software company.
Moving data off-site — to the cloud — has become an increasingly common part of tech-disaster plans as well as a big part of regular operations.
“We’re seeing a shift in data centers to virtual machines,” Vanover said. “It’s a great way to keep costs down.” And, in case of disaster at a company’s main office, it usually allows for a seamless transition to recovery.
However, while moving to the cloud has advantages, “The problem is it means that there’s a big, single point of failure for a lot of businesses,” Vanover said. Thus, if a tornado or flood imperils the offsite server that hosts the data, it won’t matter if the disaster passes the host’s dozens of clients by — in fact, they all will be in big trouble.
“I have one client who is completely on the cloud,” Mead said. “I asked him, where are they located? So we’re doing a continuity plan with their service provider, too.”
For Dorsey, recovery after the devastating fire came quickly in the tech area, but not as quickly or smoothly in other arenas.
“We found that people saw the news and thought we were out of business. So we had to get the word out that we were still in business.”
And even though Dorsey’s company survived, it took a long while to fully recover financially. “ The good news is that the cost of the technology side of disaster recovery is down 90 percent from five years ago,” he said, but his company’s $2.6 million insurance claim for the 2009
fire took until October 2012 to be completely resolved.
That’s why, he said, “Disaster recovery can be like root canal or it can be fun.”
LINK: http://www.dispatch.com/content/stories/business/2013/05/24/staying-afloat-in-a-tech-disaster.html
