Helping clients manage their technology for over 30 years.

What happens when disaster strikes the DR guys?

Traffic sign indicating disaster

It was a stormy Wednesday morning commute with intense wind and driving rain, when a driver lost control of her car, struck a utility pole and ultimately caused eight to fall all along the road in front of the Quest building. The power went out, and live wires and downed transformers blocked traffic. Everyone in the office was trapped.

This wasn’t a “major” event — not the kind of incident we typically think of when we talk disaster. Yet even something this mundane could have put our company completely out of operation for at least several days.

We executed our own disaster recovery plan

As a Disaster Recovery and Business Continuity services provider for scores of clients, Quest was better positioned than most companies to handle just such a disruption.

Initially, battery and generator backup provided phone and Internet capabilities. By utilizing resources at several other locations, Quest was able to continue functioning until they got the all-clear to evacuate, and fortunately no one was injured — that’s when we began executing on Quest’s own Disaster Recovery procedures.

By three o’clock the same afternoon we were fully operational at remote locations, with some of our staff at our Business Resumption Center and others working from home. Customer service calls, billing, email, phones — everything we needed to keep functioning was operational.

No operational disruptions

For the Quest team, the event was an unqualified success—not a drill, but the real deal providing employees with a window to what the company does for clients. As for Quest customers, they didn’t experience any difference in service.

There’s nothing Quest could do before that we can’t do now. That’s precisely why we have Disaster Recovery capability.

If you never drill, it’s just theory

“Part of the success of the plan’s execution,” says Quest CTO, Mike Dillon, “came from disaster recovery drills, which Quest does quarterly.”

“Drills make a huge difference — we already knew what we needed for our critical systems to function,” explains Mike. “If you never drill, it’s just theory. Every drill we do teaches us something, makes us smarter about our own operations, and smarter about the operations of our clients.”

Quest’s disaster recovery experience is a big advantage for clients. Most companies, even those with a plan, don’t take disaster recovery drills seriously. Even for those that do, the disaster will still be a first-time event. Our business is helping companies, including our own, recover from disasters. Our clients have that experience to lean on.

Be prepared for the mundane and the catastrophic

Every event, real or drill, is a learning experience. It’s a sentiment shared by Quest COO Kathy Campbell. “One of our ah-ha moments came when we had to address some issues that occurred at the corporate office during our absence —no power to the employee refrigerators and freezers, and no one in the office to feed the fish. Continuing our business operations at a remote site turned out to be the easy part.” And it allowed the city, county, and power company to do their clean up for as long as necessary

Reflecting on lessons learned, I put a priority on keeping all staff up to date about what’s happening. The need-to-know rule applies to everyone in your organization. My primary take-away? Even little disasters can have a huge impact. You need to be as prepared for a mundane disruption as you are for a catastrophic one.

Downed telephone poles in front of the Quest office

How cloud computing and VoIP make IT disruption avoidance easier — and less costly

Nobody stays in business long if their business-critical data and apps are lost. So pardon me if I sound like my replay button got stuck, but I’ll say it again: make sure your critical data and apps are replicated to a secure remote environment that’s always accessible from anywhere.

You’re at least halfway there if you’re using a cloud-based backup replication service — but, of course, you need to make sure you’re dealing with a provider with a secure, scalable, fail-safe environment and plenty of flexibility when it comes to service options.

Continue reading