A storms rages, the streets are flooded and phones lines are dead. You are the network administrator for a small company in downtown Dallas and you just got paged from your boss telling you that you need to weather the storm to come into the office. He tells you the network is down, there is no documentation on your company?s Disaster Recovery Contingency plan and the backup systems has as of yet to bring the network back up. And that?s the good news. Aside from that, calls are being missed, orders are not being processed and every minute the network is down the company is losing money. Is your company ready? A little insight into our story, Jonathan Mallard is the Supervisor over the IT department at Terrance&Willie?s Graphical Designs. The company is fairly new having been in business for almost two years and is currently waiting to go I.P.O.
willie nelson @ fresno
Business continuity planning life cycle
IoSafe disaster recovery business continuity logoConsidering the fact that the company only lists less than eighty employees on the payroll, it?s monthly revenue have grown steadily surpassing a 3 million dollar gross last quarter. The company currently employs a handful of graphical designers, software programmers, managers, supervisors, IT department, a small HR/legal department along with a 24/7 call center to process incoming orders and provide technical support. Only four people man the IT department where they are responsible for the inter-connectivity of all data and telephone communications in the two-story building. Housed with a plethora of Cat 5 cabling, patch panels, an Avaya switch, Dell PC?s, hubs, routers, not to mention a ton of protocols to govern them all, the four member IT team has their work cut out for them.
Before we begin there is a distinction that needs to be noted between Disaster Recovery and Business Continuity despite the fact that the two terms may be used interchangeably. Let us start with the definition of a disaster. When the mention of the word is brought up, most people think natural (acts of God) or man made; hurricanes, tornados, floods, bombings, and wars. Disaster is an occurrence causing widespread destruction and distress. To a Network Manager these are all just precursors to an internal disaster; a network failure. Disaster Recovery in the corporate sense describes how a business reacts/recovers from the event of a disaster. Business Continuity is the ability of a business to continue its operations with minimal disruption or downtime in the event of a natural or intentional disaster. Telephone lines are down, email is down, the intercom system is down, and no one has access to centralized servers which houses the programs used to perform daily operations. This is an internal corporate disaster. Though disaster recovery and business continuity are critical in the success of a business, why is it never mentioned until the company encounters a disaster? According to a cover story in Network Magazine entitled ?Business Continuity? 79 percent of the participating corporation who participate in the survey do not have a documented and tested BCM (Business Continuity Management) plan and 64 percent had no plan in place to address this crucial business need. Using our above company, Terrance&Willie?s Graphical Designs, it is apparent that not only did they not have a plan documented and tested, but also, no one else knew about a back up plan other than the lone IT guy (Jonathan) who was at not even in the office. Upon paging him not only is the company?s business on the line but now the IT personnel?s life is at stake as he attempts to endure treacherous weather conditions to come into the office to try to restore connectivity. Now is being unprepared really worth risking the life of another human being? There are several steps needed to design and implement a Disaster Recovery plan, bearing in mind that it is an ongoing process and should never be abandoned. Since natural disasters cannot be avoided all we can do is prepare for them. A good business continuity plan first begins with assessing the needs of the business. To begin, organizations will need to determine how long the systems can be down before declaring it a disaster. After pooling the company?s departmental and business needs, managers will need to develop recovery priorities, meaning, determine which systems are most critical and will need to be restored first. Not all assets are considered business critical despite popular belief. For Terrance&Willie?s Graphical Designs the majority of their orders are processed electronically so getting the data communications network back up and running may be more important than getting the electronic doors reset. For the call center where calls are now experiencing telco greetings stating, ?The number you have reached has been temporarily disconnected or is no longer in service,? is far more business critical than getting the lights in the bathroom to stop flickering. Sounds silly but these are some of the gripes heard during a disaster.
Sanjay Dhawan, Executive Director IRM Practice at KPMG states that, ?A business continuity plan should provide an enterprise-wide risk-based approach, covering People, Processes, Technology and Extended Enterprise to ensure continuing availability of business support systems and minimize disruption risks." This leads to the second step of designing a Business Continuity Plan, Documentation and Implementation. During this process all documents and plans are prepared, any additional equipment and supplies are identified and obtained and all personnel are trained on their respective roles and responsibilities. Items such as stationary, toilet paper, light bulbs, telephones/headsets, PC?s, desks, and chairs are all items that need to be listed in the plan. Specific departmental needs should also be addressed. The accounts receivable department may need calculators, the training department may need additional PC?s or additional space allocated on the server for training programs, technicians may need laptops or additional company vehicles if a natural disaster damages the current ones. Document the plan so that non-IT staff can understand it. A business cannot assume IT staff will be immediately available to recover from an outage. Jonathan will also need to train non-IT staff members to recover from outages when possible. At least parts of the plan should be documented so that non-IT staff can understand it and perform crucial functions.
Generally speaking, disasters bring about chaos and confusion but with a trained disaster recovery team in place to take control of the situation and ensure that processes and procedures are being adhered to, may prove invaluable during a disaster. Not only did Terrance&Willie?s Graphical Designs not have a plan drafted and documented, but only one person was aware that a plan even existed, Jonathan.
According to a report published by the Strategic Research Corporation, a Santa Barbara, California, market research and consulting firm, the financial impact of a major system can be enormous: ÷ $6.5 million per hour in the case of a brokerage operation ÷ $2.6 million per hour for a credit-card sales authorization system ÷ $14,500 per hour in automated teller machine fees if an ATM system is offline More difficult to calculate are the intangible damages a company can suffer: Lower morale and lowered productivity, increased employee stress, delays or misses in key project timelines, diverted resources, unrecoverable data and a tainted public image are to name a few.
There are several options in reference to a companies business continuity plan. Sungard, as an example, is a company that comes in, mirrors a companies current business operation and sets up a mirrored office in an alternate location. Please note that if an alternate location is set up, managers need to be able to provide maps, transportation and safety to key personnel identified on the disaster recovery team. Central Offices will need to be notified as to what traffic needs to be redirected. Another option is IBM Tivoli Storage Manager which is a software product designed to help protect and manage the storage of a company's business infrastructure from mainframes and servers to laptops and desktops. Not to sound like an advertising campaign but IBM Tivoli Storage Manager provides the following functions: ÷ Backup and Restore ÷ Archive and Retrieve ÷ Disaster Recovery Preparation ÷ Hierarchical Storage Management Protection of critical 24x365 DB and Applications Requesting additional funding for business continuity plans should be at the forefront of financial prioritization. Although outsourcing may be more expensive, it would somewhat relieve the burden of responsibility of Jonathan Mallard at Terrance&Willie?s Graphical Designs.
The third and final step in developing a disaster recovery/business continuity plan is to thoroughly test your plan. Once documents have been prepared and people have been trained, drills and exercises should be conducted to ensure the plan would work. Provide a continual means to update the plan and adjust for organizational changes. Have essential personnel come to work at the alternate location and have the CO redirect all calls to the alternate call center. For Terrance&Willie?s Graphical Designs, have the servers downloaded to the alternate location and assess the performance of the business. Always encourage upper management and executives to participate in the testing process to ensure that all business needs are being addressed. Being open to constructive criticisms and adjusting the procedures as needed will only lead to a seemingly effortless traverse to your backup systems. Ultimately it is the customer satisfaction and employee moral that will determine if your business continuity plan is a success.
Though natural disasters cannot be avoided, human disasters (operator error, sabotage, and malicious codes) and technological disasters (equipment failure, software error, telecommunications network outage, and electric power failure) can be avoided or deduced. To be proactive when dealing with disasters is to be successful. Any company that has implemented a single business continuity strategy for an entire organization is likely ill prepared, or spending money unnecessarily. Few organizations have the resources to assure business continuity for every function. The key to business continuity lies in understanding your business, (a graphical design company in our case) determining which processes are critical to staying in that business, (access to the internet/email and telephones) and identifying all the elements crucial to those processes. By thoroughly analyzing these elements, we can identify risks and make informed and intelligent business decisions. Once a program has been developed, deployed and tested, critical processes and resources should be available around the clock; despite a disaster. In conclusion, while testing, assume that your plan will fail and commit to keeping your program current as business and technology infrastructures constantly change. You can never be over prepared when it comes to disaster recovery/business continuity. The more you test your plan the more prepared you will be for the unexpected. As for Terrance&Willie?s Graphical Designs we can only hope that they learn from their mistakes and that the company will continue to be successful despite a truly regrettable and financially crippling learning experience. We believe it was Confucius that once said, ?That which does not kill me can only make me stronger,? but when it comes to a business strategy and disaster recovery/business continuity plan we choose to implement a different philosophy, ?Life is all about taking chances, but there are some chances that you just don?t have to take.? Is your business ready?
Terrance the cheerleader rocks!
Terrance Topiary
Terrance Topiary