Download - A Business Continuity Plan for Government
A Business Continuity Planfor GovernmentA Business Continuity Planfor GovernmentGeorge BomarDianne CaseyTexas Department of Licensing and Regulation
A practiced logistical plan for how an organization will recover and restore
partially or completely interrupted critical functions within a predetermined time after a disaster or extended disruption.
The Focus on PeopleThe Focus on People
“For the main event, CIO Steve Yates wanted to test more than the company's technology procedures; he wanted to incorporate the most unpredictable element in any contingency planning exercise: the people.”
USAA Insurance Company
Legacy of Y2k - Computer failures in banking, power, health, telecommunications and financial institutions
September 11, 2001– “Worst case” scenario concept shifted
80% of companies worldwide are not prepared for a pandemic or a natural disaster
U.S. DOL estimates over 40% of businesses never reopen following a disaster
Of the remaining 60%, 25% close within 2 years.
Selected StatsSelected Stats
Fires permanently close 44% of businesses affected
90% of companies that lose data are forced to shut down within 2 years
1993 World Trade Center bombing 150 of 350 affected businesses failed
Selected StatsSelected Stats
More Arkansas Poultry Flocks Checked For Bird Flu (UPDATED SATURDAY, JUNE 14, 2008 5:55 PM CDT IN NEWS)
By The Associated Press
“Within a few days all commercial chicken houses in the area had been tested and the 15,000 birds affected were killed and buried. The next step was for the commission to go door-to-door, checking for other cases.”
The Food and Drug Administration is expanding its warning to consumers nationwide that a salmonellosis outbreak has been linked to consumption of certain raw red plum, red Roma, and red round tomatoes, and products containing these raw, red tomatoes.
June 5, 2008
The Emergency Email and Wireless Network
What does BCP “look like”What does BCP “look like”
Formal printed manual
Full access by employees
Stored in multiple locations
Secondary work center
Copies of critical materials
Relationship to Relationship to Disaster Recovery PlanDisaster Recovery Plan
DR - focused on information technology applications domain
Overlap with BCP
Crisis mgmt structure
Secondary work center
Data requirements between primary and secondary work centers:
Telecommunications architecture;
Data replication methodology;
Application and software availability;
Any physical data requirements at secondary site.
Recommended BCP approachRecommended BCP approach
Smaller ones always contain partial elements of larger disasters
BCP should be broader than disaster recovery alone or in case of emergency (“ICE”) procedures
Plan for the BIG disastersPlan for the BIG disasters
BCP PurposeBCP Purpose
To enable leaders to
maintain essential business
processes and practices
and equip the organization
with means of becoming
less vulnerable to incidents
The TDLR PlanThe TDLR Plan
Identifies management team members
Designates remote site(s)
Enumerates four (4) major scenarios
Itemizes recovery steps to be taken within
five (5) primary business functions
Loss of key personnel
Weather-related
Infrastructure-related
Internal system breakdowns
EventsThat might trigger an interruptionEventsThat might trigger an interruption
Failure of an external business partner
Health crisis impacting the work force
A cyber attack
An act of terrorism
EventsThat might trigger an interruptionEventsThat might trigger an interruption
Rating the TriggersRating the Triggers
1- Least likely to happen
4 - Most likely to happen
Probabilities of occurrenceProbabilities of occurrence
ImpactsImpacts
DURATIONWill the effects be short-term, or longer?
EXTENTHow much of work force is impacted?
Devising a TemplateDevising a Template
A questionnaire was circulated to capture:
Recovery procedure
Recovery time objective
Recovery location
Dependencies
Other considerations
Summary of recovery steps
The ProcessThe Process
Solicit written input from key personnel
via templates
Interview managers
Prepare draft for each business function
Obtain review comments and incorporate
into revised draft
How About Prevention?How About Prevention?
Mitigate the impact of a disaster:
Practice good housekeeping
Adhere to security procedures
Observe information security procedures
Maintain up-to-date operating guidelines
An Emergency Management Team An Emergency Management Team Convenes to decide: Convenes to decide:
Implement the BCP?
Activation prompted by Team Lead
Alternate Location(s)Alternate Location(s)
Primary Site
Alternate Site
BCP provides directions to the sites
Scenario IScenario I
The population of possible causes was condensed into four (4) major scenarios:
Loss of key executive personnel for a protracted period due to accident or other unforeseen event;
Scenario IIScenario II
Loss of building access because of weather (or other natural disaster)-related event;
Scenario IIIScenario III
Contractor default, or other supplier of a critical service to the agency, abruptly goes out of business without warning; and,
Scenario IVScenario IV
Health crisis (or act of terrorism) leads to an exorbitant rate of employee absenteeism (and temporary replacements are unavailable).
Functions ImpactedFunctions Impacted
The plan identifies five (5) main business functions adversely affected by the crisis:
Licensing of individuals and businesses
Education and examination activities
Measures to ensure compliance
Administrative support
Technological support
Initial ApproachInitial Approach
For each of the five (5) business functions,
Identify impact,
Recovery procedures, and
Dependencies
Redundancy
Adopted ApproachAdopted Approach
For each of the four (4) scenarios:
Identify how each business function
would be adversely impacted
Example IExample I
If key personnel were lost (Scenario I)
Notify the agency’s directors
Convene emergency meeting of the Commission
Formulate short-term succession plan
Notify Governor’s office and key legislators
Designate primary agency contacts
Implement plans to notify the public, equip customer service, respond to complaints
Example IIExample II
If building was inaccessible (Scenario II)
Licensing
Education and Examinations
Compliance
Administrative Support
Technological Support
Example IIIExample III
If major contractor failed (Scenario III)
Identify affected functions
Marketplace alternatives?
Make temporary process changes
Procure new/other contractor
Example IVExample IV
If a health crisis decimated the work force (Scenario IV)
Identify skills of available staff
Can skills be realigned?
Determine what functions (e.g. inspections) can be postponed or suspended
Consider tapping into regulated industries for temporary expertise
A Summary of Recovery StepsA Summary of Recovery Steps
Plan must specify:
Key actions to be taken,
By whom,
In what order,
For each business function.
Important AddendaImportant Addenda
Identify in an Appendix
BCP Team Lead and Members
with current contact information
Name and address
Phone number(s)
E-mail address(es)
Include:
a Phone Tree listing - who will contact whom;
Identify how information will be disseminated to employees;
List first group(s) to report to alternate site.
Periodically,
re-assess your BCP
and update as needed!
TestingTesting
Purpose:
Achieve organizational acceptance
Determine that the BCP solution is appropriate for recovery requirements
Identify and correct design flaws
Identify and correct implementation errors
After 9/11, those companies with
tested BCP manuals had business
resumption within days.
Selected StatsSelected Stats
45% of companies with a BCP do not test it annually
80% of companies have not developed an IT crisis management function
40% of companies that have a crisis management plan do not have a dedicated crisis management team
Mistakes and PitfallsMistakes and Pitfalls
Failing to gain senior level management support
Not identifying all critical systems (including laptop data)
Failing to bring the entire business into planning and testing
Not identifying and planning for all gaps in recovery objectives
Insufficient funding for testing
USAA StoryUSAA Story
20,000+ employees - needed HazMat training, an evacuation plan and a recovery plan
Live exercises were confined to technology assets - recovering data from backup data
Otherwise, passive exercises – tabletop and paper simulations, role-play, guessing how people would react
Post 9/11, built alternative center 200 miles away from San Antonio, on different power grid and water supply
Steve Yates designed large scale continuity exercises
At the first one, USAA discovered:
The setup process for computers and phones took nearly two hours leaving employees standing in the hot Texas sun.
USAA StoryUSAA Story
USAA ‘take-away’ from testing:
Those who walked through the simulation were in the best position to find flaws and offer suggestions.
Those who practice emergency situations are less likely to panic and are more likely to remember the plan.
USAA StoryUSAA Story
Plan Maintenance CyclePlan Maintenance Cycle
Revisit annually or biannually
Confirm information; roll out to all staff
Perform staff training
Test and verify technical solutions for recovery
Test organization recovery procedures
Questions
????
Presenters:
George Bomar – 512-936-4313
Dianne Casey – 512-463-7182
Texas Department of Licensing and Regulation