planning for the inevitable: it disaster preparedness - linda sharp

SchoolDude University 2009

Planning for the Inevitable:IT Crisis Preparedness

Linda SharpCoSN Project Director

IT Crisis Preparedness


Expect and prepare

for the unexpected!


Schools Run 24/7

• Evening use of facilities

• Backup reports running at off-instructional hours

• Students and parents accessing the district website around the clock

Other activities and uses in your district?


Reliance on Technology

• Instructional activities• Business operations• Student data and recordkeeping• Assessment and accountability• Internal and external communication

with stakeholders

Other areas of reliance in your district?


District Objectives in Any Disaster

• Safety and welfare of students

• Safety and welfare of staff

• Protection of property and facilities


District Objectives in Any Disaster

• Maintenance of essential services for as long as possible, shutting down least critical ones first

• Restoration of services - critical ones first - in the shortest amount of time possible


Think About It?

What are some predictable threats in your own community?


Potential Disasters

• Natural disasters• Violence, vandalism • Man-made threats


Potential Disasters

• Natural disasters, acts of God

• Violence, vandalism• Man-made threats• Widespread medical

emergencies and pandemics

http://www.beatingbirdflu.com/images/pandemic.jpg


Potential Disasters

• Natural disasters• Violence, vandalism• Man-made threats• Widespread medical

emergencies and pandemics

• Digital threats

http://www.cyberpunkreview.com/images/hackers08.jpg


Cyber Security for the Digital District

www.securedistrict.org

• Tools and information to:– Assess and improve security of technology

systems – To protect safety of staff and students– Contribute to educational mission of their

schools – Maintain community support


Security Planning Process

Outcome:Outcome:Security Project Description

goals processes resources decision-making standards

Phase 1: Create Leadership Team & Set Security Goals

Outcome:Outcome:Prioritized Risk Assessment

A ranked list of vulnerabilities to guide the Risk Reduction Phase

Phase 2: Risk Analysis

Outcome:Outcome:Implemented Security Plan

Risk Analysis and Risk Reduction processes must be regularly repeated to ensure effectiveness

Phase 3: Risk Reduction

Outcome:Outcome:Crisis Management Plan

A blueprint for organizational continuity

Phase 4: Crisis Management


Security Planning GridSecurity Area Basic Developing Adequate Advanced

Management

Leadership:

Little participation in IT security

Aware but little support provided

Supports and

funds security

Aligns security

with organizational

mission

Technology

Network design and IT operations:

broadly

vulnerable

security roll out

is incomplete

mostly secure

seamless security

Environmental & Physical:

Infrastructure:

not secure partially secure

mostly secure

secure

End Users

Stakeholders:

unaware of role in security

Limited

awareness

and training

Improved

awareness,

Mostly

trained

Proactive

participants in

security


Security Planning Grid

• Provides benchmarks for assessing key security preparedness factors

• Uses the same topic areas for consistency

• Helps prioritize security improvement action steps


You never have time to plan for something you don’t think will ever happen.


Disaster Planning


Mitigation and Prevention

Actions you take to identify preventable and unavoidable disasters and to address what can be done to eliminate or reduce the likelihood of a disaster and/or its accompanying risks

Cameron ParishCameron ParishSchool Board OfficeSchool Board Office


Preparedness

Consideration of worst-case scenarios and development of comprehensive plan for coordinated and effective response to any given disaster

South Cameron HighSouth Cameron High


Response

Execution of the preparedness plan and management of the disaster


Recovery

Efficient and timely restoration of mission-critical operations and processes


Risk Assessment

• Analyze processes and functions deemed mission-critical.

• Identify types of potential disasters and impact of each on mission-critical items.


Risk Assessment

• Prioritize based on acceptable period of unavailability.

• Chart the workflow, considering hardware, software, people and other resource requirements for continued operations.


Risk Assessment

Imagine worst-case scenarios for all types of potential disasters.

– What would be lost?– What data would be critical?– How would you communicate?– How would you restore mission-critical

services?


Consider Lack of Availability of Key Services and Operations

• What must be restored within 1 hour?

• What must be restored within 4 hours?

• What must be restored within 1 day?

• What must be restored within 3 days?

• What must be restored within 1 week?

• What could wait for 30 days or longer?


Disaster Recovery Plan

• Easy to understand and follow• Organized into sections• Detailed steps of tasks to be accomplished• Multiple formats for different audiences• Print and electronic


The worst case scenario . . .

No Plan!No Plan!


First Steps

Identify Planning Team


First Steps

• Identify and Classify Services, Operations and Records– Vital– Important– Non-essential


Resources and Redundancies

• Hardware

• Software

• Communications

• Facilities

• People


Hardware

• Identify all required hardware.• Be sure to include resources required

to run and maintain hardware.• Regularly update

your list.• Maintain key

documents offsite.


Software

• Identify all required software.

• Regularly update the list.

• Keep copies of key applications offsite.

• Maintain key documents offsite.• Be certain your backup systems

are reliable - and redundant.


Software

• Data– Secure and Restore Data– Assess Capabilities of Providers


Communications

• Establish a communications plan

• Develop strategic partnerships

• Employee communications


Communications

• Single point of contact

• What is communicated

• Technical staff support

• Ensure redundancies


People

• Who is qualified to manage tasks?

• Have they been trained?

• What is their prior experience?

• Ensure key people resources are backed up.


People

• Incident Response Team

• Identify critical personnel

• Communicate roles and responsibilities

• Ensure personnel have authority needed


Facilities

• Have building blue prints available

• Have all shut-off valves clearly labeled or color coded on blue prints

• Identify evacuation sites

• Identify potential known hazard areas


Emergency Operations Center (EOC)

• Determine overall strategy and priorities.

• Allocate resources.

• Manage the incident.

• Ensure objectives are met.

• Ensure strategies are followed.


Develop a Staged Shutdown

• Move from simple preparedness to ceasing operations.

• Protect assets while staff is available to do the work.

• Ensure that mission-critical operations are the last to be stopped.

• Ensure shutdown can be reversed if needed.


Exemplary District Plans

– Fairfax County (VA) Public Schoolswww.fcps.edu/emergencyplan

– Montgomery County (MD) Public Schoolswww.mcps.k12.md.us/info/emergency/

– North Carolina’s Critical Incident Response Kit Project

www.ncjjdp.org/cpsv/cirk/cirk.htm


Those who have lived it!

• Dr. Sheryl Abshire, Ph.DChief Technology Officer Calcasieu Parish Public Schools, Lake Charles, LA

• Robert Gravina, Chief Technology OfficerPoway Unified School District, CA

• Wayne HowardTechnology DirectorPlatte Canyon School District


Calcasieu Parish School System (LA)

• Hurricane Rita struck the Louisiana / Texas border on September 24, 2005 as a category 3 storm with 120 mph sustained winds. Calcasieu Parish was hit by the hurricane eyewall and the east quadrant which has the strongest winds.

• 34,000 students and 5,000 staff displaced

• 2008 experienced Ike and Gustav


Calcasieu Parish School System

• Every school damaged. Many schools in Calcasieu Parish received extensive roof and water damage. The lack of power afterwards promoted mold and mildew growth.

• 24 hours after Rita hit, the CPSB weband email servers were back up andproviding information to evacueesacross the country.

• 34 days later, CPSB schools reopened.



Many didn’t see IT as the recovery team – yet they took the initiative and were ready when disaster hit. Take the leadership if no one else is doing it.



Document during response and

recovery

• Pictures

• Written records

• Items destroyed or damaged

• Items purchased



Don’t just create a plan—

communicate and practice it



• Develop a culture of preparedness.

• Revisit and actively practice the plan.

• Conduct periodic audits of the plan.

Practice, Practice, Practice



• Staff

• Power and capabilities

• Computer and storage options

• Facilities

• Records and files

• Communication methods

Redundancy, Redundancy, Redundancy



• You can’t over plan:

• Identify mission critical operations

• Think strategically• Pay attention to detail

“A plan needs to exist before it is needed. Making one on the fly is too late.”


Poway Unified School DistrictPoway, CA

• Poway is the third largest school district in San Diego County covering 100 square miles and serving approximately 33,000 students.

• During the fires of 2007, the school district became the county’s communication center.


Poway Unified School District

• School Business Continuity • Work on creating a stable and reliable

network for your organization



• Servers that can handle capacity in an emergencyYou may be the best form of

communication in your area• Clean data• Online access• Bandwidth for learning• Opening up applications to your stakeholders



Secure dedicated equipment, software,suppliesCapacity to Rebuild/Disaster Recovery

– Tape Drives and Juke Box (must be the same as what you are currently using)

– Back up server (work with your vendor)– Estimated time to recover– Personnel availability (cross training)



Moving your EOC• What would you do if you had to evacuate

your EOC?• Could you set up a fully functional IT

Department?• How long would it take?• Would you be able to have access to your

network?



Remote Learning• The Bird Flu, “when, not if”• Applications that allow for anywhere, anytime

learning• Content Management Systems• Online interactive tools• Online courses



What Worked…..

• Multiple Communications Systems

– Auto dialers

– Webpage

– Content systems


Platte Canyon School District, CO

• Mountain Community – 45 minutes from Denver

• Approx 1300 students• Platte Canyon High School - 480

students• Preparations in place after Columbine

shootings• Lone intruder shot student


Platte Canyon School District

• Communications and coordination with police was key

• Separate communication channels• Cameras• Constant testing

and update


“It’s not the plan that’s important—it’s the planning.”

Graeme Edwards


What are You Doing?

• Tips to share with colleagues

• What will you do now?


Consortium for School Networking

www.cosn.org

www.cosn.org/itcrisisprep


Thank you Sponsors

http://www.pearsonschool.com/index.cfm?locator=PSZ13e

http://www.dell.com/


Linda Sharp

CoSN Project Director

IT Crisis Preparedness

[email protected]

Hope is Not a Strategy!

planning for the inevitable: it disaster preparedness - linda sharp

Technology

andfunds security

security aware

cyber security

security plan risk analysis

security of technology

security area basic

critical operations

risk reduction phase