g-131 & e/l-131: exercise evaluation and improvement planning exercise evaluation overview...

G-131 & E/L-131: Exercise Evaluation and Improvement Planning

Exercise Evaluation OverviewKenneth L. Wierman Jr.

DHS/FEMA Headquarters

2010 National REP Conference

Course Goal

To familiarize individuals with exercise planning or evaluation responsibilities with the Homeland Security

Exercise Evaluation Program’s (HSEEP) exercise evaluation and improvement planning process, and to provide those students opportunities to practice key

exercise evaluation skills.

FEMA Directive 123-15 16 January 2009


Target Audience

Individuals who manage, participate in, review, or sponsor exercise evaluation and/or improvement planning

Federal, State, local, Tribal, Territorial government

Private sector

Non-governmental


Course Prerequisites

IS-120.A: Introduction to Exercises

IS-130: Exercise Evaluation and Improvement Planning

HSEEP Training Course

Radiological Emergency Preparedness Program (REP) and Planning Course (Recommended)


Course Objectives (1 of 2)

Describe the need for a systematic approach to exercise evaluation

List the eight steps of the exercise evaluation and improvement planning process

Identify pre-exercise activities necessary for a successful evaluation

Explain the function of Exercise Evaluation Guides (EEGs) and their relationship to the Target Capabilities List (TCL)/Universal Task List (UTL)


Course Objectives (2 of 2)

Practice evaluation observation, data analysis, and report writing skills

Describe post-exercise analysis activities

Explain the purpose and format of the After Action Report/Improvement Plan (AAR/IP)

Describe how a Corrective Action Program (CAP) translates exercise outcomes into continual improvements to preparedness


Course Structure (1 of 2)

Unit 1: Exercise Evaluation Overview

Unit 2: Exercise Evaluation Process

Unit 3: Planning and Organizing the Evaluation

Unit 4: Observing the Exercise and Collecting Data

Unit 5: Analyzing Data

Unit 6: After Action Report and After Action Conference

Unit 7: The Corrective Action Program

Course Summary (End of Two-Day Course)


Course Structure (2 of 2)

Unit 8: Introduction to Evaluator Skill Practice

Unit 9: Customizing EEGs from TCLs and Objectives

Unit 10: Exercise Evaluation: Skill Practice #1




Unit 14: Course Summary


Course Background

Original course introduced by FEMA in 1992

In 2002, DHS developed and introduced the Homeland Security Exercise and Evaluation Program (HSEEP)

All-hazards

Performance- and capabilities-based

All levels of government

DHS and FEMA have worked collaboratively to standardize the language and concepts used in the exercise evaluation process.

This course updated to reflect HSEEP guidance


Activity: Defining an Exercise

What is an Exercise?

Why do we Exercise?


Defining an Exercise

A focused practice activity that places players in a simulated situation

Conducted to support an overall assessment of a capability

Primary benefits:

Individual and team training

System improvement

Critical component of preparedness cycle


Activity: Defining Exercise Evaluation

What is Exercise Evaluation?


Defining Exercise Evaluation

Exercise evaluation is the act of:

Reviewing or observing and recording exercise activity or conduct

Assessing/analyzing behaviors and activities against exercise objectives/standards

Noting strengths, weaknesses, deficiencies, or other observations


Activity: Importance of Exercise Evaluation & Improvement Planning

Why is Exercise Evaluation and Improvement Planning Important?


Importance of Exercise Evaluation & Improvement Planning (1 of 2)

Exercise evaluation

Assesses a jurisdiction or organization's performance by identifying strengths and areas for improvement

Improvement planning

Leverages the outputs of the evaluation process by developing Improvement Plans (IPs)

Provides a disciplined process for implementing corrective actions and socializing best practices


Importance of Exercise Evaluation & Improvement Planning (2 of 2)

Exercise evaluation and improvement planning are critical components of the preparedness cycle

Implementation of corrective actions is the mechanism by which exercises can inform and improve other components of the preparedness cycle


Activity: Customizing Exercise Evaluation Guides (EEGs)


Plan Ref Task Metric Yes No Time

[Name, Pg/Sec#]

Order transportation services for evacuation.

[From plan]

[Name, Pg/Sec#]

Coordinate with supporting agencies and prearranged providers to obtain appropriate means of transportation for those requiring transportation assistance (e.g., buses, ambulances, handicap-assisted vans).

[From plan]

[Name, Pg/Sec#]

Implement plans for providing alternative means of transport for immobilized individuals or those needing other special assistance in transit.

[From plan]

Activity: Customizing Exercise Evaluation Guides (EEGs)


Activity Analysis

Observations (Each bullet will need a completed After Action Report Input form.)

Strengths

Areas for Improvement

Root CauseRoot CauseRoot Cause

Additional Observations:

Activity: Customizing Exercise Evaluation Guides (EEGs) – AAR Input Form


Pilot Courses

October 2009, February 2010 and April 2010

30 students

Federal Agencies

State and Local

FEMA

Chemical Stockpile Emergency Preparedness Program (CSEPP)

National Exercise Division (NED)

Radiological Emergency Preparedness Program (REPP)


Upcoming HSEEP Exercises

Columbia Generating Station – 30 August 2010

Indian Point – 20 September 2010

Susquehanna Steam Electric Station – 5 October 2010

Pilgrim and Quad Cities – 15 November 2010


Summary

What are the objectives of this course?

What is exercise evaluation and improvement planning?

Why is exercise evaluation and improvement planning important?


g-131 & e/l-131: exercise evaluation and improvement planning exercise evaluation overview...

Documents