evaluation of the iaff hazmat and wmd training programs - … · 2020-05-14 · evaluation of the...
Post on 02-Jun-2020
3 Views
Preview:
TRANSCRIPT
Evaluation of the IAFF Hazmat and WMD Training ProgramsEvaluation of the IAFF Hazmat Evaluation of the IAFF Hazmat and WMD Training Programsand WMD Training ProgramsWETP Fall Technical Workshop, 10-18-07, Sheraton Chapel Hill, Chapel Hill, NCPatrick MorrisonAssistant to the General PresidentInternational Association of Fire Fighters
Bruce Lippy, Ph.D., CIH, CSPPresident, The Lippy Group, LLC
Introductory CommentsIntroductory CommentsPatrick Morrison
Assistant to the General PresidentIAFF
Bruce LippyBruce Lippy
President, The Lippy Group
Lippy Group Lippy Group Company Picnic 2007Company Picnic 2007
First row, left to right: Lippy, B.
Lippy Group Corporate MetricsLippy Group Corporate Metrics
Evaluation TeamEvaluation Team
Bruce Lippy, Ph.D., CIH, CSPJames Weeks, Sc.D., CIHBernard Mizula, MS, CHS-V, CET, RPIH
IAFF provided us an office and full access
Bruce Bruce Lippy’sLippy’s experience with experience with fire fightingfire fighting
Setting duckpins for the Manchester, MD Fire
Company
Evaluation approachEvaluation approach1. Interviews of board members,
headquarters staff, and students2. Evaluation of NIEHS and DHS
progress reports3. Focus groups with IAFF instructors 4. Analysis of IAFF student database5. Comparison of findings against
recommendations from guidance documents
6. Onsite evaluation of training program
7. Follow-up interviews of federal, state and county officials
Final Issues for evaluationFinal Issues for evaluation1. Methods of communication 2. Student database3. Student registration process4. Class records (storage, accuracy, etc.)5. Class sizes 6. Make-up of instructor cadre 7. Current programs 8. New programs9. Instructor resource center10. Blended learning
Evaluation timelineEvaluation timelineInitial discussions March 06
Interviews, observations and data analysis
Sep to Nov 06
Presentation to General President and Advisory Board of initial findings
Oct 06
Evaluation of presentation at Trainers Exchange
March 07
Draft reports March to May 07
Final report (v. 12) July 07
Broad Finding:Broad Finding:
NIEHSNIEHSMinimum Minimum Criteria is Criteria is excellent basis excellent basis for program for program evaluationevaluation
Is there evidence that the program is accomplishing its objectives?
IAFF ranked #2 in cost per contact hr.IAFF ranked #2 in cost per contact hr.(NIEHS Annual Report 2003)(NIEHS Annual Report 2003)
IAFF ranked 7/17 in cost per studentIAFF ranked 7/17 in cost per student(NIEHS Annual Report 2003)(NIEHS Annual Report 2003)
IAFF Number of traineesIAFF Number of traineesProjected v. actual, Projected v. actual, (FY96 (FY96 –– FY04)FY04)
Are appropriate facilities and staff available and committed to the program?
YES!“Training coordinators” are key to effectivenessStaff meetings short, focused and productive“Lead” instructor facilitates communication with the training site and with headquartersTraining at the host site with their equipment
Methods of CommunicationsMethods of CommunicationsRecommendationRecommendation
Emphasize prerequisites for courses, particularly T-t-T, and don’t rely solely on the
requesting organization
Methods of CommunicationsMethods of CommunicationsRecommendationsRecommendations
Include more actual case studies in reports to NIEHS
NIEHS official:*“Firefighters save lives every day, but their progress reports don’t reflect that sufficiently, they can be boring.”
*Ted
Student DatabaseStudent DatabaseImpressive File-Maker Pro database maintained by an outside contractor for the last 8 years38,000 records in database described as a “learning management system”
System covers shipping, System covers shipping, finance, and classesfinance, and classes
IAFF Shipping ProcessIAFF Shipping Process
Unlike other NIEHS grantees, IAFF relies upon host for equipment Student notebooks and reference materials (Pocket Guide, DOT ER book) Shipped out of headquartersRecommended positioning Guides in active locations, like AZ, rather than constantly shipping from HQ
Gender
Male91%
Female6%
NA0%
Missing3%
Student Demographics from Student Demographics from database: Genderdatabase: Gender
0.010.020.030.040.050.060.070.0
Percent
Cauca
sian
Hispan
ic
African
America
n NA
Asian/Pac
ific Is
lande
r
Native
America
n
Student Demographics: Student Demographics: EthnicityEthnicity
Age of IAFF StudentsAge of IAFF StudentsMean age is 33 yearsMean age is 33 years
Histogram: Age
-10 0 10 20 30 40 50 60 70 80 90 1000
2000
4000
6000
8000
10000
12000
14000
16000
18000
No.
of o
bs.
Schooling of IAFF StudentsSchooling of IAFF Students
Hig
h Sc
hool
Som
e C
olle
ge NA
Gra
de S
choo
l
Bach
elor
's D
egre
e
Asso
ciat
e D
egre
e
Post
Bac
helo
r's D
egre
e
Mas
ter's
Deg
ree
02000400060008000
10000120001400016000180002000022000240002600028000
No
of o
bs
0.05.0
10.015.020.025.030.035.040.045.0
Percent Completed
FRO
ERT:OpsCSO
Tech
TtT FRO
WMD TtT
zOther ID
Inc. Mgt.
Courses ProvidedCourses Provided
0102030405060708090
Percent Employed
Fire Serv
iceOthe
rNA
Private
Indu
stry/C
onsul
tant
Law Enfo
rcement
Public
Safety
Private
EMS
Indus
trial F
ire Brig
ade
Emergency
Man
agemen
t
Public
EMS
Employment of studentsEmployment of students
05
101520253035
Percent
Less
than
one y
ear
1 - 5
year
s
Over 2
0 yea
rs6 -
10 ye
ars
11 - 1
5 yea
rs16
- 20 y
ears
Not App
licable NA
Years of employment Years of employment experience of studentsexperience of students
What are the program’s strengths?
The instructor cadre. Currently 91 instructors (84 at time of survey)Headquarters staffTechnical quality of the curriculaSupport from General PresidentGuidance from Advisory BoardCoordination with Safety and Health staff
What are the program’s What are the program’s weaknesses?weaknesses?
Feedback to instructorsQuality Assessment Reports are generated by headquarters from student evaluations, but provision to instructors needs to be more consistent and timely
What are the program’s What are the program’s weaknesses?weaknesses?
Annual evaluation of instructors.A point for discussion by all Awardees:
“The annual review of instructor competency shall include, at a minimum, observation by the Training Director or his or her designee of instructional delivery, review and discussion of observations with the instructor, and an analysis of the instructor performance based upon evaluations completed by trainees during the previous year.”
Minimum Criteria guidance (9.3.3)
Frontline SafetyFrontline SafetyCurriculum development meetingCurriculum development meeting
September 21, 2006September 21, 2006
Is the evaluation tool current and appropriate for program
content?
Student evaluation form unchanged in 8 yearsExcellent and consistent efforts by Alexander Cohen, Ph.D.New efforts underway by Dr. Harold Stolovitch look quite promising
Harold Stolovitch in venturing into Kirkpatrick Level 4
evaluations
Blended LearningBlended Learning
IAFF has a talented individual leading the charge: Vilma Perez-AtwoodIAFF has created impressive web-based courses, such as the Avian Flu unitGiven the importance of skills training, not as much emphasis on web-based training as other granteesAvailability of computers and access to web still issues
IAFF NearIAFF Near--Miss ProgramMiss Program
IAFF NearIAFF Near--Miss ProgramMiss Program
An important program focused on an area that can have significant impact– OSHA insists on a near-miss program in
VPP– DOE’s Lessons Learned culture has made
a differenceReaching out to the airline industry is bold and can pay real dividends
Do you enjoy being an Do you enjoy being an instructor in the IAFF program?instructor in the IAFF program?
“EXTREMELY! Any of the problems/challenges outlined here are normal organizational stuff. They are
overshadowed by the chance to share with fellow responders in their own
environment. You really get the feeling (most of the time) that you’re making a positive difference for these students
and that’s what it’s all about .”
Questions?Questions?
top related