22
ObjectivesObjectives
Review Means and Methods Review Means and Methods Employed to Evaluate & Document Employed to Evaluate & Document LBFD Recruits.LBFD Recruits.
Introduce Crew Resource Introduce Crew Resource Management (CRM) as an Evaluation Management (CRM) as an Evaluation Tool. Tool.
44
1. Cognitive / Head1. Cognitive / Head Validated / Peer Reviewed ExamsValidated / Peer Reviewed Exams IP SystemIP System
2. Psychomotor / Hand2. Psychomotor / Hand Realistic Manipulative TestingRealistic Manipulative Testing IP SystemIP System
3. Affective / Heart3. Affective / Heart Multiple Simulated EmergenciesMultiple Simulated Emergencies Artificial StressorsArtificial Stressors CRMCRM IP SystemIP System
55
4. Data Collection4. Data Collection&&
Continual Peer Review. Continual Peer Review.
““If they’re not If they’re not learning, learning,
we’re we’re not teaching.”not teaching.”
77
What are Improvement PointsWhat are Improvement Points&&
why do you need them?why do you need them?
Consequence based quantitative Consequence based quantitative feedback measurements to indicate feedback measurements to indicate the appropriate level of improvement the appropriate level of improvement needed to achieve competency. needed to achieve competency.
88
Key pointsKey points
IP’s measure degrees below IP’s measure degrees below competency established by validated competency established by validated standards.standards. Do not describe as Do not describe as “failing a test”.“failing a test”.
Develop Develop “Major”“Major” and and “Minor” “Minor” sliding sliding scales.scales. Error tolerantError tolerant
99
Key Points (cont.)Key Points (cont.)
Regular individual status reports.Regular individual status reports. Not for public distribution. Not for public distribution.
Key tracking tool.Key tracking tool. IndividualIndividual TopicTopic AcademiesAcademies
1010
If disparity is noted:If disparity is noted:
? Curriculum? Curriculum ? Teacher? Teacher ? Student? Student
Minimize variables caused by the first two.
1111
1.1.Cognitive / HeadCognitive / Head
Validated / Peer Validated / Peer Reviewed ExamsReviewed Exams
1212
ExamsExams
Peer Review During DevelopmentPeer Review During Development Confirm ValidityConfirm Validity Confirm CurriculumConfirm Curriculum
Reference CheckReference Check Post Test AnalysisPost Test Analysis
CritiqueCritique Item AnalysisItem Analysis
1313
Date: November 1, 2004
To: Training Staff
From: S. Raganold, Captain
Subject: RECRUIT ACADEMY 2004 BLOCK 4 QUIZ & EXAM STAFF REVIEW
Enclosed within this package are three documents for your review:
Block 4 Exam Reading List Quiz 8 Outcomes Block 4 Exam Outcomes
Please review, comment and note compliance on the table below:
Name Date Initial Buchanan Grego Jackson, D. Rexwinkle Rohr Samuelson Segura Sherrat
Only one copy of the Quiz and Exam exist at this time. They are to be considered as confidential.
The City of Long Beach
Fire Department
RECRUIT TRAINING MEMORANDUM
1414
Name: ________________________________________________
LONG BEACH FIRE DEPARTMENT RECRUIT TRAINING
Recruit Academy 2004A
April 12, 2005
DDOO NNOOTT UUSSEE TTEESSTT BBOOOOKKLLEETT.. NNAAMMEE OONNLLYY.. Answer Sheet Provided.
PROBATIONARY EXAM 1
1515
Post Test AnalysisPost Test AnalysisApril 12, 2005 2004 A Proficiency 1
Nam
e
# C
orr
ect
# In
co
rre
ct
%
Bayley, J. 113 7 94.2% Borders, I. 111 9 92.5% Chan, A. 112 8 93.3% Chang, M. 114 6 95.0% Demetropolis, A. 112 8 93.3% Farley, J. 117 3 97.5% Healy, B. 113 7 94.2% Hopkins, D. 113 7 94.2% Johnson, J. 109 11 90.8% Johnson, T. 115 5 95.8% Knapp, K. 112 8 93.3% Lawrence, M. 114 6 95.0% Lee, J. 113 7 94.2% Magana, M. 112 8 93.3% Marantz, N. 110 10 91.7% Marshal, C. 111 9 92.5% Oh, D. 116 4 96.7% Rindone, K. 114 6 95.0% Rose, K. 112 8 93.3% Rosenstein, J. 117 3 97.5% Van Egdom, M. 118 2 98.3%
P 1 Total
120 Totals
2378 142
Average 113.2 6.8 94.3%
Median 113.0 7.0 94.2%
StDev 2.3 2.3 1.9%
1616
2004A Probationary Exam 1 Question Outcomes
? # -# Previous Use
? # # Previous Use
? # -# Previous Use
? # -# Previous Use
1. 0 P 31. 9 N 61. 0 P 91. 1 P 2. 0 B2/78 32. 0 B3/63 62. 0 P 92. 0 P 3. 0 P 33. 2 B1/35 63. 0 P 93. 0 P 4. 0 B2/78 34. 1 Q5/6 64. 0 P 94. 0 P 5. 0 P 35. P 65. 0 P 95. 1 P 6. 0 B1/7 36. 1 Q5/11 66. 0 P 96. 1 P 7. 0 B1/56 37. 0 Q5/2 67. 7 P 97. 0 P 8. 0 P 38. 0 Q5/7,B1/37 68. 0 P 98. 2 P 9. 1 P 39. 0 Q5/8 69. 0 P 99. 2 P 10. 0 P 40. 0 Q5/1,B1/34 70. 1 P 100. 0 P 11. 0 B1/64 41. 1 P 71. 4 N 101. 1 P 12. 0 B1/58 42. 1 B1/19 72. 1 P 102. 5 N 13. 0 B1/62 43. 1 B1/15 73. 0 P 103. 0 P 14. 0 B1/65 44. 1 B1/18 74. 8 P 104. 0 P 15. 2 N 45. 0 P 75. 13 P 105. 0 P 16. 0 B1/59 46. 1 B2/74 76. 0 P 106. 0 P 17. 0 B1/60 47. 0 B2/73 77. 1 P 107. 0 P 18. 0 B1/63 48. 0 N 78. 0 P 108. 19. 0 N 49. 0 P 79. 0 P 109. 20. 0 P 50. 5 B2/75 80. 1 P 110.
1 P
21. 0 P 51. 1 B2/71 81. 0 P 111. 0 N 22. 0 P 52. 0 Q6/9 82. 1 P 112. 10 N 23. 0 P 53. 7 N 83. 2 P 113. 24. 0 P 54. 0 P 84. 2 P 114. 25. 0 P 55. 0 Q6/8 85. 6 P 115. 26. 0 B1/22 56. 0 P 86. 1 P 116. 27. 0 B1/23 57. 0 P 87. 0 P 117. 28. 0 P 58. 0 P 88. 0 P 118. 29. 0 P 59. 9 P 89. 0 P 119. 30. 0 B3/104 60. 0 P 90. 2 P 120.
24 N
Legend: B2/78 = Previously used in 2004A Block 2 Exam, question number 78.
P = Previously used in past Probationary Exam(s). N = New Question
Previously used questions may have been edited for this exam, if references were not accurate or if the answers were poorly formatted The substance remained consistent with the original.
Red font denotes questions used in Item Analysis.
1717
2004A Probationary Exam 1 Student Outcomes # Correct #
+95. Fail +96.
80%
+97. +98. +99. +100. +101. +102. +103. +104. +105. +106. +107. +108. +109. 1 +110. 1 +111. 2 +112. 5 +113. 4 +114. 3 +115. 1 +116. 1 +117. 2 +118. 1 +119. +120.
2005 A EMT Final
Day 15- 9-29-05
1 10
2 2
1
2
4
2
1
2
11 10
3
0
2
10 00 0 0 0
10 0 0 00 0
0
1
2
3
4
5
# Cor rect
2005 A EMT Final
Day 15- 9-29-05
1 10
2 2
1
2
4
2
1
2
11 10
3
0
2
10 00 0 0 0
10 0 0 00 0
0
1
2
3
4
5
# Cor rect
2005 A OPS 3
Day 19 - 10-6-05
97 6
4 2 0 0 00
5
10
15
20
25
30 29 28 27 26 25 24 Fail
# Cor rect
2005 A OPS 4
Day 21 - 10-10-05
1
8 75 4 1 1 1
0
5
10
15
20
25
30 29 28 27 26 25 24 Fail
# Cor rect
2005 A Block 2Day 33 - 10-31-05
0 0 2 0 1 0 2 24
3 0 2 0 3 1 0 2 0 1 1 0 20
5
10
15
20
25
# Correct
2005 A OPS 6Day 37 - 11-7-05
3
9
52 2 0 1 0
0
5
10
15
20
25
30 29 28 27 26 25 24 Fail# Correct
# St
uden
ts
1818
Partial Item Analysis High student scores vs. high incidents incorrect answers.
Question # / Amount Incorrect Probationary Exam 1 2004A 31 59 74 75 12
Student Outcome -9 -9 -8 -13 -10 Van Egdom 118 + + + + Farley 117 + + + + Rosenstein 117 + + + + + Oh, D. 116 + + + + + Johnson, T. 115 + +
- Incorrect / + - Correct
Quiz Review – Done individually
1919
#N
ame
Aca
dem
y IP
T
ota
l
To
tal
Qu
esti
on
s E
MT
To
tal
Wro
ng
E
MT
EM
T %
To
tal
Qu
esti
on
s O
ps
To
tal
Wro
ng
O
ps
Op
s %
To
tal
Aca
dem
y Q
ues
tio
ns
To
tal
Aca
dem
y Q
ues
tio
ns
Wro
ng
Aca
dem
y %
T
ota
l
Tes
t IP
Old
-
To
tal
Tes
t IP
New
-
To
tal
1 Bayley, J. 57 336 20 94.0% 736 59 92.0% 1,072 79 92.6% 0 82 Bohm, E. 54 336 11 96.7% 736 42 94.3% 1,072 53 95.1% 0 43 Borders, I. 50 336 17 94.9% 736 58 92.1% 1,072 75 93.0% 0 84 Chan, A. 38 336 14 95.8% 736 54 92.7% 1,072 68 93.7% 6 135 Chang, M. 83 336 21 93.8% 736 78 89.4% 1,072 99 90.8% 6 156 Dano, L. 28 336 44 86.9% 35 3 91.4% 371 47 87.3% 13 187 De Hart, A. 111 336 18 94.6% 504 43 91.5% 840 61 92.7% 0 58 Demetropolis, A. 33 336 18 94.6% 736 49 93.3% 1,072 67 93.8% 0 69 Farley, J. 50 336 12 96.4% 736 28 96.2% 1,072 40 96.3% 0 1
10 Filson, M. 4 336 4 98.8% 195 2 99.0% 531 6 98.9% 0 011 Healy, B. 72 336 34 89.9% 736 86 88.3% 1,072 120 88.8% 6 1612 Hopkins, D. 86 336 17 94.9% 736 70 90.5% 1,072 87 91.9% 6 1313 Johnson, J. 53 336 25 92.6% 736 64 91.3% 1,072 89 91.7% 6 1214 Johnson, T. 42 336 13 96.1% 736 50 93.2% 1,072 63 94.1% 0 715 Kepner, K. 113 336 24 92.9% 470 59 87.4% 806 83 89.7% 21 1716 Knapp, K. 32 336 17 94.9% 736 38 94.8% 1,072 55 94.9% 0 317 Lawrence, M. 41 336 15 95.5% 736 40 94.6% 1,072 55 94.9% 0 418 Lee, D. 112 336 52 84.5% 504 61 87.9% 840 113 86.5% 18 3119 Lee, J. 81 336 14 95.8% 736 68 90.8% 1,072 82 92.4% 9 1920 Magana, M. 53 336 9 97.3% 736 82 88.9% 1,072 91 91.5% 21 2121 Marantz, N. 62 336 21 93.8% 736 51 93.1% 1,072 72 93.3% 6 1322 Marshal, C. 43 336 15 95.5% 736 69 90.6% 1,072 84 92.2% 0 1023 Oh, D. 34 336 12 96.4% 736 44 94.0% 1,072 56 94.8% 0 324 Rindone, K. 74 336 10 97.0% 736 68 90.8% 1,072 78 92.7% 0 1425 Rose, K. 76 336 12 96.4% 736 104 85.9% 1,072 116 89.2% 48 4226 Rosenstein, J. 65 336 14 95.8% 736 62 91.6% 1,072 76 92.9% 0 927 Van Egdom, M. 63 336 9 97.3% 736 32 95.7% 1,072 41 96.2% 0 228 White, M. 138 336 31 90.8% 470 91 80.6% 806 122 84.9% 52 45
Average 56.57 336 16.14 95.20% 736 59.71 91.89% 1072 75.86 92.92% 5.43 11.38
Median 53.00 336 15.00 95.54% 736 59.00 91.98% 1072 76.00 92.91% 0.00 10.00
StDev 17.30 0 5.84 1.74% 0 18.76 2.55% 0 21.01 1.96% 11.01 8.96
Average 92.67 336 30.00 91.07% 453.17 49.83 88.86% 789.17 79.83 89.37% 17.33 20.00Median 111.50 336 27.50 91.82% 487.00 51.00 89.66% 823.00 72.00 88.52% 15.50 17.50StDev 42.08 0 15.63 4.65% 228.29 29.00 4.76% 228.29 31.75 3.90% 19.16 15.75
Pas
sed
359218Written IP TotalsTotals
Fai
l
2020
Exercise 1Exercise 1
Write a Multiple Choice Question.Write a Multiple Choice Question. Parameters:Parameters:
– ““Real World” SituationalReal World” Situational– ValidValid
To standardsTo standards To student levelTo student level
– Well ReferencedWell Referenced
2121
2. 2. Psychomotor / HandPsychomotor / Hand
Realistic Manipulative TestingRealistic Manipulative Testing IP SystemIP System
Consequence Based Sliding ScaleConsequence Based Sliding Scale
2323
Grade SheetGrade Sheet
OBJECTIVE: Measurement of Recruit capability to complete inverted “L” cutting technique on a roll-up door and disable hinges with a gasoline powered rotary saw and metal cutting blade. SCENARIO: Recruit is member of first arriving truck crew at a commercial building fire, one roll-up door and an outward swinging conventional hinged door need to be rapidly failed to facilitate engine crew entry. PROPS & EQUIPMENT NEEDED:
1. Roll-up door simulator with available material. (Each student will use similar material for the test.) 2. Available rotary saw, either K-12 or 950, inboard or outboard, with new blade. The student is to attach blade
and test the specific saw used in the test. (Note the occasional difficulty with the 950 flange.) 3. Three sections of heavy gauge flat-steel (4” vertically) attached at points next to roll-up door to simulate hinges
(ends exposed minimum of 2" horizontally). 4. Full P.P.E. for Recruits, including SCBA.
ESTABLISH TESTING ENVIRONMENT:
1. The entire inverted “L” maneuver is not done, but will be demonstrated after the manipulative phase of the test. 2. Available “Cut Lines” are identified by the instructor. The Student will identify the specific line used for the
test. a. Horizontal cut minimum (5’) long above shoulder height. b. Vertical cut minimum (4’)
3. “Hinge” cuts are then cut in order – lowest, middle, highest. 4. Start time when instructor tells Recruit to lock-in regulator 5. End time when Recruit stops saw and disconnects regulator.
6. Inverted “L” maneuver is explained to the instructor / drawn on dry erase board. GRADING CRITERIA: 2-Improvement Points
1. Poor feed speed, RPM’s decrease 2. Horizontal cut below shoulder height 3. Each occurrence of poor control of saw. (i.e., bouncing, delay in starting cut) 4. Inverted “L” drawing incorrect (direction / numbering)
5-Improvement Points (AND FAILURE OF EVENT)
1. Incomplete cut on roll-up door. 2. Incorrect hinge cutting sequence. 3. Lateral or side loading of blade. (Indicated by diagonal, notched or offset kerf or blade fragmentation.) 4. Using building / wall to stabilize self / saw during hinge cuts 5. Inability to accomplish task within 4-minutes 6. Unable to explain the inverted “L” process 7. Unsafe acts / inability to control saw. 8. Inability to start saw 9. Cut begins / extends 6” beyond indicated points.
2424
TEST SEQUENCE 1._____ Instructor reviews criteria 2._____ Recruit in full P.P.E. - regulator not locked in 3._____ Recruit checks equipment, starts and stops saw after saw is warm. 4._____ Start time is when instructor tells student to lock in regulator 5._____ Recruit starts saw, makes (minimum 5’) horizontal cut at shoulder height 6._____ Recruit makes (minimum 4’) vertical inverted “L” cut 7._____ Recruit cuts 3 “hinges” (bottom, middle, top) 8._____ Saw is stopped by Recruit following completion of all cuts. 9._____ Regulator is disconnected from face piece. This is evolution stop time 10.____ Explanation (verbal/drawing) of inverted “L” PASS/FAIL: ________________ TIME: _____________________ TOTAL IMPROVEMENT POINTS: _______________ EVALUATOR: ______________________________ COMMENTS: _______________________________ ____________________________________________ _____________________________________________ Tape number and time:____________________________
3131
Day
16-
PM
- 1
1/14
/200
3 (N
o S
CB
A)
Day
17
-AM
- 11
/17/
03 (
SC
BA
)
Day
18
- A
M -
11/
18/0
3 (S
CB
A)
Day
18
- P
M -
11
/18/
03 (
SC
BA
)
Day
19
- A
M -
11/
20/0
3 T
ES
T
PO
INT
S
Barron, Stephen 83 122 109 102 102
Girard, Cory 73 110 89 93 74
Hogan, J oshua 100 120 111 92 92
K opstein, Paul 105 206 132 118 137
Medina, Henry 135 190 127 132 117 2
Nicoll, Thomas 133 80 105 110 109
Scott, K evin 78 118 113 115 112
Strecker, Daren 71 124 168 111 98
Wirtz, Mike 72 90 79 82 78
Mean 94.44 128.89 114.78 106.11 102.11
Standard Dev. 25.47 42.14 25.91 15.41 19.61
Median 83.00 120.00 111.00 110.00 102.00
All tests done with Sheet Curtatin segments. First introduction to skill - AM Rotation, Day16. First timed evolution noted above. First rotation done to test format, Day 17-AM.
No PM Rotation on Day 17.
2.00
3232
Exercise 2Exercise 2
Develop a manipulative test.Develop a manipulative test. Parameters:Parameters:
– ““Real World” SituationalReal World” Situational– ValidValid
To standardsTo standards To student levelTo student level
– Well ReferencedWell Referenced
3333
Grade SheetGrade Sheet
OBJECTIVE: Measurement of Recruit capability to complete inverted “L” cutting technique on a roll-up door and disable hinges with a gasoline powered rotary saw and metal cutting blade. SCENARIO: Recruit is member of first arriving truck crew at a commercial building fire, one roll-up door and an outward swinging conventional hinged door need to be rapidly failed to facilitate engine crew entry. PROPS & EQUIPMENT NEEDED:
1. Roll-up door simulator with available material. (Each student will use similar material for the test.) 2. Available rotary saw, either K-12 or 950, inboard or outboard, with new blade. The student is to attach blade
and test the specific saw used in the test. (Note the occasional difficulty with the 950 flange.) 3. Three sections of heavy gauge flat-steel (4” vertically) attached at points next to roll-up door to simulate hinges
(ends exposed minimum of 2" horizontally). 4. Full P.P.E. for Recruits, including SCBA.
ESTABLISH TESTING ENVIRONMENT:
1. The entire inverted “L” maneuver is not done, but will be demonstrated after the manipulative phase of the test. 2. Available “Cut Lines” are identified by the instructor. The Student will identify the specific line used for the
test. a. Horizontal cut minimum (5’) long above shoulder height. b. Vertical cut minimum (4’)
3. “Hinge” cuts are then cut in order – lowest, middle, highest. 4. Start time when instructor tells Recruit to lock-in regulator 5. End time when Recruit stops saw and disconnects regulator.
6. Inverted “L” maneuver is explained to the instructor / drawn on dry erase board. GRADING CRITERIA: 2-Improvement Points
1. Poor feed speed, RPM’s decrease 2. Horizontal cut below shoulder height 3. Each occurrence of poor control of saw. (i.e., bouncing, delay in starting cut) 4. Inverted “L” drawing incorrect (direction / numbering)
5-Improvement Points (AND FAILURE OF EVENT)
1. Incomplete cut on roll-up door. 2. Incorrect hinge cutting sequence. 3. Lateral or side loading of blade. (Indicated by diagonal, notched or offset kerf or blade fragmentation.) 4. Using building / wall to stabilize self / saw during hinge cuts 5. Inability to accomplish task within 4-minutes 6. Unable to explain the inverted “L” process 7. Unsafe acts / inability to control saw. 8. Inability to start saw 9. Cut begins / extends 6” beyond indicated points.
3434
3.3.Affective / HeartAffective / Heart
Multiple Simulated EmergenciesMultiple Simulated Emergencies Artificial StressorsArtificial Stressors ““Non-Exam” RubricNon-Exam” Rubric
IP SystemIP System Multiple Formal Counseling Multiple Formal Counseling
SessionsSessions Crew Resource ManagementCrew Resource Management
Attitudinal / Behavioral RubricAttitudinal / Behavioral Rubric
3535
The “attitudinal standard” The “attitudinal standard” isis
“Introduction to Recruit “Introduction to Recruit Fire Training”Fire Training”
Acts as a Training Contract.Acts as a Training Contract. Referenced throughout Academy.Referenced throughout Academy. Students tested early to establish Students tested early to establish
level of understanding. level of understanding.
3636
The "offer of employment" letter sent to Recruits in October of 1940 by Chief A. C. Duree contains a description of professional behavior that rings true today.
"You have been selected from the eligible list for appointment to the position of firefighter on this department. The fact that you passed your examination and placed on the list shows you were anxious to secure this position. I want you to understand that it is up to you to make your position here secure. You will be furnished with the necessary knowledge and material required for the duties of firefighter, and it will be up to you to hold the job. [Emphasis added]
3838
1 0 0 0 0 0 0 0 0 0 0 0 0 0 02 0 0 0 0 0 0 0 0 0 0 0 0 0 03 0 0 0 0 0 0 0 0 0 0 0 0 0 04 0 0 0 0 0 0 0 0 0 0 0 0 0 05 0 0 0 0 0 0 0 0 0 0 0 0 0 06 0 0 0 0 0 0 0 0 0 0 0 0 0 07 0 0 0 0 0 0 0 0 0 0 0 0 0 08 0 0 0 0 0 0 0 0 0 0 0 0 0 09 0 0 0 0 0 0 0 0 0 0 0 0 0 0
10 0 0 0 0 0 0 0 0 0 0 0 0 0 011 0 0 0 0 0 0 0 0 0 0 0 0 0 012 0 0 0 0 0 0 0 0 0 0 0 0 0 013 0 0 0 0 0 0 0 0 0 0 0 0 0 014 0 0 0 0 0 0 0 0 0 0 0 0 0 015 0 0 0 0 0 0 0 0 0 0 0 0 0 016 0 0 0 0 0 0 0 0 0 0 0 0 0 017 0 0 0 0 0 0 0 0 0 0 0 0 0 018 0 0 0 0 0 0 0 0 0 0 0 0 0 019 0 0 0 0 0 0 0 0 0 0 0 0 0 020 0 0 0 0 0 0 0 0 0 0 0 0 0 021 0 0 0 0 0 0 0 0 0 0 0 0 0 022 0 0 0 0 0 0 0 0 0 0 0 0 0 023 0 0 0 0 0 0 0 0 0 0 0 0 0 024 0 0 0 0 0 0 0 0 0 0 0 0 0 025 0 0 0 0 0 0 0 0 0 0 0 0 0 026 0 0 0 0 0 0 0 0 0 0 0 0 0 027 0 0 0 0 0 0 0 0 0 0 0 0 0 028 0 0 0 0 0 0 0 0 0 0 0 0 0 029 0 0 0 0 0 0 0 0 0 0 0 0 0 030 0 0 0 0 0 0 0 0 0 0 0 0 0 031 0 0 0 0 0 0 0 0 0 0 0 0 0 032 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
P -
Pro
pan
e
To
tal N
ozz
le
H -
F E
Hal
igan
D-
So
un
d
S -
Saf
ety
V-
Veh
icle
D -
Deb
ris
T -
To
wer
W -
Saw
Totals
5 -
Cap
tain
E -
En
gin
eer
3 -
Hyd
ran
t
R -
F E
Ro
tary
Nozzle Positions
0 -
O'C
ady
T -
To
wer
P -
Pro
pan
e
To
tal N
ozz
le
0 -
O'C
ady
S -
Saf
ety
V-
Veh
icle
H -
F E
Hal
igan
D-
So
un
d
W -
Saw
Nozzle Positions
5 -
Cap
tain
E -
En
gin
eer
3 -
Hyd
ran
t
R -
F E
Ro
tary
D -
Deb
ris
3939
““Non-Exam” RubricNon-Exam” Rubric
Provides descriptive consistency.Provides descriptive consistency. Consequence based sliding scale.Consequence based sliding scale.
4040
Point Allocation
S Safety
D Delay
ET Efficiency & Team Work
DF Drill Format Compliance
EF Error & Frequency Prompt Response
EK Equipment Knowledge & Use
0
Routine good and safe practice.
Actions cause no delay in any operation
Efficient and optimal performance.
Competent the grade sheet parameters
No observable errors. Requests Feedback.
Detailed knowledge. Good use of maint. & Storage
1
No potential injury to self or others, but poor practice.
Minor delay Self corrected
Efficient, but less then optimal. Hesitation is observed.
Some minor elements self corrected during the evolution.
Few minor errors. Student readily recognizes errors.
Minor specifications are not known or acted on
2
Potential for minor injury, to self or others, if un- corrected.
Minor delay Requires Prompting
Minimal efficiency. Occasional extra movements. Poor recognition of needs of others
Multiple minor elements major self-correction.
Noticeable minor errors with quick recognition.
General equip. specifications are not known or acted on.
3
Potential for moderate injury, to self or others, if not corrected
Major delay Requires Prompting.
Inefficient. Wasted Movements. Requires prompts.
Minor elements are overlooked. Potential for delay.
Multiple errors Student requires explanation to recognize errors.
Poor usage of equipment maintenance, storage
4
Potential for major injury, to self or others. Some intervention.
Major delay No attempt to correct.
Inefficient. Delays in operation. Requires prompts.
Major elements are ignored. Potential of objective failure.
Multiple errors, explanation of the underlying reason
Requires major prompting to operate & use.
5
Instructor intervention required to prevent injury.
Serious delay and/or failure of the operation.
“Fumbling”, multiple redirects. Acts in an isolated manner.
Behavior that causes the failure of the evolution’s objectives.
Argumentative Poor response to prompting.
Unable to operate or potential for damage.
4141
Why?Why? Manipulative Exams are a Contrivance.Manipulative Exams are a Contrivance. Needed Consistent Means to Measure Needed Consistent Means to Measure
Other Events.Other Events. Rubric Provides:Rubric Provides:
Documentation LanguageDocumentation Language Context (Consequence Based)Context (Consequence Based) Enhanced Peer Review ProcessEnhanced Peer Review Process
““A work in progress.”A work in progress.”
4242
Exercise 3Exercise 3
Use the “Non-Exam” RubricUse the “Non-Exam” Rubric Document Justification for Document Justification for
Improvement Points:Improvement Points:– Only Use a Few SentencesOnly Use a Few Sentences– Set EnvironmentSet Environment– Describe ConsequencesDescribe Consequences– Allocate Points Allocate Points – Base Example on Personal ExperienceBase Example on Personal Experience
4343
““Non-Exam” RubricNon-Exam” Rubric
Point Allocation
S Safety
D Delay
ET Efficiency & Team Work
DF Drill Format Compliance
EF Error & Frequency Prompt Response
EK Equipment Knowledge & Use
0
Routine good and safe practice.
Actions cause no delay in any operation
Efficient and optimal performance.
Competent the grade sheet parameters
No observable errors. Requests Feedback.
Detailed knowledge. Good use of maint. & Storage
1
No potential injury to self or others, but poor practice.
Minor delay Self corrected
Efficient, but less then optimal. Hesitation is observed.
Some minor elements self corrected during the evolution.
Few minor errors. Student readily recognizes errors.
Minor specifications are not known or acted on
2
Potential for minor injury, to self or others, if un- corrected.
Minor delay Requires Prompting
Minimal efficiency. Occasional extra movements. Poor recognition of needs of others
Multiple minor elements major self-correction.
Noticeable minor errors with quick recognition.
General equip. specifications are not known or acted on.
3
Potential for moderate injury, to self or others, if not corrected
Major delay Requires Prompting.
Inefficient. Wasted Movements. Requires prompts.
Minor elements are overlooked. Potential for delay.
Multiple errors Student requires explanation to recognize errors.
Poor usage of equipment maintenance, storage
4
Potential for major injury, to self or others. Some intervention.
Major delay No attempt to correct.
Inefficient. Delays in operation. Requires prompts.
Major elements are ignored. Potential of objective failure.
Multiple errors, explanation of the underlying reason
Requires major prompting to operate & use.
5
Instructor intervention required to prevent injury.
Serious delay and/or failure of the operation.
“Fumbling”, multiple redirects. Acts in an isolated manner.
Behavior that causes the failure of the evolution’s objectives.
Argumentative Poor response to prompting.
Unable to operate or potential for damage.
4444
ReviewReview
Established academic parameters.Established academic parameters. HeadHead
Established basic manipulative Established basic manipulative standards.standards. HandHand
Began operational capability Began operational capability assessment.assessment. Heart?Heart?
4545
This is sufficient for most This is sufficient for most Students most of the time.Students most of the time.
However…..However…..
4747
Adaptability /
Flexibility (AF)
Assertiveness (AS)
Communication (CM)
Decision Making (DM)
Leadership (LD)
Mission Analysis (MA)
Situational Awareness (SA)
O
uts
tan
din
g ► Determine if SOP
is appropriate ► Offers and advocates for alternative solutions ►Quick recovery after redirection
►Appropriate advocacy of alternatives. ►Plans ahead ►Provides useful information ►Drives events
►Utilizes multiple communication forms ►Recognizes and overcomes communication filters
►Utilizes “OODA Loop” in analysis and implementation of problem solving strategies
►Acts as, and is accepted as, a functional / situational leader ►Excels in fluid environment
►Effective utilization of pre and post incident analysis. ►Develops multiple feasible courses of action
►Maintains control ►Timely assessment ►Multiple information sources ►Monitors results ►Able to rapidly alter tactics
Op
tim
al ► Anticipates
problems ► Identifies change and asks for assistance
►Communicates if comfort level is exceeded ►Establishes dialogue to offset group mindset
►Active listener ►Paraphrases feedback ►Succinct transfer of complex information ►Corrects misunderstandings
►Verifies information ►C omplete assessment of new problem / situation ►Develops a hypothesis
As a designated leader: ►Decisive ►P rofessional ►Provides feedback ►Leads by example
►Effective multi-element planning ►Able to monitor and respond to changing mission parameters ►Identifies actual and potential problems
Occasionally succumbs to: ►Fatigue / Stress ►Task overload ►Group minds et
No
min
al The ability to alter a
course of action when new information becomes available.
The willingness & readiness to actively participate, state and maintain a position, until convinced by alternate facts.
The clear and accurate sending and receiving of information, instructions, or commands, and providing useful feedback.
The ability to use logical and sound judgment; to make decisions based on available information. Recognizes Error Troika.
The ability to direct and coordinate the activities of others and to encourage the crew to work together as a team.
The ability to develop short term, long-term and contingency plans, as well as to coordinate, allocate and monitor crew and resources.
The degree of accuracy by which one's perception of his / her current environment mirrors reality.
Nee
ds
Imp
rov
eme
nt ► Requires specific
prompting to develop appropriate response to changing environment and/or task.
►Impulsive ►Resignation ►Avoids conflicts ►Overly courteous
►Limited feedback ►Poor co nveyance ►I nappropriate timing ►No recognition of misunderstandings
►Dependant on SOP’s ►Does not cross check information ►Unable to define rational for decision
►Passive ►Does not delegate ►Limited information transfer
Does not routinely utilize opportunities for: ►Planning ►Preparation ►Pre & Post incident briefs
Identifies situational elements but is unable to: ►Prioritize ►Assign values ►Respond to changes
Un
sat
isfa
cto
ry
► Unable to respond to changing environment and/or task. ►Does not respond to redirection
►Anti-authority ►Abusive / Hostile ►Invulnerability ►Macho
►No feedback ►Little or no dialogue ►Inaccurate ►Inappropriate modalities
►Dependant on specific and clear direction prior to acting on simple tasks. ►Blames others
►Divisive ►Overbearin g ►Arbitrary ►Demeaning ►Lets others take charge ►Places others at risk
►Requires close direction ►Not able to define tasks or resource allocation ►Ignores problems
►Misinterprets or confuses key situational parameters ►Unable to determine cause and effect ►Places self at risk
4949
Why?Why?
““Head & Hand” Methods Well EstablishedHead & Hand” Methods Well Established Poorly Defined Acceptable Attitudinal / Poorly Defined Acceptable Attitudinal /
Behavioral Parameters.Behavioral Parameters. ““That was weird!”That was weird!”
Uses:Uses: Documentation Tool / No PointsDocumentation Tool / No Points Help Define “Intangibles”Help Define “Intangibles” Context for Peer ReviewContext for Peer Review
““A work in progress.”A work in progress.”
5151
IP Collection (Student / Points)IP Collection (Student / Points)
M & Q1 M2 M3 M4 & Q2 M5 M6 M7 M8 M9+ Q3 M Final Q4 Q5 B1 Q6 B2 Q7 B3 Q8 B4 Q9 B5Day 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 7 Day 8 Day 9 Day 10 Day 11 Day 12 Day 13 Day 14 Day 15 Day 16 Day 17 Day 18 Day 19 Day 20 Day 21 Day 22 Day 23 Day 24 Day 25 Day 26 Day 27 Day 28 Day 29 Day 30 Day 31 Day 32 Day 33 Day 34 Day 35 Day 36 Day 37 Day 38 Day 39 Day 40 Day 41 Day 42 Day 43 Day 44 Day 45 Day 46 Day 47 Day 48 Day 49 Day 50 Day 51 Day 52 Day 53 Day 54 Day 55 Day 56Tues Wed Thurs Fri Mon Tues Thurs Fri Mon Tues Thurs Fri Mon Tues Thurs Fri Mon Tues Thurs Fri Mon Tues Thurs Fri Mon Tues Thurs Fri Mon Tues Thurs Fri Mon Tues Thurs Fri Mon Tues Thurs Fri Mon Tues Thurs Fri Mon Tues Wed Fri Mon Tues Thurs Fri Mon Tues Thurs Fri
09/07/04 09/08/04 09/09/04 09/10/04 09/13/04 09/14/04 09/16/04 09/17/04 09/20/04 09/21/04 09/23/04 09/24/04 09/27/04 09/28/04 09/30/04 10/01/04 10/04/04 10/05/04 10/07/04 10/08/04 10/11/04 10/12/04 10/14/04 10/15/04 10/18/04 10/19/04 10/21/04 10/22/04 10/25/04 10/26/04 10/28/04 10/29/04 11/01/04 11/02/04 11/04/04 11/05/04 11/08/04 11/09/04 11/11/04 11/12/04 11/15/04 11/16/04 11/18/04 11/19/04 11/22/04 11/23/04 11/24/04 11/26/04 11/29/04 11/30/04 12/02/04 12/03/04 12/06/04 12/07/04 12/09/04 12/10/04
Bayley, J. 57 10 1 1 1 1 3 1 25 4 3 7Bohm, E. 54 1 5 4 1 1 1 9 4 13 3 1 2 2 7Borders, I. 50 5 1 3 9 3 2 10 1 7 9Chan, A. 38 5 2 5 1 1 6 1 3 2 6 6Chang, M. 83 9 3 6 1 12 2 2 3 24 5 15 1
Dano, L. 28 4 4 10 10De Hart, A. 111 15 5 1 3 3 4 1 27 2 2 2 28 5 6 7Demetropolis, A. 33 10 9 1 1 10 2Farley, J. 50 10 4 5 9 3 15 1 1 2Filson, M. 4 4
Healy, B. 72 5 4 6 1 1 2 6 5 6 9 6 18 3Hopkins, D. 86 4 10 10 2 2 13 5 6 17 6 2 5 4Johnson, J. 53 10 1 1 9 1 1 3 18 1 5 3Johnson, T. 42 3 1 6 2 16 2 7 5Kepner, K. 113 15 6 2 1 2 22 2 9 3 51
Knapp, K. 32 2 6 2 15 2 5Lawrence, M. 41 1 1 11 8 9 7 4Lee, D. 112 4 4 4 5 5 1 6 16 1 17 5 4 2 35 3Lee, J. 81 3 10 1 1 4 4 6 12 3 2 19 3 2 1 1 7 2Magana, M. 53 5 1 2 4 2 5 22 2 1 7 2
Marantz, N. 62 10 1 9 1 6 5 12 1 2 7 8Marshal, C. 43 3 6 2 5 2 14 2 9Oh, D. 34 4 1 2 1 1 5 12 4 4Rindone, K. 74 1 1 1 1 12 5 8 6 19 20Rose, K. 76 4 3 1 7 25 2 25 1 6 2
Rosenstein, J. 65 5 5 1 2 1 9 18 4 1 14 2 3Van Egdom, M. 63 3 1 1 18 4 4 2 10 8 5 7White, M. 138 4 10 6 1 7 2 6 2 3 39 16 5 2 35Points per Day 0 8 12 3 8 0 0 0 15 0 0 150 0 0 7 1 6 2 0 0 113 27 21 10 25 49 14 5 303 1 0 61 31 20 10 16 445 0 0 40 31 14 15 0 151 0 0 0 0 20 23 5 86 0 0 0
TOTAL: 1748 Total 297 Total 747 Total 1331 Total 955 Total 1108 Total 1242 Total 1188AVERAGE 62.43 AV. 11 AV. 28.73 AV. 51.19 AV. 43.41 AV. 50.36 AV. 56.45 AV. 56.57MEDIAN 55.50 STDEV 10.01 MEDIAN 23 MEDIAN 41 MEDIAN 39.5 MEDIAN 47.5 MEDIAN 53.5 MEDIAN 53STDEV 30.10 MEDIAN 9 STDEV 18.43 STDEV 30.76 STDEV 15.11 STDEV 15.72 STDEV 16.89 STDEV 17.3
09/07/04 09/08/04 09/09/04 09/10/04 09/13/04 09/14/04 09/16/04 09/17/04 09/20/04 09/21/04 09/23/04 09/24/04 09/27/04 09/28/04 09/30/04 10/01/04 10/04/04 10/05/04 10/07/04 10/08/04 10/11/04 10/12/04 10/14/04 10/15/04 10/18/04 10/19/04 10/21/04 10/22/04 10/25/04 10/26/04 10/28/04 10/29/04 11/01/04 11/02/04 11/04/04 11/05/04 11/08/04 11/09/04 11/11/04 11/12/04 11/15/04 11/16/04 11/18/04 11/19/04 11/22/04 11/23/04 11/24/04 11/26/04 11/29/04 11/30/04 12/02/04 12/03/04 12/06/04 12/07/04 12/09/04 12/10/04Tues Wed Thurs Fri Mon Tues Thurs Fri Mon Tues Thurs Fri Mon Tues Thurs Fri Mon Tues Thurs Fri Mon Tues Thurs Fri Mon Tues Thurs Fri Mon Tues Thurs Fri Mon Tues Thurs Fri Mon Tues Thurs Fri Mon Tues Thurs Fri Mon Tues Wed Fri Mon Tues Thurs Fri Mon Tues Thurs FriDay 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 7 Day 8 Day 9 Day 10 Day 11 Day 12 Day 13 Day 14 Day 15 Day 16 Day 17 Day 18 Day 19 Day 20 Day 21 Day 22 Day 23 Day 24 Day 25 Day 26 Day 27 Day 28 Day 29 Day 30 Day 31 Day 32 Day 33 Day 34 Day 35 Day 36 Day 37 Day 38 Day 39 Day 40 Day 41 Day 42 Day 43 Day 44 Day 45 Day 46 Day 47 Day 48 Day 49 Day 50 Day 51 Day 52 Day 53 Day 54 Day 55 Day 56
EMT Block 1 Block 2
DateDay
Day #Block Block 4 Block 5 Grad Week
To Day 37 - Successful To Day 44 - SuccessfulTo Day 21 - Successful
Grad WeekBlock 4 Block 5Block 2 Block 3
To Day 45 - Successful To Day 53 - Successful To Day 56 - SuccessfulTo Day 29 - SuccessfulAll Students
LBFD
2004
A Impro
vement
Points
Block 3
EMT Block 1Block
Day #Day Date
Written Tests
5252
Class IP (With)Class IP (With)2004A with Written
11
28.7
3
51.1
9
50.3
6
56.4
5
56.5
7
62.4
3
9
23
41
47.5
53.5 5355.5
10.01
18.43
30.76
15.69
16.89
17.3
30.1
0
10
20
30
40
50
60
70
Day 21 Day 29 Day 37 Day 45 Day 53 Day 56 Total
0.00
10.00
20.00
30.00
40.00
50.00
60.00
70.00
Day 21 Day 29 Day 37 Day 45 Day 53 Day 56 Total
AVERAGE
STDEV
MEDIAN
5353
Class IP Comparison (With)Class IP Comparison (With)IP "With" - 2001A to 2004A
10.5
7
25.9
0
38.7
5
45.7
5
51.4
4
50.7
9
60.3
1
9.79
19.5
9
28.0
6
23.0
3
21.5
2
20.8
1
30.4
6
7.00
21.00
31.00
41.00
49.00 48.50
53.00
0
10
20
30
40
50
60
70
Day 21 Day 29 Day 37 Day 45 Day 53 Day 56 Total
0.00
10.00
20.00
30.00
40.00
50.00
60.00
70.00
Day 21 Day 29 Day 37 Day 45 Day 53 Day 56 Total
AVERAGE
STDEV
MEDIAN
5454
Class IP (Without)Class IP (Without)2004A - Without Written
9.7
24.4
43.62 44
.73
50.82
50.67
54.29
9
21
35.5
43
50 5052
7.11
14.2
23.41
13.84
15.67
16.04
24.88
0
10
20
30
40
50
60
70
Day 21 Day 29 Day 37 Day 45 Day 53 Day 56 Total0.00
10.00
20.00
30.00
40.00
50.00
60.00
70.00
Day 21 Day 29 Day 37 Day 45 Day 53 Day 56 Total
AVERAGE
STDEV
MEDIAN
5555
Class IP Comparison Class IP Comparison (Without)(Without)
IP "Without" - 2001A to 2004A9.
34
22.2
8
36.6
8 40.9
8
46.9
6
46.3
1
55.2
5
7.00
18.00
30.00
37.00
44.00 43.50
49.00
0
10
20
30
40
50
60
70
Day 21 Day 29 Day 37 Day 45 Day 53 Day 56 Total
0.00
10.00
20.00
30.00
40.00
50.00
60.00
70.00
Day 21 Day 29 Day 37 Day 45 Day 53 Day 56 Total
AVERAGE
STDEV
MEDIAN
5757
Academy 2002B 2004A 2005A
Written if new used 198 219
Written 54 8.17% 121 13.18% 342 25.18%
EMT Quiz 12 1.82% 28 3.05% 93 6.85%
EMT Exam 0 0.00% 0 0.00% 43 3.17%
Ops Quiz 42 6.35% 51 5.56% 76 5.60%
Ops Exam 0 0.00% 42 4.58% 130 9.57%
EMT Skills 115 17.40% 165 17.97% 190 13.99%
Ops Skills Tests 411 62.18% 437 47.60% 710 52.28%
Knots 44 6.66% 62 6.75% 42 3.09%
Knots Practice 0 0.00% 0 0.00% 0 0.00%
Ladders 31 4.69% 87 9.48% 26 1.91%
Ladder Practice 0 0.00% 5 0.54% 0 0.00%
Hose lays 134 20.27% 112 12.20% 125 9.20%
Hose lay Practice 12 1.82% 33 3.59% 43 3.17%
SCBA 35 5.30% 13 1.42% 66 4.86%
SCBA Practice 0 0.00% 0 0.00% 0 0.00%
Force Entry / Tools 10 1.51% 1 0.11% 43 3.17%
F E / Tools Practice 0 0.00% 0 0.00% 0 0.00%
Roof 103 15.58% 94 10.24% 310 22.83%
Roof Practice 22 3.33% 21 2.29% 55 4.05%
Auto Ext 0 0.00% 0 0.00% 0 0.00%
Auto Ext. Practice 0 0.00% 0 0.00% 0 0.00%
Skills Misc. (Hose Ad.) 20 3.03% 9 0.98% 0 0.00%
Operations 66 9.98% 185 20.15% 91 6.70%
Knots 0 0.00% 0 0.00% 0 0.00%
Ladders 12 1.82% 27 2.94% 20 1.47%
Hose lays 23 3.48% 28 3.05% 20 1.47%
PPE / SCBA 21 3.18% 12 1.31% 34 2.50%
Force Entry / Tools 0 0.00% 6 0.65% 6 0.44%
Roof 5 0.76% 93 10.13% 5 0.37%
Auto Ext. 0 0.00% 0 0.00% 0 0.00%
Misc. 5 0.76% 19 2.07% 6 0.44%
Misc 15 2.27% 10 1.09% 25 1.84%
IP C
om
par
iso
n a
s o
f P
re-B
lock
3 E
xam
Total 661 918 1,358
Notes: Characterized by training phase and primary function.. (i.e. SCBA donning points accrued during hose lays are typed as a hose lay point.)
Notes: Characterized by type only. (i.e. SCBA points during a live fire are typed as "PPE/SCBA".
5959
CounselingCounseling
Regular Contact with Recruit.Regular Contact with Recruit. Benchmark Points @ 25, 50 & 75 points.Benchmark Points @ 25, 50 & 75 points. Audio Recording / FormalAudio Recording / Formal
Constant Review with Legal.Constant Review with Legal. Established relationship with City Established relationship with City
AttorneyAttorney Early Use of Referrals and Early Use of Referrals and
Constructive Action Plans.Constructive Action Plans.
6161
Regular CounselingRegular Counseling
LONG BEACH FIRE DEPARTMENT RECRUIT TRAINING
COUNSELING FORM
NAME: Recruit Xxxxx DATE: November 15, 2002 LOCATION: Training Center REASON FOR COUNSELING: Accrual of 75 Improvement Points During the block III skills exam Recruit Xxxxx accrued 28 improvement points bringing his total to 93 improvement points. According to The Introduction to Recruit Fire Training, a formal counseling will take place whenever 75 points are accrued. The firefighter's job is one of the most hazardous of all occupations. Recruit firefighters are trained to function in a safe and proficient manner consistent with the aggressive fire fighting nature of the Department. Carelessness in any work routine, drill, or simulated emergency can result in serious injury or death. Therefore, up to five improvement points will be assessed for each safety violation. Repeated failure to comply with Department safety policies, practices and procedures, will result in a recommendation for termination. Recruit Xxxxx has been accumulating improvement points at a rapid rate since starting the operations portion of the Recruit Academy. Recruit Xxxxx has received coaching sessions and formal counseling sessions at 25, and 50 improvement points. In those sessions, among other things, we discussed safety as it applies to both manipulative exercises and simulated emergency responses and attitude. “As a Recruit you are expected to make mistakes while learning new skills. That is why you are in the
Academy. If you make a mistake, admit it, correct it and move on to the next challenge. But, mistakes cannot go on forever.”
Recruit Xxxxx listens attentively during lectures and asks questions pertinent to the subject. He accepts responsibility for his mistakes, listens attentively to corrective coaching and tries to apply the techniques or information in subsequent evolutions. Recruit Xxxxx is to be cautioned that the points he accrued today represent potential delays in evolutions or safety concerns that jeopardize his safety, that of his crew or any potential victims that he may encounter during emergency operations. The consequences of these actions have been emphasized during classroom lectures, NIOSH reports, and “impact” videos. The rapid accrual of improvement points have raised some serious concerns among the Training Staff as to Recruit Xxxxx’s ability to operate safely in the field. Recruit Xxxxx is cautioned that 93 improvement points is a very significant amount at this stage in the academy. Recruit Xxxxx is cautioned that safe, competent performance of skills and satisfactory academic performance is necessary to successfully pass this academy. If 110 improvement points are accrued a recommendation for termination will be forwarded to the Director of Training. I HAVE BEEN COUNSELED ON THIS DATE SIGNATURE COUNSELORS
6262
After all of this, the most After all of this, the most effective tool is…..effective tool is…..
6565
Attitudinal MeasurementAttitudinal Measurement
OxymoronOxymoron Can Measure Can Measure BehaviorBehavior, not , not AttitudeAttitude..
Consider Crew Resource Consider Crew Resource Management as an effective model.Management as an effective model. Genesis of Attitudinal / Behavioral Genesis of Attitudinal / Behavioral
RubricRubric ““Consequence based sliding scale.”Consequence based sliding scale.”
6666
Crew Resource Crew Resource ManagementManagement
for for Long Beach Fire Long Beach Fire
DepartmentDepartment
6868
avoid error, avoid error, to trap errors committed to trap errors committed and to mitigate the and to mitigate the
consequences of errorconsequences of error. .
Definition-
An error management approach defining behavioral strategies taught as error countermeasures that are employed to:
6969
“In the ten years it will take CRM to be introduced nationally, we will attend 1000 firefighter funerals…
I can’t get that out of my mind.”
Gary Briese,
Executive Director
IAFC
7070
Multiple PerspectivesMultiple Perspectives
Origin from Military Aviation over 30 Origin from Military Aviation over 30 years ago.years ago.
Integrated to Commercial Aviation 20 Integrated to Commercial Aviation 20 years ago.years ago.
Discussed as a management tool in Discussed as a management tool in medicine 10 years ago,medicine 10 years ago,
Fire Service text from 2004. Fire Service text from 2004.
7171
Basic PremisesBasic Premises
Technology has greatly minimized Technology has greatly minimized “Tool Error”.“Tool Error”.
Human Factors primary causative Human Factors primary causative agent for errors. agent for errors.
Rigid, hierarchal organizations Rigid, hierarchal organizations especially prone to this type of especially prone to this type of failure.failure.
7272
An Error Mitigation TroikaAn Error Mitigation Troika
Training on how to avoid errors.Training on how to avoid errors. Potential errors are "trapped" before Potential errors are "trapped" before
they are committed.they are committed. Planning for mitigation of error Planning for mitigation of error
consequences. consequences.
7474
Although the following was designed Although the following was designed for Military Aviation, multiple terms for Military Aviation, multiple terms are easily transferred to our are easily transferred to our environment.environment.
““Mission” or “Flight” = “Run” Mission” or “Flight” = “Run”
7575
Reference to your own Reference to your own experience.experience.
How do these descriptions compare How do these descriptions compare to your well functioning teams?to your well functioning teams?
7676
Crew Resource ManagementNaval Air
https://wwwnt.cnet.navy.mil/crm/crm/stand_mat/seven_skills/sev_skills.asp
7777
Basic ElementsBasic Elements
Adaptability / Flexibility (AF)Adaptability / Flexibility (AF) Assertiveness (AS)Assertiveness (AS) Communication (CM) Communication (CM) Decision Making (DM)Decision Making (DM) Leadership (LD) Leadership (LD) Mission Analysis (MA)Mission Analysis (MA) Situational Awareness (SA)Situational Awareness (SA)
7878
ADAPTABILITY/FLEXIBILITY (AF)
WHAT IS ADAPTABILITY/FLEXIBILITY?The ability to alter a course of action
when new information becomes available.
7979
SITUATIONS THAT REQUIRE QUICK ADAPTATIONWhen:
Un-briefed Situations Arise
A Routine Mission Becomes an Emergency
Transitions Occur
A Crew Member is Incapacitated
Interactions are Strained
8080
MAINTAINING ADAPTABILITY / FLEXIBILITY
Anticipate Problems
Recognize and Acknowledge any change
Determine if an SOP or Habitual Response is Appropriate
Offer alternative solutions
Provide and Ask for Assistance
Interact Constructively with Others
8181
SETTING THE TONE FOR
ADAPTABILITY / FLEXIBILITY
Establish an open, professional atmosphere
Ensure the crew understands the mission
8282
““AF” ExampleAF” Example
““During live fire exercises, Recruit During live fire exercises, Recruit Jones did not respond to instructor Jones did not respond to instructor prompting that the exposure to the prompting that the exposure to the debris fire was smoking.”debris fire was smoking.”
8383
Assertiveness (AS)
WHAT IS ASSERTIVENESS?The willingness/readiness to actively
participate, state and maintain a position,
until convinced by the facts that other options are better.
Requires the initiative and the courage to act.
8484
COMFORT LEVEL(Risk Homeostasis)
Comfort level is the degree to which you feel comfortable with what is happening, while taking into account that flying a mission can be dangerous and demanding.
Whenever comfort level is exceeded, "Speak Up".
8585
BEHAVIOR CONTINUUM
PASSIVEASSERTIVENESS
OVER AGGRESSIVENESS
Overly courteous"Beats around the bush"Avoids Conflicts"Along for the ride."
Active InvolvementReadiness to take actionProvide useful informationMakes suggestions
DominationIntimidation Abusive / Hostile
8686
BARRIERS TO ASSERTIVENESS
Position of Authority
Experience
Rank
Lack of Confidence
Fear of Reprisal
8787
““AS” ExampleAS” Example
““Recruit Jones continues to stay at Recruit Jones continues to stay at the back of the group during field the back of the group during field training”training”
““Recruit Jones has interrupted field Recruit Jones has interrupted field training with irrelevant comments training with irrelevant comments several times.” several times.”
8888
Communication (CM)
WHAT IS COMMUNICATION?Communication is the clear and accurate
sending and receiving of information,
instructions, or commands, and providing useful feedback.
8989
TYPES OF COMMUNICATION
Verbal (Involves Words)
Spoken
Written
Non-Verbal (Everything but Words)
Gestures
Voice Intonation
9090
PROCESS OF COMMUNICATION
Sender Conveys ideas or information to others Receiver hears or takes in information and provides feedback
9191
SENDER’S RESPONSIBILITIES
Communicate in Appropriate Mode
Verbal versus Non-verbal
Convey Information Accurately and Concisely
Provide Information at Appropriate Time
Request Verification or Feedback
9292
RECEIVERS RESPONSIBILITY
Actively Listen
Active Vs. Passive Role
Take action as a result of Communication
Answer, or Respond, to Communications
Ask for Clarification of Unclear Communication
9494
““CM” ExampleCM” Example
““Recruit Jones gives ladder Recruit Jones gives ladder commands is an overly subdued commands is an overly subdued manner.”manner.”
““During training evolutions, after During training evolutions, after redirection is given, Recruit Jones redirection is given, Recruit Jones often looks away from the speaker often looks away from the speaker and does not respond.”and does not respond.”
9595
Decision Making (DM)
What is Decision Making? Effective decision making refers to the ability
to use logical and sound judgment
to make decisions based on available information.
9696
This includes:
Assessing the problem
Verifying information
Identifying solutions
Anticipating consequences of decisions
Informing others of decision and rationale
Evaluating decisions
9797
A Decision Making Strategy for Troubleshooting:
Identify all the symptoms
Make a hypothesis as to the possible cause
Test your hypothesis
Apply appropriate remedies
9898
Another variation isAnother variation is John Boyd’s John Boyd’s OODA LoopOODA Loop
ObserveObserve OrientateOrientate DecideDecide ActAct
100100
Factors Which PromoteGood Decision Making:
Teamwork
Extra time to make a decision
Alert crew members
Decision strategies and experience
101101
Barriers to Good Decision Making:
BarrierHow to Overcome
Time
Inaccurate or ambiguous info
Pressure to perform
Rank Difference
Use SOP's and select the best decision using available information
Cross-check info
Evaluate the rationale for making a decision
Use assertive behaviors
102102
Decision Strategy for Risk Assessment:
(AESOP Model) Once a hazard has been detected,
evaluate it to determine its potential effect on the planned flight
by considering its impact on the:
Aircraft Aircraft EnvironmentEnvironment SituationSituation OperationOperation PeoplePeople
103103
The analysis should consider the crew's relative ability
to cope with changes in each of the five basic elements
listed in the AESOP model.
104104
Remember:
Good decisions optimize risk management
and minimize errors, while poor decisions can increase
them.
Poor judgment or decision making is a leading cause of failure to complete missions and of
mishaps.
Each decision affects your future options.
105105
““DM” ExamplesDM” Examples
““Recruit Jones continued to open the Recruit Jones continued to open the spreader on an auto extrication spreader on an auto extrication exercise in spite of obvious exercise in spite of obvious indications that the vehicle was indications that the vehicle was becoming destabilized.”becoming destabilized.”
106106
Leadership (LD)
WHAT IS LEADERSHIP?
The ability to direct and coordinate the activities of other crew members or wingmen,
and to encourage the crew to work together as a team.
107107
DESIGNATED LEADERSHIP
Responsible
Makes Final Decisions
Normal Mode of Leadership
Leadership by Knowledge or Expertise
Occurs when the Need Arises
TYPES OF LEADERSHIP
FUNCTIONAL LEADERSHIP
109109
TRAITS OF AN EFFECTIVE LEADER
Respected
Decisive
Delegates Tasks
Provides Feedback
Leads by Example
Keeps Crew Informed
Open to Suggestions
Builds Team Spirit
Directs and Coordinates Activities
Maintains a Professional Atmosphere
Knowledgeable of how to do the Mission
110110
““LD” ExampleLD” Example
““As the assigned Truck Captain for As the assigned Truck Captain for the Recruit Academy, Recruit Jones the Recruit Academy, Recruit Jones has been noted to not assign tasks to has been noted to not assign tasks to other Recruits. He attempts to other Recruits. He attempts to accomplish directives from the staff accomplish directives from the staff without appropriate delegation.” without appropriate delegation.”
111111
MISSION ANALYSIS (MA)
What is Mission Analysis?
Mission Analysis refers to the ability to develop short term, long-term
and contingency plans, as well as to coordinate, allocate and monitor
crew and aircraft resources.
114114
CHARCTERISTICS OF AN EFFECTIVE BRIEF
Professional
Stays Focused
Assigns Responsibilities
Interactive Complete
115115
IN-FLIGHT MISSION ANALYSISinvolves:
Short-term Planning
Monitoring Mission Progress
Identifying and Reporting Challenges or Changes
117117
““MA” ExampleMA” Example
““As the assigned Recruit Engine As the assigned Recruit Engine Captain, Recruit Jones was unable to Captain, Recruit Jones was unable to prioritize a rescue task within the prioritize a rescue task within the context of a live fire exercise.” context of a live fire exercise.”
118118
SITUATIONAL AWARENESS (SA)
What is Situational Awareness?
Situational Awareness refers to the degree of accuracy
by which one's perception of his / her current environment
mirrors reality.
119119
PERCEPTION VERSUS REALITY
View of Situation
Incoming information
Expectations & Biases Incoming Information versus Expectations
120120
FACTORS THAT REDUCE SITUATIONAL AWARENESS:
Insufficient Communication
Fatigue / Stress
Task Overload
Task Under load
Group Mindset
"Press on Regardless" Philosophy
Degraded Operating Conditions
122122
Give Examples:
Insufficient Communication
Fatigue / Stress
Task Overload
Task Under load
Group Mindset
"Press on Regardless" Philosophy
Degraded Operating Conditions
123123
““SA” ExampleSA” Example
““Recruit Jones did not recognize or Recruit Jones did not recognize or respond to the kink in the supply respond to the kink in the supply line.”line.”
““During the extrication simulation, During the extrication simulation, Recruit Jones stopped operation of Recruit Jones stopped operation of the spreader until the adjoining the spreader until the adjoining destabilized vehicle was cribbed.”destabilized vehicle was cribbed.”
124124
Format 2Format 2Crew Resource Management Crew Resource Management
For the For the Fire ServiceFire Service
Okray & Lubnau 2004Okray & Lubnau 2004
125125
Safety Safety CultureCulture
““Not another program!”Not another program!” Individual ResponsibilityIndividual Responsibility Non-Punitive Culture Non-Punitive Culture and Policyand Policy to to
Error Error
126126
Mission Analysis & PlanningMission Analysis & Planning
Micro – Training OpportunitiesMicro – Training Opportunities AccountabilityAccountability Risk Versus Gain AnalysisRisk Versus Gain Analysis Risk AcceptanceRisk Acceptance
127127
Situational AwarenessSituational Awareness
Killer Equation-Killer Equation- Reality times perception still equals Reality times perception still equals
reality.reality. A Enhancement Strategy:A Enhancement Strategy:
Maintain ControlMaintain Control Timely Assessment Timely Assessment Multiple SourcesMultiple Sources Monitor ResultsMonitor Results
128128
CommunicationsCommunications
Recognize different forms.Recognize different forms. No assumptionsNo assumptions System Approach:System Approach:
InquiryInquiry AdvocacyAdvocacy Monitoring (Situational Awareness)Monitoring (Situational Awareness) FeedbackFeedback
129129
Effective CommunicationEffective Communication
Utilizes more than one form.Utilizes more than one form. Both sender and receiver recognize Both sender and receiver recognize
perceptions, influences and situations perceptions, influences and situations (Filters) that affect the message.(Filters) that affect the message.
Must have active listeners.Must have active listeners. Communication loop (sending, Communication loop (sending,
receiving, feedback)receiving, feedback)
130130
CRM LeadershipCRM Leadership
Three Steps:Three Steps: IntroductionIntroduction IntegrationIntegration TrustTrust
Recognize formal and informal Recognize formal and informal (situational) leaders.(situational) leaders.
131131
CRM “Followership”CRM “Followership”
InteractionInteraction Listening SkillsListening Skills Receiving, interpreting and following Receiving, interpreting and following
instructionsinstructions Making decisions together.Making decisions together. Watching out for one another.Watching out for one another.
132132
Hazardous AttitudesHazardous Attitudes
Anti-authorityAnti-authority ImpulsivityImpulsivity InvulnerabilityInvulnerability MachoMacho ResignationResignation PressingPressing
133133
Decision MakingDecision Making
Define ProblemDefine Problem Generate a course of actionGenerate a course of action Evaluate a course of actionEvaluate a course of action Carry out a course of actionCarry out a course of action
134134
Keys to Good Decision Keys to Good Decision MakingMaking
Maintain good situational awareness.Maintain good situational awareness. Maintain technical proficiencyMaintain technical proficiency Know your resourcesKnow your resources Evaluate, Evaluate, EvaluateEvaluate, Evaluate, Evaluate
136136
HazMat & WMDHazMat & WMD G – Gathering G – Gathering Information E – Estimating Course and Harm D – Determining Strategic Goals A – Assessing Tactical Options &
Resources. P – Planning & Implementing Actions E – Evaluating R - Reviewing
138138
Debriefings & CritiquesDebriefings & Critiques
Similar to CISDSimilar to CISD ““The Blue Line”The Blue Line”
Template:Template: 1. Just the facts1. Just the facts 2. What did you do?2. What did you do? 3. What went wrong?3. What went wrong? 4. What went right?4. What went right? 5. What can be done?5. What can be done?
139139
“Mandatory refresher training wastes the instructor's time, the firefighter’s time and the citizen’s dollars. Saving “a problem” for refresher training is a disservice to our firefighters and the public we protect.”
Page 254
140140
Strategies for Strategies for ImplementationImplementation
Step 1 – Train to technical Step 1 – Train to technical proficiency.proficiency.
Step 2 – Train to CRM proficiency.Step 2 – Train to CRM proficiency. Train risk vs. gain.Train risk vs. gain. Include CRM in department culture. Include CRM in department culture.
141141
Strategies for Strategies for ImplementationImplementation
Step 1 – Train to technical Step 1 – Train to technical proficiency.proficiency.
Traditional perception of Traditional perception of “Training”. “Training”.
We do this well!We do this well!
142142
Strategies for Strategies for ImplementationImplementation
Step 1 – Train to technical proficiency.Step 1 – Train to technical proficiency. Step 2 – Train to CRM proficiency.Step 2 – Train to CRM proficiency.
““Error Tolerant” Culture, Error Tolerant” Culture, especially in Recruit especially in Recruit
Training. Training.
AssertivenessAssertiveness
143143
Strategies for Strategies for ImplementationImplementation
Step 1 – Train to technical proficiency.Step 1 – Train to technical proficiency. Step 2 – Train to CRM proficiency.Step 2 – Train to CRM proficiency. Train risk vs. gain.Train risk vs. gain.
““Fuzzy Logic”Fuzzy Logic”
ValuesValues are assigned to situational are assigned to situational elements.elements.
Train “Experience”.Train “Experience”.
144144
Strategies for Strategies for ImplementationImplementation
Step 1 – Train to technical Step 1 – Train to technical proficiency.proficiency.
Step 2 – Train to CRM proficiency.Step 2 – Train to CRM proficiency. Train risk vs. gain.Train risk vs. gain. Include CRM in department culture. Include CRM in department culture.
??
145145
Managing The Managing The UnexpectedUnexpected
Presenters:Presenters:Karl WeickKarl Weick
Kathleen SutcliffeKathleen Sutcliffe
Jacksonville FloridaFebruary 28, 2005
146146
Rate PreoccupationRate Preoccupationwith Failurewith Failure
Regard close calls and near misses as Regard close calls and near misses as a kind of failure that reveals potential a kind of failure that reveals potential danger rather than as evidence of our danger rather than as evidence of our success and ability to avoid danger.success and ability to avoid danger.
We treat near misses and errors as We treat near misses and errors as information about the health of our information about the health of our system and try to learn from them.system and try to learn from them.
147147
Rate Reluctance to Rate Reluctance to SimplifySimplify
People around here take nothing for granted.People around here take nothing for granted.
People are encouraged to express different People are encouraged to express different points of view.points of view.
148148
Rate Sensitivity to Rate Sensitivity to OperationsOperations
During an average day, people During an average day, people come into enough contact with come into enough contact with each other to build a clear picture each other to build a clear picture of the situation.of the situation.
People are familiar with People are familiar with operations beyond one’s own job.operations beyond one’s own job.
149149
Rate Commitment to Rate Commitment to ResilienceResilience
There is a concern with building There is a concern with building people’s competence and people’s competence and response repertoires.response repertoires.
People have a number of People have a number of informal contacts that they informal contacts that they sometimes use to solve sometimes use to solve problems.problems.
150150
Rate Deference to Rate Deference to ExpertiseExpertise
If something out of the ordinary
happens, people know who has the expertise to respond.
People in this organization value expertise and experience over hierarchical rank.
153153
Cognitive / HeadCognitive / Head Validated / Peer Reviewed Validated / Peer Reviewed
ExamsExams Psychomotor / HandPsychomotor / Hand
Realistic Manipulative TestingRealistic Manipulative Testing Affective / HeartAffective / Heart
Multiple Simulated EmergenciesMultiple Simulated Emergencies Artificial StressorsArtificial Stressors CRM CRM
155155
Our FeedbackOur Feedback
Exit Interview with Chief Officer.Exit Interview with Chief Officer. They say “Thank You”They say “Thank You”
Multiple Post Academy Professional Multiple Post Academy Professional Evaluations.Evaluations.
Graduates volunteer and come back Graduates volunteer and come back to help. to help.
156156
Exercise 3Exercise 3
Use Behavioral / Attitudinal RubricUse Behavioral / Attitudinal Rubric Describe Event to be used for Describe Event to be used for
counseling.counseling. Use only a few sentences. Use only a few sentences. Describe Environment.Describe Environment. Discuss Potential Consequences.Discuss Potential Consequences. Based on severity, what follow-up is Based on severity, what follow-up is
necessary.necessary. Base example on personal experience. Base example on personal experience.
158158
““SOAPE”SOAPE”
Format as belowFormat as below
1. Subjective / Summary1. Subjective / Summary 2. Objective2. Objective 3. Assessment3. Assessment 4. Plan4. Plan 5. Evaluate5. Evaluate
159159
““SOAPE”SOAPE”
Develop Document as Below:Develop Document as Below:
5.5. Subjective / Summary Subjective / Summary 1. Objective1. Objective 2. Assessment2. Assessment 3. Plan3. Plan 4. Evaluate4. Evaluate
160160
Subjective / SummarySubjective / Summary
IntroductionIntroduction Inform that there is a potential for Inform that there is a potential for
discipline, if applicable discipline, if applicable Brief description of behavior, from Brief description of behavior, from
your perspective.your perspective. Distillation of the entire document in a Distillation of the entire document in a
few sentences. few sentences. This section is presented first; This section is presented first;
however written last. however written last.
161161
ObjectiveObjective
Clear, critical description of behavior. Clear, critical description of behavior. (Anyone can see it.)(Anyone can see it.)
No emotionNo emotion ChronologicChronologic Frequency Frequency FactualFactual Identify witnessesIdentify witnesses
Report the individual’s comment’s and Report the individual’s comment’s and responses to the Counseling / Coaching.responses to the Counseling / Coaching.
Include actual negative consequences of Include actual negative consequences of action.action.
162162
AssessmentAssessment(Explanation)(Explanation)
Reference the violation.Reference the violation. P & P’sP & P’s StandardsStandards Common PracticeCommon Practice Crew Resource ManagementCrew Resource Management
Describe potential consequences of Describe potential consequences of future similar actionsfuture similar actions
163163
PlanPlan(Administrative Consequences)(Administrative Consequences)
Potential Treatments: Potential Treatments: EAP (Employee Assistance)EAP (Employee Assistance) CAP (Constructive Action Plan)CAP (Constructive Action Plan) Plan for ImprovementPlan for Improvement Discipline recommendationDiscipline recommendation Additional TrainingAdditional Training
Outline future potential progressive Outline future potential progressive discipline in general terms. discipline in general terms.
164164
EvaluationEvaluation(Administrative Controls)(Administrative Controls)
Remediation (Treatment) follow-up Remediation (Treatment) follow-up Establish specific time frameEstablish specific time frame Establish specific behavior change(s) Establish specific behavior change(s)
required. required. Establish metricsEstablish metrics
165165
Adaptability /
Flexibility (AF)
Assertiveness (AS)
Communication (CM)
Decision Making (DM)
Leadership (LD)
Mission Analysis (MA)
Situational Awareness (SA)
O
uts
tan
din
g ► Determine if SOP
is appropriate ► Offers and advocates for alternative solutions ►Quick recovery after redirection
►Appropriate advocacy of alternatives. ►Plans ahead ►Provides useful information ►Drives events
►Utilizes multiple communication forms ►Recognizes and overcomes communication filters
►Utilizes “OODA Loop” in analysis and implementation of problem solving strategies
►Acts as, and is accepted as, a functional / situational leader ►Excels in fluid environment
►Effective utilization of pre and post incident analysis. ►Develops multiple feasible courses of action
►Maintains control ►Timely assessment ►Multiple information sources ►Monitors results ►Able to rapidly alter tactics
Op
tim
al ► Anticipates
problems ► Identifies change and asks for assistance
►Communicates if comfort level is exceeded ►Establishes dialogue to offset group mindset
►Active listener ►Paraphrases feedback ►Succinct transfer of complex information ►Corrects misunderstandings
►Verifies information ►C omplete assessment of new problem / situation ►Develops a hypothesis
As a designated leader: ►Decisive ►P rofessional ►Provides feedback ►Leads by example
►Effective multi-element planning ►Able to monitor and respond to changing mission parameters ►Identifies actual and potential problems
Occasionally succumbs to: ►Fatigue / Stress ►Task overload ►Group minds et
No
min
al The ability to alter a
course of action when new information becomes available.
The willingness & readiness to actively participate, state and maintain a position, until convinced by alternate facts.
The clear and accurate sending and receiving of information, instructions, or commands, and providing useful feedback.
The ability to use logical and sound judgment; to make decisions based on available information. Recognizes Error Troika.
The ability to direct and coordinate the activities of others and to encourage the crew to work together as a team.
The ability to develop short term, long-term and contingency plans, as well as to coordinate, allocate and monitor crew and resources.
The degree of accuracy by which one's perception of his / her current environment mirrors reality.
Nee
ds
Imp
rov
eme
nt ► Requires specific
prompting to develop appropriate response to changing environment and/or task.
►Impulsive ►Resignation ►Avoids conflicts ►Overly courteous
►Limited feedback ►Poor co nveyance ►I nappropriate timing ►No recognition of misunderstandings
►Dependant on SOP’s ►Does not cross check information ►Unable to define rational for decision
►Passive ►Does not delegate ►Limited information transfer
Does not routinely utilize opportunities for: ►Planning ►Preparation ►Pre & Post incident briefs
Identifies situational elements but is unable to: ►Prioritize ►Assign values ►Respond to changes
Un
sat
isfa
cto
ry
► Unable to respond to changing environment and/or task. ►Does not respond to redirection
►Anti-authority ►Abusive / Hostile ►Invulnerability ►Macho
►No feedback ►Little or no dialogue ►Inaccurate ►Inappropriate modalities
►Dependant on specific and clear direction prior to acting on simple tasks. ►Blames others
►Divisive ►Overbearin g ►Arbitrary ►Demeaning ►Lets others take charge ►Places others at risk
►Requires close direction ►Not able to define tasks or resource allocation ►Ignores problems
►Misinterprets or confuses key situational parameters ►Unable to determine cause and effect ►Places self at risk