evaluating a firearms qualification program eac 584 north carolina state university team 4 ashlea...

Evaluating a Firearms Qualification Program

EAC 584North Carolina State University

Team 4Ashlea Anderson

Selby BassTaylor Francis

Stephanie GoinsAshleigh Watts

Overview

• Wilson, NC Police Department Annual In-Service Firearms Training

• Level 1, 2, and 3 measures incorporated– Analysis of levels 1 and 2

• Primary conclusions and suggestions for improvement in future courses

Program

• Course mandated by the State of North Carolina for all law enforcement officers– One day, 10-hour course with three parts– 4 hours of classroom academics with final exam– Day and night qualifying on firing range– Obstacle Course

• Eighteen participants– Approximately 9% of total force– Approximately 20% of total annual qualifiers

Methodology

• Level 1– Reaction survey created by Team 4– 7 five-point Likert scale questions, 3 open-ended questions

• Level 2– Observational checklist created by Team 4 and completed by course

instructor (SME)– Exam created and mandated by North Carolina Justice Academy

(NCJA)

• Level 3– Obstacle course simulating physical and mental stress and fatigue of

possible real-world experience– Requirements and pass/fail criteria developed by NCJA

Level 1

Results and Data

• Level 1– Three types of analysis for Likert scale questions• Average response per respondent• Average response per question• Variance

– Open-ended answers analysis• Answers categorized by themes• Overall summary of themes

0

1

2

3

4

5

Resp A Resp B Resp C Resp D Resp E Resp F Resp G Resp H Resp I Resp J Resp K Resp L Resp M Resp N Resp O Resp P

Average Response per Respondent

Level 1 – Likert Scale

0

0.1

0.2

0.3

0.4

0.5

0.6

Resp A Resp B Resp C Resp D Resp E Resp F Resp G Resp H Resp I Resp J Resp K Resp L Resp M Resp N Resp O Resp P

Variance per Respondent

Level 1 – Likert Scale

3.8

3.9

4

4.1

4.2

4.3

4.4

4.5

4.6

4.7

4.8

Q 1 Q 2 Q 3 Q 4 Q 5 Q 6 Q 7

Average Score per Question

Level 1 – Likert Scale

0

0.2

0.4

0.6

0.8

1

1.2

Q 1 Q 2 Q 3 Q 4 Q 5 Q 6 Q 7

Variance per Question

Level 1 – Likert Scale

Level 1 – Open-ended

List two things that you would change or improve:Category Number of Comments Example of Comments

More shooting/combat drills 11 I would like to see more time on the range shooting

Increased training on Firearm maintenance and mechanical knowledge

4 More training on the various parts of the firearm

Different types of firearms 3 I would like to get experience with different weapons

Increased instructor feedback

3 Instructors watch each person shoot and critique them

Improved shooting range 2 Better, more up to date range

Level 1 – Open-ended

Category Number of Comments Example of Comments

Shooting opportunity 11 Opportunity to fire my service weapon

Proper Stance 5 Practicing stances

Use of force instruction 4 Reviewing use of force – policy statement

Policy and procedures review 4 Where you can carry your handgun off duty

Quality training 4 Great teachers

List two things that you benefited from the most:

Level 1 – Open-ended

Category Number of Comments Example of Comments

Yes 10 Yes, to be retrained

Yes, more often 6 Yes, probably semi-annually

Should this training be provided on an annual basis?

Level 1 – Open Ended Summary

• 47% of responses indicate that participants would prefer more hands on time at the firing range

• Participants value the opportunity to practice firing their weapon during training

• Policy and procedure instruction also significantly impacted participants’ knowledge regarding accuracy and use of their firearm

• Trainees felt the course fosters better efficiency, proficiency, and consistency when it comes to application as needed in the field. All participants agree that the training should be provided on an annual basis, if not more often– 100% stated the training should be provided on an annual basis– Approximately 31% voted semi-annually

Analysis

• Level 1– Overall positive response– Respondent 1 – Outlier? Misunderstanding?– Minimal variance– Strongest area: “Use of Force” v. “Use of Deadly

Force” IAW NC State Law– Weakest areas (tie): Overall impact on

marksmanship and Identification of individual handgun parts

Level 2

Results and Data

• Level 2– Observation Checklist• Meets Standards or Does Not Meet Standards• Individual participant data and comments provided by

instructor• Summary of data and comments created by Team 4

– Exam• Average score for class provided by instructor• No further analysis conducted

– Tool not created by Team 4 and individual data not available

Level 2 - Observation

Level 2 – Observation

• Eleven separate measures– Safety– Stance– Accuracy– Loading/unloading revolver– Loading/unloading semi-automatic– Drawing and holstering weapon– Malfunctions– Night procedures without emergency equipment– Night procedures with emergency equipment– Day qualification– Obstacle course performance

Analysis

• Level 2– Observation

• Every participant met standards in all categories– Strongest area: Safety– Weakest area: Stance

• Attempted to avoid subjective judgments

– Exam• All students achieved passing grade• Average score: 92%• Individual scores not available

Level 3

Results and Data

• Level 3– Obstacle Course• Controlled simulation dictated and governed by NCJA• Limited data provided and no analysis conducted

Analysis

• Level 3– Obstacle Course• Graded as pass or fail

Conclusions

• Positive response from students to program

• All measures of success met by all participants

• Actual impact of class on performance?

• Changes to the qualification time period?

Areas for Improvement

• Larger and more diverse sample size

• Development of more in-depth level 3 observation tool to accompany simulation

• Further observation and analysis of on the job performance

Q & A

Questions?