NEXT GEN QUALITY ASSURANCE FOR PSAPJohn Wynia May 7, 2014
Overview of Quality Assurance
Current QA Methods
Next Gen QA Solution
Q&A
WHY QA
Purpose of QA
Promote service excellence
Adherence to standards, procedures and policies
Consistent performance and level of service
In the QA process we want to evaluate the operator’s strength as well as identify areas for improvement
CALEA Public Safety Communications Accreditation Program
APCO Project 33 Training Program Certification
IAED – International Academies of Emergency Dispatch
NENA’s Recommended QA/QI Standards (to be released)
Main Requirements/Recommendations Common to all: Establish fair and consistent evaluation program Evaluate both call taking and dispatch processes Document the QA evaluations Enable the operator to provide feedback on their evaluations Generate reporting on evaluations
Standards and Accreditation Authority’s
Developed by NENA Development Standards Committee Quality Assurance Working Group
Created to assist with the establishment, implementation and maintenance of a QA/QI program
Defines R&R of the Agency and the Evaluator
To be approved by APCO Standards Development Committee and released within few weeks
Upcoming NENA/APCO Standards for QA/QI
Adherence to protocols and standard operating procedures
Overall performance of each operator
Customer service (Positive attitude
Effective use of resources (Caller details verified? Entered into CAD properly)
Quality of operation as a whole
What to QA
Seamless/Non-intrusive – should not impact operators behavior
Random/Fair – not picking specific bad / good calls
On-going – not one time but part of standard process
Consistent – eval form, etc
Comprehensive
Documented
Automated
Attributes of an Effective QA Program
METHODS OF QA
Manual
Semi automated
Automated
Next Gen QA
Methods of QA Programs
Word/Excel-based evaluation forms
Search and replay call
Advantages: Easy to learn Low investment Doesn’t require additional application
Disadvantages: Cumbersome process Hard to follow consistent and fair selection Highly time consuming reporting
Manual QA Process
N/ A YES NO 0- 2 2
#2 Correct Location Asked/ Entered : N/ A YES NO 0- 10 10
#3 Location Verbally Verified: N/ A YES NO 0- 10 10
N/ A YES NO 0- 5 5
#5 Address/ Name of RP Verified: N/ A YES NO 0- 2 2
#6 Call Classification YES NO 0- 10 10
N/ A YES NO 0- 2 2
N/ A YES NO 0- 5 5
N/ A YES NO 0- 5 5
N/ A YES NO 0- 2 2
N/ A YES NO 0- 3 3
N/ A YES NO 0- 5 5
N/ A YES NO 0- 5 5
N/ A YES NO 0- 5 5
N/ A YES NO 0- 2 2
N/ A YES NO 0- 2 2
N/ A YES NO 0- 5 5
N/ A YES NO 0- 10 10
#19 Ascertained RP Contact: N/ A YES NO 0- 3 3
N/ A YES NO 0- 5 5
N/ A YES NO 0- 2 2
0- 100ptsAvailable
Scores Of: 0- 64 65- 79 80- 94 95- 100
Comments:
U N S EOverall Rating:
#11 Medical Needed / Fire added
#16 Direction of Travel:
100
#15 Vehicle Information:
Event #
#21 Proper Closing Remarks:
#1 Proper Greeting:
#4 Telephone Number Verified:
SHOO
#8 Customer Service:
#10 Victim Information Requested:
#9 Positive Attitude towards Caller:
#14 Suspect Information / Entered:
#20 Completed in Appropriate Time:
#7 Used Language Line (If nec)
#18 Appropriate CAD Updates:
#12 Drugs or Alcohol Involved:
#17 Kept RP on line if Appropriate:
#13 Weapons Involved (In Resd):
QA COMPETENCIES
COMMUNICATIONS CENTER "911" Quality Assurance Audit
Dispatcher Name:Audit Date:
Event Time:
Event Date:
Reviewed By:
Location of Event:
Score % - Rating
Automated form based but not necessarily tied to audio recording
Doesn’t include screen and other media
Provided by non-logging vendors
Advantages: Procedure specific, e.g. medical dispatch Built for accreditation – e.g. IAED, Total Response
Disadvantages: The user has to deal with two applications: reply and evaluation Manual selection of calls – could be perceived as inconsistent and unfair Limited to call-taker (phone only) evaluation
Semi Automated
Based/built for Commercial Call Center
Limited to single call – call-taker phone interaction
Partial compliance with QA accreditations/standards
Screen recording is optional, limited to single screen
Advantages: Allows random selection of calls Complete evaluation while listening to the call and watching screen activity Includes reporting capability
Disadvantages: Not comprehensive – no way to look at the entire incident handling process No way to include multimedia incident data
Automated
NEXT GEN QA SOLUTION
Next Gen QA
Designed bottom up for Public Safety environments
Compliance with CALEA, APCO P33 and the upcoming NENA Standards
Tied to the incident communication capture system: phone, radio, text, screen, video, GIS, etc.
Next Gen QA Requirements
Call selection Manual – important for high attention calls Truly random Flexible – minimum number of calls per incident type, user, etc.
Complete incident capture and review Capture and synch all incident related voice recordings: phone and radio Capture and synch operator’s screen Assess operator’s quality of incident handling involving all forms of incoming data
Targeted coaching and training Easily pin point knowledge gaps, e.g.: text interaction, reference to GIS data, etc. Provide needs-based coaching and training to groups or specific operators Ensure operator on track acquiring new skills and adhering to evolving procedures
Next Gen QA Requirements - Screen Recording
Capture all screens of operator’s workstation
Compliments entire incident evaluation
What action did the operator take on CAD / CPE / other?
Operator’s Workstation
Replay Workstation
Next Gen QA Requirements - Reporting
Activity reporting – insight into the volume of phone and radio traffic to help support decisions on staffing
QA reporting - quality performance and trending each individual staff member
Measure performance against KPIs to maintain QA and Training accreditations
Supervisors and Training Managers: Reconstruct and evaluate Identify knowledge gap Provide proper training
Operator: ‘how am I doing?’ Review evaluations Feedback and sign-off
PSAP Managers: Periodic reports to gauge overall
team’s and center’s performance Take improvement measures
Workflow that Involves All Users
Call / Incident Evaluation
OperatorReview
ManagementReports
EXAMPLE OF A NEXT GEN QA APPLICATION
20
Automatic Selection of Calls for QA
Evaluation of Incident as a Whole
Phone call and related
dispatch radio
Fill evaluation while listening
Multimedia Incident Evaluation
Operator’s Feedback
Flexible Evaluation Form Design
Reporting Capability
Reduced turnover
Empowered call taker/dispatcher
Increased Professionalism
Adherence to existing and evolving protocols (e.g. Text-to-911)
Effectively and efficiently run PSAP
The Value of Next Gen QA Program
Thank You