Download - Taube
NASA ASRS: Evolution Toward Full Electronic Processing
NASA Aviation Safety Reporting System (ASRS)and Patient Safety Reporting System (PSRS)
PM Challenge 2010
Lester GongIT Manager, Booz Allen Hamilton
Elisa TaubeResearch Manager, Booz Allen Hamilton
Linda J. ConnellDirector, NASA ASRS & PSRS
Jessica AriasPSRS Deputy PM, Booz Allen Hamilton
Used with Permission
Agenda
• Background: ASRS / PSRS & Report Processing• The Evolution Towards Full Electronic Processing• Solutions for the Evolution of Report Processing• Approach Methodology • Before & After: The Evolution of Report Processing• Current State of the Evolution• Next Steps / Future Enhancements
ASRS and PSRS
What are ASRS and PSRS?
• Two separate incident reporting systems: ASRS = Aviation Safety Reporting System PSRS = Patient Safety Reporting System
• Managed and operated at NASA Ames Research Center, Moffett Field, California
• Based on three guiding principles:VoluntaryConfidentialNon-punitive
The Aviation Safety Reporting System (ASRS)
• Established in 1976 through interagency collaboration between NASA and the Federal Aviation Administration (FAA): An independent system for aviation safety One of the first lines of defense in identifying
safety issues NASA chosen as “Honest Broker”
• ASRS database is a national asset of U.S. aviation safety data
The Aviation Safety Reporting System (ASRS)
• Receives, processes and analyzes voluntarily submitted incident reports from Pilots Air Traffic Controllers Flight Attendants Maintenance Technicians, and others
• Reports submitted to ASRS may describe both unsafe occurrences and hazardous situations
• ASRS's particular concern is the quality of human performance in the aviation system
The Patient Safety Reporting System (PSRS)
• Initiated in May 2000 via an Interagency Agreement with the Department of Veterans Affairs (VA)
• Modeled after the ASRS• A venue for frontline hospital staff to report patient
safety issues externally• Accepts reports from:
Physicians Nurses Ancillary staff (Pharmacy, Physical / Respiratory
Therapy) Other
• Collects and analyzes voluntarily submitted patient safety reports including: Close Calls Events Suggestions
• Strengthens the culture of safety• Supports the foundation of medical human
factors safety research
The Patient Safety Reporting System (PSRS)
Report Processing
What is Report Processing?
• Report Processing: the methodology for codification of incident reports to facilitate extraction from the database for learning and safety information
• Both programs follow a standard report processing model of rapid screening, multiple report matching, alert identification and codification
• Begins with the receipt of the report and ends with the final coded report entered in either the ASRS or PSRS Database
Report Processing Flow
Each report is read by two expert safety analysts within three working days
The Evolution Towards Full Electronic Processing
Why Evolve?…Paper Based for Decades
Report ProcessingUsers sent 100% of reports by U.S. Mail
All incoming reports were routed in hardcopy Analysts used a 12 page paper coding form to select relevant
information for database insertionCapitalize on data that is generally more available in
electronic form In 2002 ASRS began to receive airline ASAP reports sent
electronically that required in-house printing to enter the processing flow
Database “keyers” typed in all coded reports and narrative textPublic Accessibility to Data
Our public ASRS database was not searchable onlineAccess to data was by request only
ASRS Reporting Volume
0
500
1,000
1,500
2,000
2,500
3,000
3,500
4,000
4,500
5,000
'81 '82 '83 '84 '85 '86 '87 '88 '89 '90 '91 '92 '93 '94 '95 '96 '97 '98 '99 '00 '01 '02 '03 '04 '05 '06 '07
Smoothed / ForecastActual Intake
January 1981 – December 2008 ASRS receives an
average of 4,200 report/month - 193 per working day
Total Report intake for 2008 was 50,405
Why Evolve?…
• Increasing pressure for: Users to submit reports electronically Decreasing processing time / cost
Paper process has limited throughputReport Volume continues to increase
A Method that leverages electronically submitted reporting form data and reduces data input costs
Searchable databases to access relevant reports for research, rulemaking, and safety purposes
Why Evolve?…
• Identified Challenges: Need a well planned and smooth transition
Immediate cutover with no effect on high volume report production
No opportunity to cease or pause production for the evolution
Innovative solutions needed to be developedInternal staff must shift paradigm for report handling
including special cases such as alert level reports
Solutions for The Evolution of Report Processing
Solutions for Evolution
Launched development of 3 Essential Tools:
Electronic Report Submission (ERS) – A method for ASRS and PSRS Reporters to submit reports electronically
External, on website
Analyst Workbench – A browser based internal tool used by ASRS or PSRS Analysts to process reports through 25 stages
Internal for Processing
Database Online (DBOL) – Internet accessible, browser based self-serve application for users to search the databases
External, on website
Approach Methodology
Approach Methodology
• Used Standard Project Management Techniques1: Identified requirements Established clear and achievable objectives Balanced quality, scope, time, and cost concerns Accommodated concerns of key stakeholders
• In order to identify requirements and accommodate stakeholders, a Usage-Centered Approach2 was used
1 PMBOK Guide, p. 9, 102 Constantine, L., & Lockwood, L. (1999). Software for use.
Addison-Wesley-Longman, Inc., NY, NY., p. 23-25
Approach Methodology
• Usage-Centered Approach1 – focuses on the work that users are trying to accomplish and on what the software will need to supply via the user interface to help them accomplish it User centered design represents a shift of focus
from technology to people, from user interfaces to users. To design dramatically more usable tools, however, it is not just users who must be understood, but usage
1 Constantine, L., & Lockwood, L. (1999). Software for use. Addison-Wesley-Longman, Inc., NY, NY., p. 23
Approach Methodology
• Utilized Basic Elements of Usage-Centered Approach 1
Pragmatic Design Guidelines Model-driven design process (task models or use
cases, content models, role models) Organized development activities Iterative improvement Measures of quality
1 Constantine, L., & Lockwood, L. (1999). Software for use. Addison-Wesley-Longman, Inc., NY, NY., p. 24-68
Before & After: The Evolution of Report Processing
Solutions for Evolution
Launched development of 3 Essential Tools:Electronic Report Submission (ERS) – A method for ASRS and PSRS Reporters to submit reports electronicallyFrom Reporting Form to Electronic Reporting
External, on website
Analyst Workbench – A browser based internal tool used by ASRS or PSRS analysts to process reports through 25 stagesFrom Coding Form to Online Data Coding
Internal for Processing
Database Online (DBOL) – Internet accessible, browser based self-serve application for users to search the databasesFrom Formal Requests to Direct Access
External, on website
Electronic Report Submission (ERS)
• General Requirements Internet accessible, web browser based to accommodate
diverse geographic locations and computer workstations Mimics paper form; Intuitive and easy to use Ability to print form online for paper submission No caching or saving on user computer to protect
confidentiality Secure Transmission of data and “on the fly” encryption Autopopulates to new Workbench Tool Ability to reconstruct original report for Analysts in
electronic form for analysis
BEFORE -Paper Reporting Forms sent by U.S. Mail
AFTER -Reporting Forms Securely Submitted Online
http://asrs.arc.nasa.gov/
AFTER -Reporting Forms Securely Submitted Online
Analyst Workbench
• General Requirements Browser based & cross platform Customized interface based on role Single electronic application for completing
and tracking 25 steps in report processing Leverages staff “mental models” for analyzing
paper reports Reduce errors and keying labor hours by pre-
populating electronic data
AFTER
Before & After Report Processing
BEFORE
BEFORE -Coding Form – 12 Pages in Hardcopy
BEFORE -Javascript Data Entry GUI
AFTER -Dual Monitors, Synchronized Displays
AFTER -Analyst Workbench Main Menu
AFTER -Analyst Workbench Full Form Codification Screen
Database Online (DBOL)
• General Requirements: Internet accessible, web browser based self-serve
query application Enable stakeholder communities to construct complex
SQL queries utilizing a simple interface Export to variety of formats (Excel, CSV, Word) for
ease of data manipulation Rapid topical searches of over 150,000 records and
10 million rows of dataSimple approach (just text) or more advanced queries
are possible
BEFORE – Database Online (DBOL)
Internal tool - complex with a high requirement for data structure knowledge
AFTER – Database Online (DBOL)
Public tool – Intuitive, easy to use. Serves casual user and domain expert.
Current State of the Evolution
• On Day 1, October 16, 2006: First electronic report from ASRS website came within minutes
of launch
ASRS received 53 ERS reports in the first week
• ASRS received 50,405 total reports in CY 2008. Breakdown in table below*:
Totals Total %
Electronic 37,076 73.6%
Paper 13,329 26.4%
Totals 50,405 100.0%
Current state of Electronic Report Submission (ERS)
*Data thru October 2009
Current State of Database Online (DBOL)
• Database Online Metrics In 30 years, 7,100 Search Requests (SRs) have been directly
provided by ASRS Research Staff to various aviation organizations and agencies
Since July 2006, 58,857 data queries have been accomplished by external users since the launch of ASRS Database Online (DBOL)1
1Thru October 2009
May 5, 2009 – smooth transition from paper to electronic processing was accomplished
Report analysis staff have been active in quickly identifying issues and proposing innovative solutions
Productivity was improved. Average time to process a report initially has shown increased efficiency
Need for data keying and printing reports eliminated Report tracking requirements have dramatically
improvedEase of tracking location of each report in process
Current State of Analyst Workbench
Next Steps / Future Enhancements
Next Steps
• Continue to improve usability of the tools• Electronic Report Submission:
Shift to html versions to avoid technical difficulties due to advancing .pdf technology
• Analyst Workbench: Enhance tool to allow for adaptability to new domains Incorporate workflow editing
• DBOL: Use of javascript libraries to simplify handling of client
side code Leverage AJAX technology to enhance user
experience