trial sms assessment 4 th arast meeting bangkok, thailand 21 to 25 february 2011
TRANSCRIPT
Trial SMS Assessment
4th ARAST Meeting
Bangkok, Thailand
21 to 25 February 2011
Objective
To Report on the Trial SMS Assessment completed by COSCAP in 2010
COSCAP 4th ARAST 2011 2Trial SMS Assessment
Background
Effective implementation of SMS is key to achieving the desired safety benefits
Service Providers will implement SMS to meet State requirements
Civil Aviation Authorities will conduct oversight to verify effective implementation
Traditional oversight audit and inspections methods measure compliance (i.e., implementation and use)
Traditional oversight audit and inspections methods not well-suited to measure effectiveness of SMS processes
COSCAP 4th ARAST 2011 3Trial SMS Assessment
Assessment Method
Based on ICAO Safety Management Manual SMS Gap Analysis
Preparation ~~ Development of method and assessment tools Conduct of Assessment ~~ Four Parts:
Completion of a Documentation Review Guide by the Air Operator
Review of the completed Review Guide and Company documentation by the Assessment Team
On-site assessment
Report Completion
COSCAP 4th ARAST 2011 Trial SMS Assessment 4
ICAO Safety Management Manual (DOC 9859)
Provides guidance on the features of an SMS SMM includes a “Gap Analysis” table to assist implementation “Gap Analysis” table identifies features that an SMS should
have COSCAP developed SMS “Expectations” from the Gap
Analysis
COSCAP 4th ARAST 2011 Trial SMS Assessment 5
Gap Analysis ~~ SMS Assessment Expectations
Chapters 3, 9 &10Does the [organization] have reactive processes that provide for the capture of information relevant to safety and risk management?
Chapters 9 &10Has the service provider developed training relevant to reactive methods of safety data collection?
Chapters 9 &10Has the service provider developed communication relevant to reactive methods of safety data collection?
COSCAP 4th ARAST 2011 Trial SMS Assessment 6
Chapters 3, 9 & 103.5.3; 3.5.12; 9.9.510.4.1
3) The organization has reactive processes that provide for the capture of information relevant to safety and risk management
Chapters 9 & 104.4.6; 9.11.6; 9.11.710.4.2
4) The service provider has developed training relevant to reactive methods of safety data collection
Chapters 9 & 107.2.8; 9.11.2; 9.11.79.12.2
5) The service provider has developed communication relevant to reactive methods of safety data collection
Assessment Documentation
Outline of Trial Assessment (pdf) Expectations (pdf) Documentation Review Guide (pdf) Assessment Worksheet (sample)
COSCAP 4th ARAST 2011 Trial SMS Assessment 7
Assessment Conduct
Review of Company safety management documents & records
Review of actual processes Interviews with personnel (President, middle management &
staff) Observation of actual work practices
COSCAP 4th ARAST 2011 Trial SMS Assessment 8
Evaluation of Assessment Method
Generally good Expectations
Some revision of expectations required for clarity
Eliminate some duplicate expectations Worksheets
Reduce number of sample questions
Clarify sample questions
Improve alignment of questions to their expectation
COSCAP 4th ARAST 2011 Trial SMS Assessment 9
Evaluation of Assessment Method
Model report required Assessment Team training required Guidance for Assessment timeline required
Formal initiation by CAA
Notification of Service Provider
Service Provider requires time to prepare Review Guide (minimum 30 days)
Completed Review guide and Service provider documentation needed by Assessment Team 30 days prior to on-site review
Assessment Team face-to-face meetings to complete final review preparations
COSCAP 4th ARAST 2011 Trial SMS Assessment 10
SMS Assessment ~~ Next Steps
Revise COSCAP SMS Assessment tools Conduct second Trial Assessment Refine models Develop related COSCAP Guidance material Distribute to COSCAP members Provide support and training as requested
COSCAP 4th ARAST 2011 Trial SMS Assessment 11
Assessment TeamCOSCAP Kim Trethewey
Len CormierNie Junjian
AAPA Martin Eran-TaskerAirbus Bruno DelpratBoeing Dr. Bill RankinCAA Singapore Huei Wang Kang
Capt Michael HohMichael Pang
Center Aviation Safety Technology, China Zhang Yuan
CAA State of Operator Four CAA InspectorsIATA Tony HoustonKorean Air Lines Peter BlakeSR Technics Eduard Schweizer
COSCAP 4th ARAST 2011 Trial SMS Assessment 12
Appreciation
Tremendous support from COSCAP Safety Partners Airbus Association of Asia Pacific Airlines (AAPA) Boeing Commercial Airplanes International Air Transport Association (IATA) Korean Air Lines SR Technics
COSCAP 4th ARAST 2011 Trial SMS Assessment 13
Comments ~~ Questions
Kim Trethewey
Chief Technical Advisor
COSCAP North Asia
COSCAP 4th ARAST 2011 Trial SMS Assessment 14