human factors associates, inc. improving performance of high risk organizations copyright © hfa...

Copyright © HFA

Human Factors Associates, Inc.Improving performance of high risk organizations

Aviation Human Factors and SMSLove Field Dallas

Assessing Safety Climate and Culture in Aviation and Aerospace

Anthony Ciavarelli

31March 2010

Copyright © HFA


AGENDA

• Classic Organizational Accidents• Sources of Human Error• High - Reliability Organizations• Organizational and Safety Culture• Assessing Safety “culture” -- Climate• Safety Status of Aviation and Aerospace• SMS Implementation and Safety Culture

Copyright © HFA


Aviation and Aerospace Accidents

• Air Florida Washington DC “start up airline”• ValuJet Florida Everglades “outsourced cargo services”• Buffalo “aircraft control to stall”• Cockpit fly past destination “pilot distraction”

• Challenger Space Shuttle “risk perception -- decision”• Columbia Space Shuttle “risk perception--decision”• TIROS weather satellite “procedural compliance”

Copyright © HFA


Sources of Human Errors

• Attitudes about job safety and performance• Culture of Safety is deficient in key areas, such as

procedure compliance and reporting• Inadequate standards, training and monitoring of

practitioner qualifications • Poor communications and information flow• Non-compliance with rules, best practices• Stress, production pressure and resources

Copyright © HFA


• Failure to instill strong safety values/culture• Pressure to complete a job or meet schedule.• Failure to establish or enforce standards.• Over-tasking pilots or maintainers• Failure to manage fatigue or stress.• Inadequate resources to perform job safely.• Poor equipment status/reliability.• Failure to manage known risks, including high-risk

pilots or maintainers.

Supervisory Contributions

Copyright © HFA


Angled DecksAviation Safety Center

Naval Aviation Maintenance ProgramRAG (FRS) Concept Initiated

NATOPS ProgramSquadron Safety Program

System Safety Aircraft Design CRM (crew-resource-mgmt)

Aircrew reviews Risk management (ORM) Safety climate-culture

776 aircraftdestroyed in

1954

Naval Aviation Major (Class A) Flight MishapsNaval Aviation Major (Class A) Flight Mishaps

Fiscal Year

11 aircraftdestroyed

inFY-09

all in flightmishaps

2000-2010

Copyright © HFA


US NAVAL SAFETY CENTER DATA

QuickTime™ and a decompressor

are needed to see this picture.

Copyright © HFA


Key Organizational Accident Theories

• Incubation Periods (Turner,1978)• The Normal Accident (Perrow, 1984)• The High-Reliability Organization (Roberts, 1993)• Organizational Culture (Shein, 1990)• Normalization of deviance (Vaughn, 1997)• Organizational Sense-making (Weick, 1999)

Human Factors, Inc.

Copyright © HFA


Naval Aviation Risk Mitigation

• Recruitment and selection• Aeromedical screening• Training standardization and qualifications• Command supervision and risk management• Aircrew performance reviews• Human Factors reviews (Boards & Councils)

• Crew Resource Management (CRM)• Safety Climate and Culture Assessments

Copyright © HFA


Safety Culture: The Naive View

Human Factors, Inc.

Copyright © HFA


High - Reliability Organization (HRO)

• A culture of trust, shared values, and risk mitigating communication processes.

• Communication that provides opportunities for open discussion and improvement.

• Distributed decision-making, “where the buck stops everywhere.”

(Roberts, 1997)

Copyright © HFA


Organizational Culture

Shared Values (What is important) and Beliefs (How things work) that interact with an organizations structures and control systems to produce Behavioral Norms (The way things work around here).

(Adapted: DuPont Culture 98 -- originally Reason 1990)

Copyright © HFA


Safety Culture

• Shared values about what is safe and unsafe

• Common beliefs about how to conduct safe operations

• Behavioral norms that govern risk-taking, everyday procedures and precautions

• Transmission of values, beliefs and accepted practices to others.

Copyright © HFA


Emergence of Just Culture Culture Concept

The phrase just culture refers to the principles for achieving a culture in which frontline personnel feel comfortable disclosing errors—including their own—while maintaining professional accountability... A just culture recognizes that individual practitioners should not be held accountable for system failings over which they have no control.

A just culture also recognizes anyone can produce errors as a consequence of predictable interactions between human operators and the systems in which they work. However, in contrast to a culture that turns to blame as its governing principle, a just culture does not tolerate conscious disregard of rules, reckless behavior or gross misconduct (AHRQ 2008)

Copyright © HFA


A Concise Definition of Safety Culture

Strictly speaking, the survey metrics are measuring “safety climate”, which is considered a snapshot of employee attitudes that reflect the underlying culture of the organization Flin, Mearns, O’Connor and Bryden (2000). We use the findings of the survey to address specific issues regarding the state of a company’s safety culture, which is defined as: “The values, beliefs, and prevailing practices regarding safety that are routinely demonstrated and passed on to all employees from generation to generation.”

Copyright © HFA


Survey based on HRO Principles

MEASUREMENT AREAS

1. Safety Process auditing (SPA)

2. Safety Culture & Reward system (SCRS)

3. Quality assurance (QA) - best work practices

4. Risk management (RSKMGMT)

5. Leadership and Supervision (LDSHP)-- Command Control

SAMPLE SURVEY ITEMS

• My [organization] conducts adequate safety reviews and updates.

• Anyone who intentionally violates a safety rule is swiftly corrected.

• My [organization] has a reputation for high quality performance.

• I believe that our leaders have a clear picture of operational risks.

• My [organization] is genuinely concerned about safety.

Copyright © HFA


Example Survey Items

Copyright 2001-2007 HFA, Inc.

Copyright © HFA


Sample Survey Items

Copyright 2001-2007 HFA, Inc.

Copyright © HFA


20%

30%

40%

50%

60%

70%

80%

90%

100%

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59

2,0

2,5

3,0

3,5

4,0

4,5

5,0

SPA QA SCRS RSK MNGT LDSHP

Copyright 2001-2007 Human Factors Associates, Inc.

Diagnostic Feedback: Fictitious Data

Copyright © HFA


Placement of a specific rating on Normal Distribution Curve

Rating is 1 standard deviation below mean

Rating is in 14th percentile86% of ratings are above this one

Copyright © HFA


Normative Data: Aviation or Aerospace

Relative Percentage of Responses over 1 - 5 Scale

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

1 2 3 4 5

Rating Scale (1 - 5)

Relative Percentage of Responses3.05%

6.60%

14.23%

42.56%

33.47%

9.65% Problematic

76.03% Favorable

Survey Return Rate: 50%

Copyright © HFA


Normative Comparison Across Domains

Comparison % Problematic Mean

Major Airlines [ Wouldn't it be great to have data?]

Commuter AirlinesFreight TransportBusiness Jet

Oil and Gas helicopter transport 18.5% 3.5Airmedical One 8.9% 4.0Airmedical Two 7.5% 4.0Electric Power 10.6% 3.8Aerospace 6.4% 4.1

Copyright © HFA


Relationship between Safety Climate and Accidents

• A study conducted by the US Navy in 2006, found that the survey item average for Risk Management, a category of the High Reliability Organization (HRO) and survey-climate scale for CSA, was a good indicator of accident risk.

• Those squadrons that took the survey were divided into four quartiles based upon their average ratings for the survey’s Risk Management items. The figure on the right shows that squadrons with the lowest HRO-climate ratings had significantly more accidents.

0

2

4

6

8

10

12

Accidents

Climate Verses Accidents

LowestClimateRatings

Highest ClimateRatings

MiddleClimateRatings

Schimpf and Figlock 2006

Copyright © HFA


Components of an SMS

Safety Management System building blocks published by ICAO (2009, p.60), which are: Sounds GREAT! But how do you know it is working?

1. Senior management commitment to the management of safety.2. Effective (non – punitive) safety reporting (just Culture point of view)3. Continuous monitoring through hazard data collection during

operations.4. Investigation of safety concerns in order to identify systemic

deficiencies.5. Sharing Safety lessons learned and best practices.6. Integration of operational and safety training (continuous training).7. Dedicated safety training for all employees.8. Effective implementation of standard operating procedures (with

compliance)

Copyright © HFA


Impediments to SMS Implementation

– Cultural barriers exist that resist organizational change– All Economies are not equally equipped because they are in various

stages of economic development and have different business challenges

– The Labor Climate may require building or rebuilding trust in management

– Organizations do not recognize the expanding role of woman in the work force and fail to consider gender issues in the formulation of SMS plans

– There is no systematic method to determine the state of SMS development and its implementation status.

– There is seldom an established feedback loop to continuously assess and communicate to management the status of status SMS implementation, and to provide information as to whether or not specific practices in fact working.

Copyright © HFA


What the People on the ground have to say -- over 12 years of survey data in aviation and aerospace.

• Flight Operations

– Schedule pressure– Pilot shortage– Lack of trust– Low morale– Reluctance to report– Low time pilots

• Maintenance

– Schedule pressure– Inadequate staffing– Lack of trust– Low morale– Reluctance to report– Hi workload - fatigue– Low experience

service personnel

Copyright © HFA


How to YOU answer these questions?

1. ICAO promotes world wide implementation of SMS -- and has published comprehensive guidance -- have you read it?

2. Do you really understand the SMS program, the process, and the implementation requirements?

3. From your position in supervision or on the ground -- is SMS working as advertised?

4. If so, how do you know? If not why not?

5. What can we do as professionals in air safety to ensure that the great ideas of SMS are in fact implemented and are working as intended?

Copyright © HFA


Survey results: low, med, highAgainst SMS Elements

SMS1 Aviation Aerospace Healthcare

safety policy med - high med - high low - med

structure med - high high low - med

safety plan med - high med - high low - med

evaluation low - med low low

reviews med - high low - med low - med

feedback low low low

SMS items adapted from Lowe 2008

Copyright © HFA


REFERENCES

ICAO (2009). Safety Management Manual (9859). Montreal.JPDO (2010) Safety Culture Improvement Resource Guide (SCIRG v1.6). Joint Planning

and Development Office, Washington DC.Ciavarelli, A.P. (2007). Assessing safety climate and organizational risk. In, the 51st Annual

Meeting of the Human Factors and Ergonomics Society. Baltimore, MDCiavarelli, A. (2007). Culture Counts. http://flightsafety.org/asw_feb08_p18-23.pdfCiavarelli, A. (2007). Safety Climate and risk culture: How does your organization measure

up http://hfa.oses.com/HFA_Research_Background.pdfLowe, C. (2009). A human factors perspective on safety management systems. UK.Reason, J. (1997). Managing the risks of organizational accidents. Brookfield: AshgateRoberts, K. H. (1993). Culture characteristics of reliability enhancing organizations. Journal

of Managerial Issues, 5, 165-181.Schein, E.H. (1999). The corporate culture survival guide. San Francisco: Jossey-BassTurner, B.A., (1991). The development of a safety culture. Chemistry and Industry. pp. 241-

243.Weick, K.E. (1987). Organizational culture as a source of reliability. California

Management Review, 19, 112-127

human factors associates, inc. improving performance of high risk organizations copyright © hfa...

Documents