Safety Culture Assessment: What are we Trying to Achieve?Joelle MitchellSafety Psychology ConferenceSydney, October 2012

Outline

• What is safety culture?– Theory and research

• Are perception surveys useful?– Assumptions– Evidence– Challenges

• Is safety culture a valuable concept?

Culture: Theory and Research

Culture Theory

• Safety culture is understood as a component of organisational culture

• Schein (1985) is commonly referenced

4A248801 29/10/2012

Schein’s Model

Artefacts

Espoused Values

Basic Assumptions

5A248801 29/10/2012

How is culture built?

6A248801 29/10/2012

How Is Culture Built?

• A definable group with a shared history • Experience and solve challenges• Solutions work well enough to be considered

valid• Taught to new members as the correct way to

perceive, think, and feel in relation to the problem

• Become basic assumptions• Informs future behaviour

7A248801 29/10/2012

Key Defining Features

• Abstract construct (not a concrete phenomenon)• Relatively stable and enduring• Multiple dimensionality• Shared group norms• Socially constructed• Provides meaning about observed things

8A248801 29/10/2012

Safety Culture: A Definition

• “The product of individual and group values, attitudes, perceptions, competencies, and patterns of behaviour that determine the commitment to, and the style and proficiency of, an organisation’s health and safety management.”– International Nuclear Safety Advisory Group (1991)

• The way we do things around here

• Why we do things the way we do them around here

9A248801 29/10/2012

Safety Culture Research

• Perception surveys • Normative sets of factors• Many, many questionnaires • Many, many factors• Factor structures difficult to replicate• A few factors are common to most studies

10A248801 29/10/2012

Criticisms

• “We have been too focused on a traditional research paradigm… that has produced very reliable results about very unimportant things.

• …we have lost touch with some of the important phenomena that occur in organizations… simply because they were too difficult to study by the traditional methods available.” – Schein (1993)

11A248801 29/10/2012

Criticisms

• “As applied by safety researchers, the culture concept is deprived of much of its depth and subtlety, and is morphed into a grab bag of behavioural and other visible characteristics, without reference to the meaning these characteristics might actually have, and often infused with normative overtones.” – Guldenmund, 2010

12A248801 29/10/2012

Perception Surveys

Why Perception Surveys?

• We want a leading indicator – To intervene before something goes wrong– To prevent major events

• We like to benchmark and measure change– Between-groups comparisons– Trends

• Easy to analyse• Lots of nice graphs• Relatively cheap• Relatively minimal time “off-tools”

14A248801 29/10/2012

Problems with Perceptions

• Perceptions are subjective– Difficult to interpret meaningfully

• Perceptions are volatile• Self-report issues– Honesty– Selective Memory

• Construct Validity– Mono-method bias– What are we actually measuring?

• Predictive Validity

15A248801 29/10/2012

Core Assumption

• Safety Culture has a causal impact on safety performance– Safety culture can be reliably measured with perception

surveys– Therefore, perception surveys provide a leading indicator

of safety performance

• Is this assumption accurate?

16A248801 29/10/2012

Core Assumption Part 1

• Can safety culture be reliably measured with perception surveys?

17A248801 29/10/2012

Is a Normative Approach Appropriate?

• How does a culture develop?

• Are all organisations likely to hold the same basic assumptions?

• Are basic assumptions likely to impact organisational outcomes in uniform ways?

18A248801 29/10/2012

Do perception surveys measure safety culture?

19A248801 29/10/2012

Artefacts

Espoused Values

Basic Assumptions

So What?

• What cultural issues are identified after major incidents?– High level decision-making– Top-level leadership practices– Initiatives, priorities and targets– Allocation of resources– Routine violations

• Unlikely to be detected by perception surveys

20A248801 29/10/2012

Core Assumption Part 2

• Safety culture is not reliably measured with perception surveys

• But…

• Do perception surveys provide a leading indicator of safety performance?

21A248801 29/10/2012

What has research found?

• Weak to moderate correlations betweenperception survey scores and injury frequency rates (e.g. TRIFR)

• Used to demonstrate tool validity

22A248801 29/10/2012

Common Interpretation

Safety Perceptions

Safety Behaviours

Injury Rate

23A248801 29/10/2012

Another possibility…

24A248801 29/10/2012

Safety Behaviours

Injury Rate

Safety Perceptions

…and another…

Injury Rate

Safety Behaviours

Safety Perceptions

25A248801 29/10/2012

…and another!

Variable ‘X’

Safety Perceptions

Injury Rate

26A248801 29/10/2012

Is it a Useful Correlation?

27A248801 29/10/2012

• Can we have poor safety outcomes and a low TRIFR?

• Longford Gas Plant 1998• Texas City Refinery 2005• Deepwater Horizon 2010

Predictive Value

• Perception survey scores correlate with occupational injury rates

• HOWEVER…• Low TRIFR ≠ Safe operations

• Do perception surveys predict major incidents?

28A248801 29/10/2012

Case Study

• Snorre Alpha, Norway– Antonsen, 2009

• Positive perception survey results• Gas blowout 12 months later• No lives lost – luck not design

• No significant changes to the organisation or facility over that 12 months

29A248801 29/10/2012

Contradictory Findings

30A248801 29/10/2012

Perception Survey results

Safety a priority

Rigorous risk assessments

High levels of compliance

Good safety communication

Incidents reported

Learning and change from incidents

Incident investigation findings

Production targets prioritised

Lack of risk assessments and poor understanding

Non-compliance was normal

Poor communication climate

Not all incidents reported

Limited use of own and others’ incident information

Key Question

• Are perception surveys the best way to spend limited safety resources?

• Perception surveys do not predict major events– some correlation with occupational injuries

• There are other lead indicators for occupational injury– accuracy– cost-effectiveness– time-efficiency

31A248801 29/10/2012

Safety Culture is a Valuable Concept

Qualifiers

• Don’t try to ‘measure’ culture

• OUTCOMES:– understand the meaning of observable things– uncover basic assumptions– develop effective improvement strategies– identify potential change barriers

• NOT:– as a benchmarking exercise– as a replacement for other technical and systems

processes33A248801 29/10/2012

How?

• Triangulate your methodology!• Use qualitative methods– Interviews– Observations– Focus Groups– Action Research

34A248801 29/10/2012

But we want to benchmark!

• Use existing measurable data– Reporting rates– Inspection findings– Workplace observations– Audit findings– Quality Assurance– Incidents– Near misses– Outcomes and lessons learnt

• Changing themes• Follow-up on previous findings

35A248801 29/10/2012

What about Perception Surveys?

• Are the questions the right ones?• How was the survey developed and validated? – Industry– Company– Links to safety outcomes

• Use data to drive qualitative inquiry– Open-ended questions– Interviews– Focus groups

• Remember the limitations of perception data

36A248801 29/10/2012

Final Thoughts

• The ultimate goal is to keep people safe

• Safety resources are not unlimited

• Target the tools and strategies that are most likely to lead to improved safety outcomes

• Take an evidence-based approach

37A248801 29/10/2012

Thank you!Questions?

References

• Antonsen, S. (2009). Safety Culture: Theory, Method and Improvement. Surrey: Ashgate Publishing Group.

• Guldenmund, F. W. (2010). (Mis)understanding safety culture and its relationship to safety management. Risk Analysis, 30, 1466-1480.

• International Nuclear Safety Advisory Group (1991). Safety Culture. Vienna: International Atomic Energy Agency.

• Schein, E. H. (1985). Organizational Culture and Leadership. San Francisco: Jossey-Bass.

• Schein, E H. (1993). Legitimating clinical research in the study of organizational culture. Journal of Counselingand Development, 71, 703-708.

29/10/2012A248801 39