use of performance measurement and goal setting to improve construction managers' focus on...
TRANSCRIPT
This article was downloaded by: [University of California Davis]On: 26 October 2014, At: 15:29Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House,37-41 Mortimer Street, London W1T 3JH, UK
Construction Management and EconomicsPublication details, including instructions for authors and subscription information:http://www.tandfonline.com/loi/rcme20
Use of performance measurement and goal setting toimprove construction managers' focus on health andsafetyIain Cameron a & Roy Duff ba School of Built and Natural Environment , Glasgow Caledonian University , UKb Dr A Roy Duff Ltd , 4 The Milling Field , Holmes Chapel , UKPublished online: 14 Aug 2007.
To cite this article: Iain Cameron & Roy Duff (2007) Use of performance measurement and goal setting to improveconstruction managers' focus on health and safety, Construction Management and Economics, 25:8, 869-881, DOI:10.1080/01446190701268848
To link to this article: http://dx.doi.org/10.1080/01446190701268848
PLEASE SCROLL DOWN FOR ARTICLE
Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) containedin the publications on our platform. However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of theContent. Any opinions and views expressed in this publication are the opinions and views of the authors, andare not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon andshould be independently verified with primary sources of information. Taylor and Francis shall not be liable forany losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoeveror howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use ofthe Content.
This article may be used for research, teaching, and private study purposes. Any substantial or systematicreproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in anyform to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions
Use of performance measurement and goal setting toimprove construction managers’ focus on health andsafety
IAIN CAMERON1* and ROY DUFF2
1 School of Built and Natural Environment, Glasgow Caledonian University, UK2 Dr A Roy Duff Ltd, 4 The Milling Field, Holmes Chapel, UK
Received 3 October 2006; accepted 8 February 2007
Previous attempts to use goal setting and feedback to improve safety behaviour on construction sites have
focused on operatives, but recognized management commitment as essential for success. This research
addressed two problems: validly and reliably measuring the safety performance of construction managers; and
motivating managers to improve their safety performance. A measure of management safety performance was
developed, covering seven items: induction training; toolbox talk training; safety committees; subcontractor
safety; maintenance of safety records; safety manager actions and safety consideration (interaction,
communication and worker engagement). This was used on a case study site to improve motivation of the
management team. During an intervention, using monthly feedback and goal-setting techniques, their
performance improved from 49% to 82% of maximum expected performance. The results demonstrate that a
valid and reliable measure of management safety performance is practicable and support the proposition that
goal setting can improve management safety performance in the construction industry.
Keywords: Behavioural safety, feedback, goal setting, organizational behaviour, organizational psychology,
safety
Introduction
Cameron and Duff (2007) have reviewed many
publications describing attempts to improve the safety
performance of workers in the manufacturing and
construction industries, from North America and
Europe to the Far East, using the behavioural
techniques of goal setting and feedback. They conclude
that the evidence, so far, suggests that, though
behavioural safety interventions can be extremely
effective, they do rely very much on the active support
of a committed management. This raises the problem
of how managers might be motivated to improve their
own safety-related behaviours, possibly also using goal
setting and feedback. Such an intervention, focused on
management behaviours, might encourage committed
management behaviour. A case study of this kind of
intervention, on a construction site in Stirling,
Scotland, is described.
This approach is developed as an extension of
successful techniques of worker behaviour improve-
ment (Duff et al., 1993; Robertson et al., 1999).
The research addressed two construction safety
problems:
(1) How can construction organizations reliably
measure the safety performance of their man-
agers?
(2) How can construction organizations motivate
these managers to improve their safety perfor-
mance when they themselves are in little
immediate danger?
Locke’s (1968) theory of work motivation termed
‘goal setting’ was integrated with the concept of safety
auditing (e.g. Saunders, 1992). A case study in the use
of a safety management behavioural audit with goal
setting and feedback will be described.*Author for correspondence. E-mail: [email protected]
Construction Management and Economics (August 2007) 25, 869–881
Construction Management and EconomicsISSN 0144-6193 print/ISSN 1466-433X online # 2007 Taylor & Francis
http://www.tandf.co.uk/journalsDOI: 10.1080/01446190701268848
Dow
nloa
ded
by [
Uni
vers
ity o
f C
alif
orni
a D
avis
] at
15:
29 2
6 O
ctob
er 2
014
Research method
The research examined the hypotheses:
N a valid and reliable measure of management
safety behaviour can be developed and imple-
mented; and
N an intervention of management audit, goals and
feedback can be used to improve the dependent
variable of safety management performance.
Development of the measurement instrument
Measures of accident incidence are today recognized
as invalid measures of safety at a workplace (except in
very large organizations, or over the very long term)
because they are insensitive to real changes in safety
behaviour or conditions, and under-reported (Health
and Safety Executive, 1993; Glendon and McKenna,
1995). Therefore, two alternative measures of safety
have been developed, measuring operative unsafe acts
and/or conditions and management actions and/or
omissions.
Focusing on unsafe acts and conditions in the
workplace is an intuitively appealing method of
differentiating a safe site from an unsafe one.
However, this has limitations because the approach
implies that it is primarily operatives that are to blame
for accidents. This is contrary to research which
suggests that operative errors are downstream ‘trigger
events’ in a series of more fundamental management
failures (Reason, 1990; Whittington et al., 1992; Dester
and Blockley, 1995).
Management actions can differentiate safe sites from
unsafe sites based on the observed level of safety
management commitment (Cohen and Cleveland,
1983). Commitment is an indicator of workplaces with
positive safety cultures (Cooper and Phillips, 1994) and
operatives can only be expected to adhere to the
minimum safety standards set by managers (Fleming et
al., 1997).
The utility of conventional safety audits as means of
measuring safety has being questioned (e.g. Eisner and
Leger, 1988). In contrast, behavioural inventories of
unsafe acts and conditions have emerged as effective
measures of safety performance (McAfee and Winn,
1989). These methods have been applied to construc-
tion sites with good results, though moderated on some
sites by low levels of management support (Duff et al.,
1994; Marsh et al., 1998; Robertson et al., 1999).
These findings seem to suggest that a behavioural
inventory of unsafe management acts and procedures
would offer utility for measuring safety performance on
construction sites.
Therefore the safety management behavioural audit
was developed, using a validation procedure derived
from Phillips (1992), and involving:
N a review of safety management literature,
accident statistics and reports to extract a list
of safety management behaviours and proce-
dures;
N evaluation of each item for inclusion in a
measure, in relation to its potential for objective
and reliable measurement;
N further validation and establishing item weights
through a postal survey of construction person-
nel;
N design of questions for the audit and determina-
tion of suitable rating scales;
N piloting of audit with industrial collaborators,
and revision where necessary.
Evaluation of the behaviours and procedures identi-
fied seven items: induction training; toolbox talk
training; safety committees; subcontractor safety meet-
ings (pre-start and operative inductions); maintenance
of safety records; safety manager inspections and
actions; and safety consideration (operative percep-
tions). A questionnaire was developed to rate these
items on a three-point scale—very important, less
important and not important. The respondents were
asked to consider ‘managerial time commitment’,
‘degree of control’ and ‘consequences of not perform-
ing a given behaviour’, as indicators of the importance
of each item. This was sent to the safety managers of
the top 50 contractors in the UK. Thirty-one responses
established item weightings of 17% (induction training;
safety committees; subcontractor safety meetings;
maintenance of safety records; safety consideration)
and 8% (toolbox training; safety manager inspections).
These weightings were used to give a total behaviour
rating known as the ‘Global Index’.
In the completed audit each item contained several
questions, each one requiring a response, on an
‘interval-rating-scale’, scoring the degree or quality of
compliance with maximum expected performance.
These questions, or sub-items, addressed:
N induction training—frequency; content; status of
trainer; time allocated; supply and instruction in
PPE;
N toolbox talk training—frequency; status of trainer;
number of attendees; presentation and format;
duration; relationship to current work;
N safety committees—frequency; management atten-
dance; representation of direct and subcontract
employees; availability of minutes; resolution of
concerns; discussion of the behavioural safety
audit;
870 Cameron and Duff
Dow
nloa
ded
by [
Uni
vers
ity o
f C
alif
orni
a D
avis
] at
15:
29 2
6 O
ctob
er 2
014
N subcontractor safety meetings—management pre-
start meetings; frequency of pre-start meetings;
seniority of attendees; allocation of resources;
risk assessments; subcontractor inductions (fre-
quency, quality, time allocation);
N maintenance of safety records—safety issues in
progress meeting minutes; safety committee
minutes; induction records and risk assessments;
toolbox training records; workplace register;
quarterly reviews of safety plans;
N safety manager inspections and actions—regularity
of inspections; provision of feedback; safety
manager participation in other areas of safety
performance;
N safety consideration (operative ratings)—quality of
induction training; quality of toolbox talks;
quality of safety committee; recognition of safe
working and criticism of unsafe working; discus-
sion of safety in work instructions; assessment of
day-to-day management safety focus.
The sub-item scores measure relative frequency of
conformance of site management against percentage
bands (e.g. actual frequency of toolbox talks against
planned frequency), and quality of conformance
against comprehensive scale anchor descriptors (e.g.
operative perceptions of quality of organization and
presentation of induction training). Pilot trials of the
audit suggested that the scales were understood, suited
for rating management tasks and easy to use.
This behavioural audit was applied in the field to
determine whether these ratings:
(a) are a valid measure of management safety;
(b) accurately and consistently (reliably) measure
management safety.
A literature search (Cameron and Duff, 2007) has
demonstrated that goal setting and feedback have been
an effective method of motivating improved worker
safety behaviour, across manufacturing industries in
North America, Europe and the Middle East as well as
in the United Kingdom and Hong Kong construction
industries. This suggests that goals and feedback may
also offer utility as a method of improving management
safety and so, in this study, goals and feedback are
applied to management safety behaviours.
Experimental design
Experimental designs exist that allow researchers to
draw causal conclusions about their results, by elim-
inating alternative explanations (‘Hawthorne’ effect,
history, maturation, regression and instrumentation) as
the cause of change in the dependent variable. Perfect
experimental control would mean that the planned
independent variables did in fact cause the observed
change in the dependent variable.
‘Natural designs’ are best but these can be dis-
counted in work settings because it is impractical to
randomly assign people to equivalent experimental and
control groups (Barlow and Herson, 1984). ‘Quasi-
designs’, which use non-equivalent control groups, are
the recommended alternative, relying on comparisons
made within the group and thus serving as their own
control. Two types of quasi-design were considered,
but dismissed on methodological grounds. First,
‘reversal designs’ that rely on withdrawal of a, possibly
successful, treatment could be unethical in the safety
field; and would have been unacceptable to the
industrial collaborator. Secondly, ‘multiple designs’
that rely on the stepped application of a treatment
across items were dismissed because interventions
aimed at one item could affect other items and nullify
control (Kazdin, 1982) and because they lead to longer
duration experiments than most construction sites can
accommodate.
So, a ‘before and after’ design (Figure 1) was
selected. This design does have deficiencies, for
example an environmental (confounding) event could
occur at the same time as the intervention, and this
would mean that changes in safety performance could
not be attributed solely to the experimental treatment.
However, the probability of this was low and, in any
event, experimental control could be enhanced by
monitoring and recording possible confounding factors.
A £25m, two-storey shopping precinct project was
selected, in order to engage onsite management, but
avoiding the likelihood of sophisticated H&S manage-
ment systems such as tends to be the case with very
large, high-profile projects. The main contractor
management team comprised a project manager, four
section managers and two foremen, controlling
approximately 50 main contractor and 230 subcon-
tractor employees. The construction programme
allowed nine monthly audits, three in the baseline
(before) period and six in the intervention (after)
period. This design represents a balance which is
satisfactory for the statistical testing of data.
During the baseline, measurements were conducted
at four-weekly intervals, using the safety management
behavioural audit. During the intervention, measure-
ment and goal setting were conducted at four-weekly
intervals throughout the six months of the intervention.
During this phase, specific and challenging manage-
ment team goals were set, coupled with feedback.
Strong goal commitment was sought by allowing
participants to set their own goals (Cooper, 1992;
Cooper et al., 1993). The following protocols were
used:
Measurement and goal setting to improve H&S management 871
Dow
nloa
ded
by [
Uni
vers
ity o
f C
alif
orni
a D
avis
] at
15:
29 2
6 O
ctob
er 2
014
Goals
N Challenging (difficult) percentage goals were set
in a participative manner and publicly posted.
N Qualitative sub-goals, ‘action plans’ for goal
achievement, were agreed.
Feedback
N Specific quantitative performance feedback was
displayed graphically.
N Verbal feedback, explaining changes in perfor-
mance measures, was given.
N Additional, ongoing feedback was achieved by
requiring weekly, management ‘self-report’
forms.
Statistical testing of the motivation hypothesis
Two types of statistical test were used for testing the
motivation hypothesis:
(1) visual analysis of trend: observing changes in
means and standard deviations between baseline
and intervention phases;
(2) mean treatment effect size (d-statistic): this
unit free number is an indicator of the
magnitude of the difference between experi-
mental phases, regardless of its statistical
probability.
These complementary tests assess: practical signifi-
cance (visual analysis) and substantive significance
(effect size).
The field experiment
Data were collected from three different sources, to
allow triangulation. Each monthly audit included (1)
operative recalled data; (2) management recalled data;
and (3) inspection of records and documents, the
format, in each case, prescribed by the requirement to
score each item on the ‘interval-rating-scale’.
(1) Operative recalled data. This relates to the item
‘safety consideration’. The first part of the
behavioural audit involved the confidential
structured interview of a sample of approxi-
mately 10 (20% of total) directly employed
operatives each month, and respondent scores
aggregated to obtain a rating for safety con-
sideration. The questions tested operative
experience of management’s visible support for
safety.
(2) Management recalled data. The second part of the
behavioural audit involved a 40-minute struc-
tured group interview with the management
team, collecting data on induction training,
toolbox training, safety committees, subcontrac-
tor safety and safety manager actions.
Figure 1 The ‘before and after’ experimental design
872 Cameron and Duff
Dow
nloa
ded
by [
Uni
vers
ity o
f C
alif
orni
a D
avis
] at
15:
29 2
6 O
ctob
er 2
014
(3) Inspection of records and documents. The third part
was an examination of main and subcontractor
documentation to collect data from labour
returns, induction training records, toolbox talk
training records, safety committee meeting
minutes, progress meeting minutes involving
safety, subcontractor pre-start meetings, risk
assessments and method statements, safety
policies, health and safety plans and statutory
registers.
Implementation
Pre-experiment phase
Before any measurement, the scope of the audit was
explained without revealing the research hypotheses,
two seemingly contradictory requirements. This was
resolved by issuing a narrative which outlined the audit
in general and in vague terms. Staff were advised that
measurements would be conducted each month for
three months, after which an assessment of safety
management performance would be made and safety
improvement strategies introduced. In spite of these
efforts to avoid a ‘Hawthorne effect’ (Mayo, 1933) or
unintentional performance feedback, it was possible
that the baseline scores would not represent normal
performance. To control for this, samples of data were
collected from records for the period prior to the start of
the baseline, in order to test for signs of an improvement
during the baseline phase. The preliminary meeting was
also used to build enthusiasm and support.
Baseline phase
The researcher attended site monthly and collected
data, as discreetly as possible, from operatives, manage-
ment and records. There was no feedback and the
research hypotheses were not discussed. Where these
data could be compared with those from the period
prior to the start of the baseline, no improvement was
evident.
Intervention phase
The intervention phase comprised a series of monthly
audits, immediately followed by goal-setting meetings,
involving the project manager, site management and
the researcher. This group agreed goals and action
feedback each month. The following standards were
agreed with managers and applied, to define ideal
performance and against which to set goals:
N conduct a safety induction for all labour new to
the site;
N conduct two relevant safety toolbox talks each
month;
N conduct a representative safety committee meet-
ing each month;
N conduct a pre-start meeting with all subcontrac-
tors and induct supervisory personnel;
N maintain safety records and documents (scaf-
folds, minutes, risk assessments, etc.);
N disseminate safety information to employees, e.g.
risk assessment, consultation, etc., hence show-
ing safety consideration.
Goal information, such as the number of required
toolbox talks and safety committee meetings required
to satisfy targets, was formulated to guide the behaviour
of the project manager who represented the ‘champion
of the intervention’ (Marsh et al., 1998) and was central
in the setting of, and maintaining commitment to, goals
and action plans.
Reliability of measurements
Parallel recording of data was not feasible, so site
management was used to investigate the reliability of
the data. This was done at the conclusion of each
behavioural audit by checking the consistency and
accuracy of any data collected from site records, and
interviewing the site team to ensure that they agreed
with the scoring of the data. Baseline scores were
endorsed collectively after the conclusion of the base-
line, to avoid feedback during the baseline phase.
There was no evidence of observer bias, even among
the management team. Poor scores that spoiled a
sequence of good scores were not contested.
Furthermore, because most audit scores rate objective
records it is seldom possible for unreliable data to
occur, without actual falsification of site records.
Analysis and discussion of case study results
The case study site was drawn from a ‘top five’ UK
contractor. The site, a £25 million fast-track project to
construct a shopping precinct, was in Stirling,
Scotland. The project team comprised seven managers,
around 50 direct employees and around 230 subcon-
tractor employees. The construction programme
allowed nine management safety audits, three prior to
intervention and six after.
The Global Index is recorded in Table 1 and shows
the pre-/post-intervention change in this aggregate
measure.
The data indicate that safety management perfor-
mance improved substantially, with an average increase
of around 30% on the introduction of the goal setting
and feedback intervention. However, the diary showed
three extraneous factors that may have moderated
Measurement and goal setting to improve H&S management 873
Dow
nloa
ded
by [
Uni
vers
ity o
f C
alif
orni
a D
avis
] at
15:
29 2
6 O
ctob
er 2
014
overall performance: an unexpected HSE visit shortly
after month one audit; the site’s only major accident (a
severed finger) at month six (intervention month two);
and an acceleration order that involved 24-hour work-
ing on weekdays, coinciding with the annual leave of a
manager who was involved in items such as toolbox talk
training, both in month eight (intervention month five).
It is reasonable to assume that this latter event
negatively influenced management safety and perfor-
mance evidently improved when the manager returned
from leave.
There was no significant improvement in safety
performance after the major accident and interviews
with staff at the conclusion of the research suggested
that the accident did not influence the day-to-day safety
behaviours of the site team.
Behavioural audit safety performance scores: %-
safe (individual items)
Table 2 gives a better insight into the origins of the
improvements shown in the Global Index.
Induction training
Inspection of the monthly scores reveals a gradual
improvement of approximately 15% after the introduc-
tion of the intervention. This was largely due to greater
risk assessment dissemination. No inductions occurred
in month one and it is very possible that the data
collection may have caused staff to increase effort;
however, this is judged not to interfere with the
conclusion that the intervention caused another sig-
nificant improvement.
Toolbox training
The start of the intervention coincided with an
immediate improvement of 36% over the previous
month. Though variable, this item performed at a high
level until month eight when performance dropped to
0%, almost certainly due to the absence, on annual
leave, of the responsible site manager.
Safety committees
This item scored zero during the baseline phase but
improved greatly following goal setting. The site
manager appeared embarrassed at the zero score and
immediately initiated strategies to aid goal attainment.
Feedback of scores was used to encourage improve-
ment. The only blemish was a last minute cancellation
of a meeting at month eight to accommodate a section
handover.
Subcontractors’ safety
Despite a high level of performance during baseline,
typically around 76%, performance did improve by
18% or so, after the introduction of the intervention,
particularly in relation to pre-start meetings and
dissemination of risk assessments.
Safety records
During the baseline, records were never held in a
central location and training records and risk assess-
ments were missing. Following the intervention, record
keeping was centralized and greater effort was directed
to maintenance of safety records. Intervention scores
rose by around 20%.
Table 1 Percentage-safe global behavioural audit scores, pre-intervention (months 1–3) and post-intervention (months 4–9)
Months 1 2 3 4 5 6 7 8 9
Audit score (%) 38 52 56 87 87 89 85 63 83
Table 2 Percentage-safe individual behavioural audit scores, pre-intervention (months 1–3) and post-intervention (months 4–
9)
Months
1 3 3 4 5 6 7 8 9
Induction training 0 81 75 89 95 95 95 95 94
Toolbox training 0 46 50 86 61 80 90 0 66
Safety committees 0 0 0 94 92 100 92 0 84
Subcontractor 76 72 80 98 98 97 84 97 97
Safety records 63 61 79 96 93 92 90 79 90
Safety manager 83 67 67 83 75 88 83 92 83
Consideration 43 44 42 59 66 65 60 61 55
Audit score (%) 38 52 56 87 87 89 85 63 83
874 Cameron and Duff
Dow
nloa
ded
by [
Uni
vers
ity o
f C
alif
orni
a D
avis
] at
15:
29 2
6 O
ctob
er 2
014
Safety manager actions
This item was an attempt to motivate the site manage-
ment team to ensure regular safety inspection.
Performance increased by at least 16% for five of the
six intervention months. The safety manager took a
keen interest in the scores, viewing them as a reflection
of his own performance.
Safety consideration
The baseline performance on this item was low, with a
mean value of 43%. There was little evidence to suggest
that important safety information was communicated
to those actually at risk. This baseline result seemed to
disappoint the site team and the construction director,
and the negative feedback acted as a catalyst to
motivate corrective strategies. For example, the project
manager ensured that gang leaders attended the safety
committee and that foremen and site managers con-
sulted employees about risk assessments. These beha-
viours caused an improvement of 17% immediately,
but the site never scored above 65%. Data collected by
operative interview suggested a critical, blame culture
on the site and a lack of positive reinforcement for
desired behaviour.
These data demonstrate that it is possible to
implement a safety management behavioural audit as
a measure of the effectiveness of management safety
performance. The fact that the management team set
improvement goals each month, displayed feedback
graphs on site, and endorsed the researcher’s scores
strongly suggests that the industrial partner accepted
the scores as valid and reliable ratings.
Mean average safety performance scores: MAS
In order to assess trends more effectively and to
prepare data for the mean treatment effect size
(MTES) calculations, performance data were sepa-
rated into blocks of three months: the baseline phase;
the first three-month treatment phase; and the final
three-month treatment phase. Each block was
reduced to an average and standard deviation
(Table 3).
Percentage relative change (RC)
The pre-intervention mean was 49% and the post-
intervention means were 88% and 77% representing
improvements of 38% and 28% (Table 3). As the mean
baseline value was 49%, there is a 51% scope for
improvement. The percentage relative change is calcu-
lated from the proportion of 38 and 28 parts in 51,
which suggests relative changes of 75%RC and
55%RC. Clearly, if a baseline score is 20%, a
considerable opportunity to improve exists.
Conversely, if a baseline score is 80%; a limited
opportunity to improve exists—the RC reflects this.
Because the Global Index represents all items, it is
reasonable to suggest, from these data, that goals and
feedback exerted a substantial influence on manage-
ment safety behaviour. The duration of the project did
not permit investigation of the sustainability of this
influence.
There are two general trends evident in Table 3.
First, variability exists within some items (e.g. toolbox
training and safety committees); this is doubtless a
function of sample size (Kurtz, 1983). Secondly, post-
intervention data tend to be more stable than pre-
intervention data. This observation supports the find-
ings of Duff et al. (1993) that behavioural interventions
lead to a reduction in variance due to the influence of
the treatment.
Effect size of performance scores: MTES
A more statistically analytical indication of the
magnitude of the intervention effect was achieved
by calculating a mean treatment effect size (MTES)
known as the ‘d’ statistic (Cohen, 1969/88). MTES(d)
is calculated by subtracting a pre-intervention
mean average score from the post-intervention
mean average score and dividing by the pooled
standard deviation. With small samples (n,10), the
resulting ‘d’ provides an inflated indicator. Hedges
(1985) effect-size correction factor for (n1+n222)
degrees of freedom was applied. This modified ‘d’
provides an unbiased estimate of the true value (Cohen,
1969/88).
Cohen’s ‘d’ indicates whether a difference between
two means is large or small in relation to the spread of
scores in the sample: 020.4 is ‘negligible’; 0.4120.7 is
‘medium’; and above 0.71 is ‘large’ (Cohen, 1988).
MTES(d) (Table 4) was calculated relative to base-
line (months 1, 2, 3) for:
(1) the first three months of the treatment (months
4, 5, 6);
(2) the last three months of the treatment (months
7, 8, 9).
The results show a consistent picture of large
treatment effects for most items. The results suggest
that the first three months of the intervention had a very
strong positive effect: MTES(d) were generally greater
than 1.0 for individual items and the Global Index
produced an effect of 4.6d’s; safety committee and
subcontractors demonstrated very large effect-sizes.
During the last three-month treatment phase, d’s
ranged from 0.4 to 5.2. However, it is clear that effect-
sizes were weaker during months seven, eight and nine,
a fact evident from the Global Index of 2.0d’s. The
Measurement and goal setting to improve H&S management 875
Dow
nloa
ded
by [
Uni
vers
ity o
f C
alif
orni
a D
avis
] at
15:
29 2
6 O
ctob
er 2
014
Table 3 Mean average scores (MAS) (standard deviations shown in parentheses)
Audit item Base (1,2,3) Treat (4,5,6) Treat (7,8,9) Treat (4,5,6,7,8,9) %-change RC %
Global Index 87.7 (1.2) 38.7 Improve 75
49 (9.5) 77 (12.2) 28 Improve 55
82.4 (9.8) 33.4 Improve 65
Induction training 93 (3.5) 41 Improve 85
52 (45.1) 95 (0) 43 Improve 90
94 (2.4) 42 Improve 87.5
Toolbox training 75.7 (13) 43.7 Improve 64.3
32 (27.8) 52 (46.6) 20 Improve 29.4
63.9 (33.2) 31.9 Improve 46.9
Safety committees 95.3 (42) 95.3 Improve 95.3
0 (0) 58.7 (51) 58.7 Improve 58.7
77 (38.1) 77 Improve 77
Subcontractors 97.7 (0.6) 21.7 Improve 90.4
76 (4) 92.7 (7.5) 16.7 Improve 69.6
95.2 (5.2) 19.2 Improve 80
Safety records 93.7 (2.1) 26 Improve 80.5
67.7 (9.9) 86.3 (6.4) 18.6 Improve 57.6
90 (5.8) 22.3 Improve 69
Safety manager 82 (6.6) 9.7 Improve 54.8
72.3 (9.2) 86 (5.2) 13.7 Improve 77.4
84 (5.7) 11.7 Improve 67.3
Consideration 63.3 (3.8) 20.3 Improve 35.6
43 (1) 58.7 (3.2) 15.7 Improve 27.5
61 (4.1) 18 Improve 31.6
Table 4 Cohen’s ‘d’: mean treatment effect size indicators by item and phase
Item and phase No. of observations No. of MTES Observed ‘d’ Corrected MTES
Global Index
Base-v-Treat 1 6 (3+3) 1 5.716 d 4.573 d
Base-v-Treat 2 6 (3+3) 1 2.560 d 2.048 d
Induction training
Base-v-Treat 1 6 (3+3) 1 1.282 d 1.026 d
Base-v-Treat 2 6 (3+3) 1 1.348 d 1.078 d
Toolbox training
Base-v-Treat 1 6 (3+3) 1 2.014 d 1.611 d
Base-v-Treat 2 6 (3+3) 1 0.521 d 0.417 d
Safety committees
Base-v-Treat 1 6 (3+3) 1 32.089 d 25.671 d
Base-v-Treat 2 6 (3+3) 1 1.6280 d 1.302 d
Subcontractors
Base-v-Treat 1 6 (3+3) 1 7.587 d 6.070 d
Base-v-Treat 2 6 (3+3) 1 2.778 d 2.222 d
Safety records
Base-v-Treat 1 6 (3+3) 1 3.633 d 2.906 d
Base-v-Treat 2 6 (3+3) 1 2.231 d 1.785 d
Safety manager
Base-v-Treat 1 6 (3+3) 1 1.212 d 0.970 d
Base-v-Treat 2 6 (3+3) 1 1.830 d 1.464 d
Consideration
Base-v-Treat 1 6 (3+3) 1 7.307 d 5.846 d
Base-v-Treat 2 6 (3+3) 1 6.620 d 5.296 d
876 Cameron and Duff
Dow
nloa
ded
by [
Uni
vers
ity o
f C
alif
orni
a D
avis
] at
15:
29 2
6 O
ctob
er 2
014
decrease is very largely due to zero scores for toolbox
training and safety committees in month eight. Without
these scores, the Global Index maintains an average of
85%. Despite this decrease, results are still positive,
and notable effects are in evidence: for example; 5.3d’s
for safety consideration.
Cohen’s (1988) descriptors classify these effects as
‘large’. However, Leik (1997) suggests that there is
something arbitrary about such labels and suggests that
researchers should compare their results with similar
studies. In similar research, Duff et al. (1993) found
effects generally between 3.1d and 0.6d; they also
found a lesser effect during their second treatment.
Effect-sizes are interesting when viewed against
ANOVA significance tests (for reasons of space, not
reported here). For instance, induction training, tool-
box training and safety manager all failed to reach
significance at the 5% level. Despite this, they exhibited
‘large’ effect-sizes. ANOVA’s failure to attain signifi-
cance is probably related to low statistical power
(number of data points).
Summary of safety performance statistics
N The Global Index improved over baseline—on
average, by 39% (75%RC) for the first three
intervention months, and by 28% (55%RC) for
the last three months. Mean average scores
improved for all items across both treatment
phases. All 16 item-phases improved (range, 10–
95%) after goal setting.
N The Global Index exhibited a ‘large’ effect-size
for the first three months of the intervention
(d54.6) and for the remaining three months of
the intervention (d52.0). Effect-sizes were
strong (.0.71) for all seven performance items
across both phases, with only one exception—the
second phase of toolbox training exhibited only a
‘medium’ effect (d50.42).
N The effect was stronger during the first three
intervention months, when compared with the
last three intervention months. A downward
trend was evident in %-changes and MTES(d),
but this trend is not sufficient to suggest that
performance substantially deteriorated during
the later phase of the intervention. In any case,
the individual results for items in month eight
(intervention month five) suggest that much of
this fall in performance was due to the con-
founding factors, previously described.
The quantitative results therefore support the motiva-
tional hypothesis that goal setting can improve manage-
ment safety performance.
Discussion
Experimental control
The main shortcomings associated with this design are
the presence of potentially confounding factors that can
moderate the relationship between goals and perfor-
mance.
Rater drift
The use of ‘reliability statements’, signed by managers,
corroborated scores. Using these, management
endorsed the reliability of all nine audits, including
those months with very poor scores, minimizing the
likelihood of rater drift as a confounding factor.
Subject reactivity
The ‘Hawthorne effect’ (Mayo, 1933) can be ignored.
If performance had improved simply due to the
presence of the researcher then performance would
have risen during the baseline phase. This did not
happen.
Environmental factors
Diary records were maintained, in order to document
uncontrolled environmental changes. Four potentially
confounding variables were recorded. The only one
apparently causing a significant change in performance,
a staff holiday at month eight, is accounted for in the
analysis.
Collectively, these demonstrate experimental control
and go a long way to eliminate alternative explanations
for the performance improvements that have been
recorded.
Goal setting
This comprised three related elements: participative
team goal setting, action plans and self-reporting.
Participative team goal setting
Goal theory (Locke and Latham, 1990) predicts that
when more than five goals are pursued, or the goals are
complex, diluted goal focus and effort will result. In this
case there were seven major goals, each containing a
number of elements, which led, in part, to the adoption
of a participative, team goal setting-process.
Specific and difficult management goals were nego-
tiated, before each month of the intervention, between
the project manager, two site managers and one
foreman, with the researcher acting as facilitator. The
goals were considered difficult, by the management, as
‘they require real commitment of time and effort to
attain’. Goals and goal attainment are shown in
Table 5.
Measurement and goal setting to improve H&S management 877
Dow
nloa
ded
by [
Uni
vers
ity o
f C
alif
orni
a D
avis
] at
15:
29 2
6 O
ctob
er 2
014
Therefore, participative team goal-setting procedures
operated effectively on this case study site. Specifically,
the goal mechanism of intensity maintained goal effort.
As goals were being set at around 90% for most items,
considerable goal directed effort was evident. For
instance, induction training goals were attained, or
were within 5% of the goal, for all six intervention
months. Toolbox talk training goals were within 5% of
set goals for two-thirds of the intervention. Safety
committee goals were attained, or almost attained, for
half of the time.
In total, 42 goals were set and these were attained 17
times. Of the remaining 25 goals, 12 were within 5% of
target, leaving 13 goals that were missed by a significant
margin. This suggests management behaviours were
influenced by goals and supports previous construction
behavioural safety findings (Duff et al., 1993; Cooper et
al., 1993) that participative goal setting, even when
here extended to a management team, promoted goal
directed commitment.
Action plans
Goal theory (Wood and Locke, 1990) predicts that
when complex management goals are pursued
increased effort may not necessarily cause improved
performance. Problem-solving task strategies must also
be developed for goals to be attained. This strategy
mechanism implies that motivation and initiative are
required (Locke and Latham, 1990).
The management team devised strategy in the form
of action plans to aid the achievement of the agreed
quantitative goals, for much the same reasons as
method statements will always accompany credible
construction programmes. Action plans facilitated
optimum performance, supporting previous construc-
tion behavioural studies (Hadavi and Krizek, 1994)
suggesting that setting specific intermediate goals eases
the attainment of end goals. In the development of the
action plans, open discussion and problem solving were
frequently evident. For instance, a programme of
relevant tool-box training was devised and a safety
supervisor was appointed to oversee goal effort.
Self-reporting
Goal theory (Locke and Latham, 1990) and feedback
theory (Algera, 1990) suggest that feedback is more
effective when it is immediate. Balcazar et al. (1986)
noted that monthly (distal) feedback was less motiva-
tional than weekly (proximal) feedback. Also, Luthan
(1992) noted that self-reporting was a vastly under-
utilized management technique.
Because of this evidence, weekly self-reports were
introduced to help maintain goal focus and effort. The
project manager returned a detailed progress report,
without fail, at the close of each week. This self-
reporting complemented monthly goal setting and
action plans, and maintained the visibility of both goals
and level of attainment.
Organizational outcomes
While the improvement in measured performance is
very encouraging, it does not tell the whole story. The
following findings relate to each of the elements of the
performance measure and are based on records of
monthly goal setting and review meetings and the final
‘safety project review’ meeting.
Induction training
The approach to induction training seemed ad hoc at
the start of the contract. Bearing in mind that 40% of
major accidents occur during an employee’s first month
on a project (CIRIA, 1994), this failure to conduct
training during the baseline represented a major
management omission. After the start of the interven-
tion, the site developed control strategies to ensure that
all employees received induction training, involving
video training, risk assessment and-site specific rules.
Toolbox training
The management team admitted that toolbox training
had been seen as a ‘luxury’, due to time pressures. After
the introduction of the intervention, an agenda for
ongoing safety awareness training, involving two ‘tool-
box’ sessions each month and linked to the contract
Table 5 Goals set and goal attainment (bold5attained; underlined5within 5%)
Months 4 5 6 7 8 9
Goal—Score Goal—Score Goal—Score Goal—Score Goal—Score Goal—Score
Induction train 85 89 95 95 95 95 98 95 98 95 95 94
Toolbox train 55 86 90 61 90 80 90 90 60 0 60 66
Committees 50 94 96 92 96 100 100 92 100 0 95 84
Subcontractor 85 98 98 98 98 97 98 84 95 97 98 97
Safety records 85 96 96 93 95 92 98 90 85 79 85 90
Safety adviser 83 83 95 75 83 88 83 83 85 92 85 83
Consideration 55 59 65 66 66 65 65 60 70 61 70 55
878 Cameron and Duff
Dow
nloa
ded
by [
Uni
vers
ity o
f C
alif
orni
a D
avis
] at
15:
29 2
6 O
ctob
er 2
014
programme, was devised for all employees. These
sessions involved the use of visual aids and were
conducted by line managers themselves and focused
on areas such as excavations, scaffold, mobile towers,
PPE, electricity and housekeeping.
Safety committees
Though these meetings were a requirement of the
Group Safety Management System, they had been
always cancelled. After the intervention began, they
occurred regularly and featured a substantive agenda
and attendance by line managers, operatives, subcon-
tractor supervisors and subcontractor operatives. The
format of these meetings was adopted by several other
contracts within the group.
Subcontractor safety
This site performed well during the baseline. Strict
procedures, required by executive directors, existed for
subcontractor pre-start arrangements. Even so, perfor-
mance did improve slightly after goal setting, mainly
due to greater dissemination of risk assessments.
Bearing in mind the pre-intervention performance in
other elements of the management safety measure,
there appeared to be something of a ‘don’t do as I do,
do as I say’ attitude towards subcontractors, which can
have done little for safety leadership or culture on the
site.
Safety records and documents
After the intervention, the frequency and completeness
of site inspections of scaffolds, excavations, lifting
equipment, fire, housekeeping and site hazards all
improved. Cross-checking these records with informa-
tion from management interviews verified that these
inspection documents were accurately completed.
Safety manager actions
During the intervention the safety manager became
more involved in employee consultation, employee
training and risk assessment. Site management stated
that, previously, the safety manager’s style had been
predominantly one of ‘staff advice’, in contrast to the
‘hands-on’ approach the intervention seemed to moti-
vate.
Safety consideration
A body of evidence supports the contention that
management’s concern for safety, and in particular
their visible safety behaviour, influences operative
safety behaviours at site level (Levitt and Parker,
1976; Hinze and Parker, 1978; Hinze and Gordon,
1979; Hinze, 1981). The safety consideration item was
intended to promote safety communication. After the
start of the intervention, although there was some
improvement, the evidence suggested that a blockage
was preventing information being disseminated. The
problem appeared to be one of the foremen, the only
member of the team who exhibited any kind of
resistance to the research. This absence of proactive
safety culture was illustrated by an employee, who said:
‘listen son, when gaffers say nought, you know you’ve
done no bad’.
The project manager regularly discussed safety goals
with subordinate managers and ensured that goals were
consciously pursued. In effect, this manager became
the ‘champion of the intervention’ (Robertson et al.,
1999). Minutes of site meetings had the intervention
high on the agenda. Members of the site team admitted
that, for the first time in years, they communicated to
each other about safety and this participation strongly
supported the acceptance of goals; as one manager put
it: ‘it’s difficult to dismiss your own suggestions [action-
plans] the following month’. Again, it is shown that
participation in goal setting is a very motivational
method of setting construction goals (Cooper, 1992).
There was some evidence to suggest that site
management’s behavioural commitment to the inter-
vention was related to the actions of executive manage-
ment. The director of construction took an active
interest in scores and always discussed these with site
management when visiting the contract. This beha-
viour communicated his commitment to safety and
supports much research (Griffiths, 1985; Minter, 1991;
Dester and Blockley, 1995; Fleming et al., 1997)
suggesting that executive managers are role models
who symbolize the organization’s real values, and that
management support is an essential prerequisite for
effective safety initiatives (Lingard, 1995; Marsh et al.,
1998). Future behavioural safety initiatives might
benefit from the inclusion of measures of executive
management performance.
When considered collectively, and even though no
attitude survey was conducted, reports made to the
researcher suggest that the intervention enhanced the
safety climate of the site, thus supporting Cooper et al.
(1993) that behavioural interventions are associated
with positive safety climates (Cooper and Phillips,
1994).
Conclusions
Collectively, findings from this case study:
N demonstrate that it is possible to implement a
safety management behavioural audit as a valid
and reliable measure of management safety
performance in the construction industry; and
Measurement and goal setting to improve H&S management 879
Dow
nloa
ded
by [
Uni
vers
ity o
f C
alif
orni
a D
avis
] at
15:
29 2
6 O
ctob
er 2
014
N support the proposition that goal setting can
improve management safety performance in the
construction industry.
However, many further successful cases are required
before it would be possible to draw general conclusions
about the effectiveness of these interventions in
improving management safety behaviour.
Acknowledgement
Although, for reasons of confidentiality, the name of
the collaborating contractor and exact site location
have been omitted, the authors are indebted to the
contractor, and particularly the site manager for
agreeing to take part in this experiment and supporting
our efforts so enthusiastically. It is widely accepted that
no such intervention can succeed in its aims without
the commitment of the managers involved, and this is a
perfect example.
References
Algera, J.A. (1990) Feedback systems in organisations, in
Cooper, C.L. and Robertson, I.T. (eds) International
Review of Industrial and Organisational Psychology, Vol. 5,
John Wiley & Sons Ltd, London, pp. 169–93.
Balcazar, F., Hopkins, B.L. and Suarez, Y. (1986) A critical,
objective review of performance feedback. Journal of
Organisational Behaviour Management, 3, 174–88.
Barlow, D.H. and Herson, M. (1984) Single-Case
Experimental Designs: Strategies for Studying Behaviour
Change, Pergamon, New York.
Cameron, I. and Duff, R. (2007) A critical review of safety
initiatives using goal setting and feedback. Construction
Management and Economics (in press).
CIRIA (1994) Not Just an Accident, Video by the
Construction Industry Research and Information
Association, London.
Cohen, H.H. and Cleveland, R.J. (1983) Safety program
practices in record holding plants. Professional Safety, 28(3),
26–33.
Cohen, J. (1988) Statistical Power Analysis for the Behavioural
Sciences, Academic Press, New York.
Cooper, M.D. (1992) An examination of assigned and
participative goal setting in relation to the improvement
of safety in the construction industry, PhD thesis,
University of Manchester Institute of Science and
Technology, School of Management.
Cooper, M.D. and Phillips, R.A. (1994) Validation of a safety
climate measure. Paper presented at the Annual
Occupational Psychology Conference, Birmingham,
British Psychological Society, 3–5 January.
Cooper, M.D., Phillips, R.A., Robertson, I.T. and Duff, A.R.
(1993) Improving safety on construction sites by the
utilisation of psychologically based techniques: alternative
approaches to the measurement of safety behaviour. Revue
europeenne de Psychologie Appliquee, 43(1), pp. 33–7.
Dester, W.S. and Blockley, D.I. (1995) Safety—behaviour
and culture. Engineering, Construction and Architectural
Management, 2(1), 17–26.
Duff, A.R., Robertson, I.T., Phillips, R.A. and Cooper, M.D.
(1993) Improving Safety on Construction Sites by Changing
Personnel Behaviour, Health and Safety Executive Contract
Research Report (CRR/51), HMSO, London.
Duff, A.R., Robertson, I.T., Phillips, R.A. and Cooper, M.D.
(1994) Improving safety by the modification of behaviour.
Construction Management and Economics, 12(12), 67–78.
Eisner, H.S. and Leger, J.P. (1988) The international safety
rating systems in South African mining. Journal of
Occupational Accidents, 10(2), 141–60.
Fleming, M., Mearns, K., Flin, R. and Gordon, R. (1997)
Effectively surveying your organisation’s safety climate or
culture to accurately assess your current position and
identifying where improvements need to be made. Paper
presented at ILR Conference on Behavioural Safety,
Piccadilly Hotel, July 1997.
Glendon, A.I. and McKenna, E.F. (1995) Human Safety and
Risk Management, Chapman and Hall, London.
Griffiths, D.K. (1985) The safety attitudes of management.
Ergonomics, 28, 61–97.
Hadavi, A. and Krizek, R.J. (1994) Difficulties with imple-
mentation of goal setting for construction. ASCE Journal of
Construction Engineering and Management, 10(5), 49–54.
Health and Safety Executive (1993) The Cost of Accidents at
Work, HMSO, London.
Hedges, L.V. (1985) Statistical Methods for Meta-Analysis,
Academic Press, Orlando, FL.
Hinze, J. (1981) Human aspects of construction safety. ASCE
Journal of Construction Engineering and Management, 107(1),
61–72.
Hinze, J. and Gordon, F. (1979) Supervisor–worker relation-
ship affects injury rate. ASCE Journal of Construction
Engineering and Management, 105(3), 253–61.
Hinze, J. and Parker, H.W. (1978) Safety: productivity and
job pressure. ASCE Journal of Construction Engineering and
Management, 104(1), 27–34.
Kazdin, A.E. (1982) Single Case Research Designs: Methods for
Applied Settings, Cambridge University Press, New York.
Kurtz, N.R. (1983) Introduction to Social Statistics, McGraw-
Hill, New York.
Leik, R.K. (1997) Experimental Design and the Analysis of
Variance, Pine Forge Press, California.
Levitt, R.E. and Parker, H.W. (1976) Reducing construction
accidents: top management’s role. ASCE Journal of
Construction Engineering and Management, 102(3), 465–78.
Lingard, H. (1995) Safety in Hong Kong’s construction
industry: changing worker behaviour, unpublished PhD
thesis, University of Hong Kong, Hong Kong.
Locke, E.A. (1968) Towards a theory of task motivation and
incentives. Organisational Behaviour and Human
Performance, 3(2), 157–89.
Locke, E.A. and Latham, G.P. (1990) A Theory of Goal
Setting and Task Performance, Prentice Hall International
(UK) Ltd, London.
880 Cameron and Duff
Dow
nloa
ded
by [
Uni
vers
ity o
f C
alif
orni
a D
avis
] at
15:
29 2
6 O
ctob
er 2
014
Luthan, F. (1992) Organisational Behaviour, McGraw-Hill,
New York.
Marsh, T.W., Davies, R., Phillips, R.A., Duff, A.R.,
Robertson, I.T., Cooper, M.D. and Weyman, A. (1998)
The role of management commitment in determining the
success of a behavioural intervention. Journal of the
Institution of Occupational Health, 2(2), 45–56.
McAfee, R.B. and Winn, A.R. (1989) The use of incentives/
feedback to enhance work place safety: a critique of the
literature. Journal of Safety Research, 20, 7–19.
Mayo, E. (1933) The Human Problems of an Industrial
Civilisation, Macmillan, New York.
Minter, S.G. (1991) Creating safety culture. Occupational
Hazards, 53(8), 17–21.
Phillips, R.A. (1992) The development of a safety perfor-
mance measure for the construction industry, unpublished
MSc thesis, Department of Building Engineering,
University of Manchester Institute of Science and
Technology (UMIST).
Reason, J.T. (1990) Human Error Reduction, Cambridge
University Press, Cambridge.
Robertson, I.R., Duff, A.R., Marsh, T.W., Phillips, R.A.,
Weyman, A.K. and Cooper, M.D. (1999) Improving
Safety on Construction Sites by Changing Personnel
Behaviour (Phase II), Contract Research Report 229/
1999, Health & Safety Executive, HSE Books, HMSO,
Norwich.
Saunders, R. (1992) The Safety Audit: Designing Effective
Strategies, Pitman, London.
Whittington, C., Livingston, A. and Lucas, D.A. (1992)
Research into Management, Organisational and Human
Factors in the Construction Industry, HSE Contract
Research Report No. 45/1992, Health and Safety
Executive, HSE Books, HMSO, Norwich.
Wood, R.E. and Locke, E.A. (1990) Goal setting and strategy
effects on complex tasks, in Staw, B. and Cumming, L.L.
(eds) Research in Organisational Behaviour, Vol. 12, JAI,
Greenwich, CT, pp. 443–58.
Measurement and goal setting to improve H&S management 881
Dow
nloa
ded
by [
Uni
vers
ity o
f C
alif
orni
a D
avis
] at
15:
29 2
6 O
ctob
er 2
014