Understanding the impact of a workplace

health promotion program:

How do we know who got what, and how much is enough?

Dr Fiona Cocker

Evaluating Interventions

Randomised Controlled Trial (RCT)

• Aims to evaluate program/trial efficacy

• Considered the “gold standard”

• Pros: double blind, placebo controlled, strong internal validity

• Cons: Limited external/“real world” validity, limited “generalisability”

Evaluating Interventions

Pragmatic RCTs

• Aims to evaluate program/trial effectiveness, whilst reflecting the heterogeneity of individuals encountered in the “real world”

• Minimise exclusion criteria

• Functional outcomes emphasised, and measured over a sufficient time period

• Pros: rigour of randomisation, applicable to group of interest

Evaluating Interventions

“Real World” Evaluation

Evaluation aim: Implementation

Pros: more practical in a large, diverse, populations such as a multi-site workforce

Cons: no control group, may have no control over intervention

Example: Well@Work: Promoting Active and Healthy Workplaces by researchers at Loughborough University, UK

Evaluating H@W

• Healthy@Work H&WB activities and initiatives– varied widely between agencies– implemented at different times during H@W

• How do we know who got what?

• How do we determine how much is enough? – Comprehensiveness

• Potential to influence health outcomes?– How do we control for this?

What is a comprehensive WHP programme?

• Linnan et al. Results of the 2004 National Worksite Health Promotion Survey. Am J Public Health. 2008;98(8):1503-9.– Surveyed individuals “responsible for health promotion or wellness” to

identify which worksites offered “comprehensive” worksite health promotion program

• Pilkington et al. Survey of use of occupational health support. Edinburgh: Institute of Occup Med; 2002.– Surveyed 4950 randomly selected companies of varying size, sector

and region, to determine who uses occupational health support

• Used to inform exposure variable development because:– national in scope– covered multiple worksites of different sizes and industries

Components of a comprehensive work program

– Health education (skill development, awareness building)

– Supportive social & physical environment (accessible stairs or shower facilities, time to allow participation)

– Integration into organisation’s structure (management support, dedicated H&WB position/s)*

– Linkage to related programs (EAP, OH&S)

– Worksite screening (blood pressure, health checks)Linnan et al. Am J Public Health. 2008;98:1503-09.

*Pilkington et al. Institute of Occupational Medicine; 2002.

Measuring “comprehensive” or H&WB exposure: Methodology

Dose• How many H&WB activities and initiatives

did each agency implement?

• Cumulative exposure– Add activities across life of H@W to get a total score

Example of Raw Data - 2010

Total Score per Year

AGENCY 2008/09 2010 2011 2012 TotalA 15 10 20 29 74B 19 18 26 31 94C 11 8 19 35 73D 11 7 19 24 61E 19 14 24 35 92F 20 16 26 39 101G 26 21 30 47 124H 18 16 26 40 100I 15 11 36 49 111J 11 8 28 39 86K 12 9 37 46 104L 23 16 24 40 103M 10 6 34 46 96Z 13 7 21 31 72

Total Score/Year = 56

Total Score per Year

Total Score/Year = 56AGENCY 2008/09 2010 2011 2012 Total

A 15 10 20 29 74B 19 18 26 31 94C 11 8 19 35 73D 11 7 19 24 61E 19 14 24 35 92F 20 16 26 39 101G 26 21 30 47 124H 18 16 26 40 100I 15 11 36 49 111J 11 8 28 39 86K 12 9 37 46 104L 23 16 24 40 103M 10 6 34 46 96Z 13 7 21 31 72

Agency Example

2008/09 2010 2011 20120

5

10

15

20

25

30

35

40

45

50

26

21

30

47

Agency G

Year

Num

ber o

f H&W

B Ac

tiviti

es

Sensitivity of Measurement

Can we see differences between agencies?

2008/09 2010 2011 20120

5

10

15

20

25

30

35

40

45

50

26

21

30

47

15

10

20

29

10

6

34

46

Agency GAgency AAgency M

Year

Num

ber o

f H&W

B Ac

tiviti

es

What’s next?

• Currently focused on an overall indicator– Total dose

• Employee reports of what was available & what they used– measure of individual exposure

• When did agencies implement activities?– Duration i.e. from 2008 or not until 2012

• Were activities in every domain?– Quality and comprehensiveness

• Does this impact on health outcomes?– Only 2012 important, close to measurement point?