Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008

Sarah Cotterill, Peter John and Hanhua Liu

Institute for Political and Economic Governance

http://www.ipeg.org.uk

How to get those recycling boxes out: a randomised

controlled trial of a door to door recycling campaign

Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008

Overview

• Context and background

• Research design

• Canvassing and Participation Monitoring

• Preliminary results

• Costs

• Next steps

Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008

Background and context

• Government target: 50% of household waste to be recycled or composted by 2020 (31% 2006/7)

• Key actions include “Culture Change”(Defra: Waste Strategy for England 2007)

• Citizen behaviour change essential for achieving better environmental outcomes

• Many environmental acts are low effort, but need reminders, habit or brief contacts to activate

• Kerbside recycling popular ….. but not everyone does it

Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008

Encouraging recycling

• A variety of means to encourage recycling: incentives, campaigns and leaflets

• Face to face door knocking with a professional team can impact on behaviours like voting (John and Brannan, 2008)

• Canvassing has been found in other studies to raise recycling rates (Bryce et al 1997)

Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008

EMERGE Recycling Service

• Wide range of materials

• Boxes and bags

• Weekly collection

• Visibly different vehicles, sorted on street

• Promotion of the scheme

Quality of recycling service affects recycling rates(Harder et al 2006; Woodward et al 2005)

Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008

Old Trafford and Gorse Hill

Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008

The research site

Old Trafford– 53% white– Relatively deprived (some parts in lowest 6%

of English neighbourhoods)– Mix of terraced and semi-detached housing

Gorse Hill– 85% white– Less deprived (in the lowest 30% nationally)– Mix of terraced and semi-detached housing

Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008

Sample and randomisation

• 6580 households in 194 streets

• Street based design: we expect street effect

• Streets randomly assigned to– Canvass group (97 streets, 3468 houses)– Control group (97 streets, 3112 houses)

• Stratified by district (Old Trafford/Gorse Hill) and street length

• Thanks to the York Trials Unit for doing the random assignment

Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008

Canvassing

• May/June 2008 6 weeks

• 4 canvassers recruited and trained

• 3pm-7pm Mon-Fri 11am-3pm Sat

• Whole area canvassed twice

• Spoke to 2129 of the 3468 households (61% of households)

• Awareness, Attitudes, Barriers (Recyclers and Non-Recyclers) (Shaw et al 2007)

• Leaflets

Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008

Measurement

• Participation Monitoring (WRAP 2006)

– Same day as recycling collection– Independent monitor– 3 weeks– Participation = household recycles at least once

• March/April 2008

• July 2008

• October 2008

Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008

CONSORT Flow Diagram

List of streets N = 209 streets

(N = 6616 houses)

Randomised streets N = 194 streets

N = 6580 houses

Dataset amendments: Streets without houses (N = 11)

Duplicate streets (N = 4) Non residential properties

Canvass Group N = 97 streets

N = 3468 houses

Control Group N = 97 streets

N = 3112 houses

Canvassed N = 97 streets

N = 2129 houses (61%)

Participation Monitoring 97 streets monitored pre- and

post- intervention

Participation Monitoring 97 streets monitored pre- and

post- intervention

Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008

Preliminary Results: frequencies

Households Recycled before

Recycled after Change

Canvass Group

3468

1619 46.7%

1852 53.4%

233 6.7%

Control Group

3112

1682 54.0%

1651 53.1%

-31 -1.0%

Canvassing raised recycling by 7.7%

Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008

Preliminary Results: neighbourhood effects (canvass group only)

Households Recycled before

Recycled after Change

Gorse Hill 1225 (35%)

677 55.3%

685 55.9%

8 0.6%

Old Trafford

2243 (65%)

942 42.0%

1167 52.0%

225 10%

Canvassing was more effective in raising recycling rate in Old Trafford than in Gorse Hill.

Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008

Variables involved in the analysis

• Outcome variable: – change in the means of participation rate

between the pre- and post-intervention periods

• Explanatory variables:– Group (coded 0=Control Group, 1=Canvass

Group) – District (coded 0=Old Trafford, 1=Gorse Hill)– Street size (total number of households per

street)

Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008

Analysis

• Weighted test: tests whether there is a significant

difference between the intervention and control groups

• Regression analysis using the robust standard error or

Huber-White standard errors method: examines how the

outcome variable is predicted by the three explanatory

variables Group, District and Street size.

• Analyses take into account both the variation in street

size and the clustering data structure.

Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008

0.70.60.50.40.30.20.10.0-0.1-0.2-0.3-0.4-0.5-0.6-0.7

Observed Value

5.5

5.0

4.5

4.0

3.5

3.0

2.5

2.0

1.5

1.0

0.5

0.0

-0.5

-1.0

-1.5

-2.0

-2.5

-3.0

-3.5

-4.0

-4.5

-5.0

-5.5

Exp

ecte

d N

orm

al

Figure 2. Normal Q-Q Plot of change in the means of recycling participation rate

Distribution of change in means of participation rate

Normal Q-Q Plot of change in the means of recycling participation rate

Kolmogorov-Smirnov D statistic 0.099, p=0.000

Shapiro-Wilk W statistic 0.954, p=0.000

Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008

Preliminary Results 1: Weighted test of the difference in recycling rates before and after the intervention

95% CI (SE ) t p

Lower Upper

Group (canvass/control)

0.077 (0.015)

5.220 0.000 0.048 0.106

Intercept (cons) -0.010 (0.011)

-0.930 0.354 -0.031 0.011

Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008

Preliminary Results 2: Regressions with Group, District and Street Size

95% CI (SE ) t p

Lower Upper

Group (canvass/control)

0.072 (0.013)

5.420 0.000 0.046 0.098

District (Old Trafford/Gorse Hill)

-0.049 (0.014)

-3.610 0.000 -0.076 -0.022

Street Size

0.000 (0.000)

1.780 0.077 0.000 0.001

Intercept

(cons)

-0.007 (0.017)

-0.450 0.657 -0.040 0.025

Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008

Preliminary Results: the intraclass correlation coefficient (ICC)

95% CI Source SS df MS F p ICC

Lower Upper

Between street

78.582 193 0.407 1.580 0.000 0.017 0.007 0.026

Within street

1649.217 6386 0.258

SS: square sum

df: degree of freedom

MS: mean square

Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008

Preliminary Conclusions

• Canvassing successfully raised recycling

participation

• Canvassing was most successful in Old Trafford

• Street effect is not as strong as we predicted

• Street size did not affect the change in

participation

Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008

Costs

• Canvassing costs = £5605.59– Wages of 4 canvassers– Additional hours for Emerge supervision– Expenses (phone, training)– (not bags, boxes, jackets, data entry)

• Additional recyclers = 233 households

• Cost for each new household that started recycling = £24.06

Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008

Next steps

• Participation Monitoring October 2008– Recycling decay?

• Further Analysis– Ethnicity, Poverty, Street effects

• Dissemination of findings

Presentation to Randomised Controlled Trials in the Social Sciences: Methods and Synthesis York 2008

Sarah Cotterill, Peter John and Hanhua Liu

Institute for Political and Economic Governance

http://www.ipeg.org.uk

How to get those recycling boxes out: a randomised

controlled trial of a door to door recycling campaign