finding the important bits from primary research
TRANSCRIPT
Finding the important bits from
primary research
What Works Global Summit
28th September 2016
Dr Lisa Tompson
University College London
UCL DEPARTMENT OF SECURITY AND CRIME SCIENCE
Overview
• The ‘EMMIE-ability’ of the current crime reduction evidence
base
– At systematic review level
– At primary study level
Searching for SRs - inclusion criteria
• Systematic review or meta-analysis
• Measured outcome of crime reduction
• Focused on a single intervention
~17,000 studies
838 studies
337
studies
82 studies
Sample (82 SRs of single interventions)
Effect Size (EMMIE)
Mechanisms (EMMIE)
Moderators (EMMIE)
Implementation (EMMIE)
Economics (EMMIE)
What have we learnt?
• Appraising quality is subjective automated Q-scoring
helps to standardise
• Most reviews don’t use the language of EMMIE
– Different fields have very different reporting conventions
• The evidence is generally weak on effect, and often on
other dimensions, BUT:
– Inherent selection bias
– We need to remember that reviews rely on primary study evidence
What’s in primary studies for EMMIE?
• To date we have done systematic reviews on:
Effect Size (EMMIE)
• Randomised control trials are scarce in crime prevention
– None in anti-theft tag literature
• Threats to bias are common
StudySelection
bias
Measurement
bias
Regression
to the
mean
Contamination
effects
Farrington et al. 1993 Medium Low Medium Medium
Bamfield, 1994 High Medium Unclear Medium
DiLonardo and Clarke, 1996 Medium Medium Unclear Medium
Hayes and Blackwood, 2006 Medium Low Low Medium
Beck and Palmer, 2011 Medium Medium Medium Medium
Retailer A 2015 Medium High High Unclear
Retailer B 2015 High High High Unclear
Mechanism-related information in the studies
33%
Yes,
explicitly
19% Yes,
alluded to
48% Not
mentioned
• 25 studies mentioned tag-
related mechanisms
• We spoke to retailers and
loss prevention managers
to corroborate the
literature
Mechanism > Effect
Increase the
effort
Increase the risks Reduce the
rewards
Reduce
provocation
Remove excuses
1. Target harden 6. Extend
guardianship
11. Conceal
targets
16. Reduce
frustrations and
stress
21. Set rules
2. Control access
to facilities
7. Assist natural
surveillance
12. Remove
targets
17. Avoid
disputes
22. Post
instructions
3. Screen exits 8. Reduce
anonymity
13. Identify
property
18. Reduce
emotional
arousal
23. Alert
conscience
4. Deflect
offenders
9. Utilize place
managers
14. Disrupt
markets
19. Neutralise
peer pressure
24. Assist
compliance
5. Control
tools/weapons
10. Strengthen
formal surveillance
15. Deny
benefits
20. Discourage
imitation
25. Control drugs
and alcohol
Mechanism > Effect
Increase the
effort
Increase the risks Reduce the
rewards
Reduce
provocation
Remove excuses
1. Target harden 6. Extend
guardianship
11. Conceal
targets
16. Reduce
frustrations and
stress
21. Set rules
2. Control access
to facilities
7. Assist natural
surveillance
12. Remove
targets
17. Avoid
disputes
22. Post
instructions
3. Screen exits 8. Reduce
anonymity
13. Identify
property
18. Reduce
emotional
arousal
23. Alert
conscience
4. Deflect
offenders
9. Utilize place
managers
14. Disrupt
markets
19. Neutralise
peer pressure
24. Assist
compliance
5. Control
tools/weapons
10. Strengthen
formal surveillance
15. Deny
benefits
20. Discourage
imitation
25. Control drugs
and alcohol
Contextual variations that Moderate the effect
• Type of tag
– Hard vs. soft vs. ink tags
– Overt (with or without warning message) vs. discreet
• Type of shop
– Layout
– Staff availability and willingness to intervene
– Other security: signs, guards, CCTV
• Tagging strategy
– Product lines
– Dosage
– Source-tagged vs. tagged in store
Implementation > Mechanism > Effect
Increase the
effort
Increase the risks Reduce the
rewards
Reduce
provocation
Remove excuses
1. Target harden 6. Extend
guardianship
11. Conceal
targets
16. Reduce
frustrations and
stress
21. Set rules
2. Control access
to facilities
7. Assist natural
surveillance
12. Remove
targets
17. Avoid
disputes
22. Post
instructions
3. Screen exits 8. Reduce
anonymity
13. Identify
property
18. Reduce
emotional
arousal
23. Alert
conscience
4. Deflect
offenders
9. Utilize place
managers
14. Disrupt
markets
19. Neutralise
peer pressure
24. Assist
compliance
5. Control
tools/weapons
10. Strengthen
formal surveillance
15. Deny
benefits
20. Discourage
imitation
25. Control drugs
and alcohol
Conclusions
• Extracting information that speaks to EMMIE is a murky
business
• Synthesising EMMIE data requires in-depth understanding
– Of crime prevention theory
– Of evaluation theory
• Systematic review evidence depends on primary studies,
so it is crucial that we raise awareness of EMMIE
• Reporting guidelines are an aspiration
Thank you
Aiden Sidebottom, Amy Thornton, Lisa Tompson, Jyoti Belur,
Nick Tilley, Kate Bowers and Shane D. Johnson
University College London
UCL DEPARTMENT OF SECURITY AND CRIME SCIENCE