www.3ieimpact.org philip davies identifying the problem philip davies international initiative for...

15
www.3ieimpact.org Philip Davies Identifying the Problem Philip Davies International Initiative for Impact Evaluation [3ie] BCURE Evidence-Informed Decision-Making Capacity Building Workshop 1st and 2nd June 2015 Pretoria, South Africa

Upload: rosamond-clark

Post on 23-Dec-2015

213 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Www.3ieimpact.org Philip Davies Identifying the Problem Philip Davies International Initiative for Impact Evaluation [3ie] BCURE Evidence-Informed Decision-Making

www.3ieimpact.orgPhilip Davies

Identifying the Problem

Philip DaviesInternational Initiative for Impact Evaluation [3ie]

BCURE Evidence-Informed Decision-Making Capacity Building Workshop1st and 2nd June 2015Pretoria, South Africa

Page 2: Www.3ieimpact.org Philip Davies Identifying the Problem Philip Davies International Initiative for Impact Evaluation [3ie] BCURE Evidence-Informed Decision-Making

www.3ieimpact.orgPhilip Davies

What is the Problem?

Analysis Required

i. Evidence about the nature, size and dynamics of the problem

ii. The probability that it will occur and its likely frequency

iii. Who it will impact on?

iv. Who is best placed to manage / resolve the problem?

Page 3: Www.3ieimpact.org Philip Davies Identifying the Problem Philip Davies International Initiative for Impact Evaluation [3ie] BCURE Evidence-Informed Decision-Making

www.3ieimpact.orgPhilip Davies

Is Government Intervention Appropriate

• Market Failure – market cannot deliver efficiently

• Distributional Effects – by age, gender, ethnicity etc

• Asymmetric Information – insufficient information to make an informed choice

• Public Good – benefits to all, including non-payers

• Externalities – impact on 3rd parties

• Market Power – monopoly or oligopoly

Page 4: Www.3ieimpact.org Philip Davies Identifying the Problem Philip Davies International Initiative for Impact Evaluation [3ie] BCURE Evidence-Informed Decision-Making

www.3ieimpact.orgPhilip Davies

Identifying the Problem - Data Sources

Census+ near 100% coverage of entire

population+ results reliable at small area level- only every ten years- relatively limited indicator list- extremely expensive

Surveys+ clear research focus+ potential for extensive indicator list- sampling error- results often not reliable at small area

level- very expensive

Administrative Data+ near 100% coverage of population of interest+ constantly updated+ results reliable at small area level+ already collected for operational purpose- some indicators are proxies- dependent upon support of data providers- data protection issues

Source: Michael Noble, Saspri

Qualitative Data• In-depth interviews• Group interviews (including focus

groups)• Consultative methods• Observation / Participant Observ • Ethnography • Documentary analysis• Oral histories• Case studies

Page 5: Www.3ieimpact.org Philip Davies Identifying the Problem Philip Davies International Initiative for Impact Evaluation [3ie] BCURE Evidence-Informed Decision-Making

www.3ieimpact.orgPhilip Davies

Identifying the Problem Data Sources

• Statistics South Africa• Science Councils - some promoting open access• Universities• Think tanks• DPME eg evaluations and evaluation repository• Departments on administrative data, eg Treasury Budget

data most open access• Private providers eg:

– Local government data– Employment data– Data aggregators like DataFirst

• International sources eg databases, Cochrane/Campbell/3ie libraries Source Harsha Dayal, DPME

Page 6: Www.3ieimpact.org Philip Davies Identifying the Problem Philip Davies International Initiative for Impact Evaluation [3ie] BCURE Evidence-Informed Decision-Making

www.3ieimpact.orgPhilip Davies

Page 7: Www.3ieimpact.org Philip Davies Identifying the Problem Philip Davies International Initiative for Impact Evaluation [3ie] BCURE Evidence-Informed Decision-Making

www.3ieimpact.orgPhilip Davies

Distribution of All Crime in South Africa by Province

Source: www.crimestats.com

Page 8: Www.3ieimpact.org Philip Davies Identifying the Problem Philip Davies International Initiative for Impact Evaluation [3ie] BCURE Evidence-Informed Decision-Making

www.3ieimpact.orgPhilip Davies

Distribution of Homicides in South Africa by Province

Source: www.crimestats.com

Page 9: Www.3ieimpact.org Philip Davies Identifying the Problem Philip Davies International Initiative for Impact Evaluation [3ie] BCURE Evidence-Informed Decision-Making

www.3ieimpact.orgPhilip Davies

Distribution of Sexual Crimes in South Africa by Province

Page 10: Www.3ieimpact.org Philip Davies Identifying the Problem Philip Davies International Initiative for Impact Evaluation [3ie] BCURE Evidence-Informed Decision-Making

www.3ieimpact.orgPhilip Davies

Distribution of Common Robberyin South Africa by Province

Source: www.crimestats.com

Page 11: Www.3ieimpact.org Philip Davies Identifying the Problem Philip Davies International Initiative for Impact Evaluation [3ie] BCURE Evidence-Informed Decision-Making

www.3ieimpact.orgPhilip Davies

Distribution of Residential Burglaryin South Africa by Province

Source: www.crimestats.com

Page 12: Www.3ieimpact.org Philip Davies Identifying the Problem Philip Davies International Initiative for Impact Evaluation [3ie] BCURE Evidence-Informed Decision-Making

www.3ieimpact.orgPhilip Davies

Distribution of Motor Vehicle Theftin South Africa by Province

Source: www.crimestats.com

Page 13: Www.3ieimpact.org Philip Davies Identifying the Problem Philip Davies International Initiative for Impact Evaluation [3ie] BCURE Evidence-Informed Decision-Making

www.3ieimpact.orgPhilip Davies

Violence in South Africa – Causal Factors

Source, Davies et al, 2011

Page 14: Www.3ieimpact.org Philip Davies Identifying the Problem Philip Davies International Initiative for Impact Evaluation [3ie] BCURE Evidence-Informed Decision-Making

www.3ieimpact.orgPhilip Davies

Group Exercise

Take a policy issue and discuss how you might identify the nature, size and dynamics of the problem using different types of evidence and different sources of data.

NB: Do not be confined to what has been presented in this presentation

Page 15: Www.3ieimpact.org Philip Davies Identifying the Problem Philip Davies International Initiative for Impact Evaluation [3ie] BCURE Evidence-Informed Decision-Making

www.3ieimpact.orgPhilip Davies

Thank youPhilip Davies

Email: [email protected]+44 (0)207 958 8350

Visit www.3ieimpact.org