www.gsr.gov.uk
Government Social Research Unit
Impact and Insight Seminar
Overseas Development Institute
17 October 2005
Evidence-Based Policy at the Cabinet Office
Philip Davies PhDGovernment Social Research Unit
Cabinet OfficeLondon SW1A 2WH
www.gsr.gov.uk
www.gsr.gov.uk
Government Social Research Unit
Impact and Insight Seminar
Overseas Development Institute
17 October 2005
Making Public Policy More Evidence-Based
Philip Davies PhDGovernment Social Research Unit
Cabinet OfficeLondon SW1A 2WH
www.gsr.gov.uk
www.gsr.gov.uk
Outline
• Why evidence-based policy?
• Factors other the evidence
• Different types of evidence
• Some Problems
• Some solutions
www.gsr.gov.uk
Why Make Public Policy More Evidenced-Based?
• Effectiveness - ensure we do more good than harm
• Efficiency - use scarce public resources to maximum effect
• Service Orientation - meet citizen’s needs/expectations
• Accountability - transparency of what is done and why
• Democracy - enhance the democratic process
• Trust - help ensure/restore trust in government and public services
www.gsr.gov.uk
What is Evidence-Based Policy?
“Evidence-based policy helps people make well-
informed decisions about policies, programmes and
projects by putting the best available evidence from
research at the heart of policy development and
implementation”
(Davies, P.T., 1999a)
www.gsr.gov.uk
Factors Influencing Policy Making
Evidence
Experience & Expertise
Judgement
Resources
Values and Policy
Context
Habits & Tradition
Lobbyists & Pressure Groups
Pragmatics & Contingencies
www.gsr.gov.uk
Different Types of Evidence for Policy
Impact Evidence
Implementation Evidence
Descriptive Analytical Evidence
Economic and
Econometric Evidence
Ethical Evidence
ExperimentalQuasi-Experimental
Counterfactual
SurveysAdmin DataComparativeQualitative
Cost-BenefitCost-Effectiveness
Cost-UtilityEconometrics
ExperimentalQuasi-Experimental
QualitativeTheories of Change
Social EthicsPublic Consultation
Statistical Modelling
Multivariate Analysis
Attitudinal Evidence
SurveysQualitative
www.gsr.gov.uk
Evidence-Based Policy
Opinion-Based Policy
Evidence-Based Policy
Increasing Pressure (Time)
Adapted from: Muir Gray 1997
www.gsr.gov.uk
Problem 1 - Social Scientific Knowledge
• Not all research is of sufficient quality
• Uncertainty of scientific knowledge• Different status of different fields of knowledge
• Unclear objectives• Poor research design• Methodological weaknesses• Inadequate data reporting• Selective use of data• Unsupported conclusions
www.gsr.gov.uk
Evidence-Based Policy
Adapted from: Muir Gray 1997
Opinion-Based Policy
Increasing Pressure (Time)
Evidence-Based Policy
www.gsr.gov.uk
Policy Makers’
Evidence
• Colloquial (Contextual)
• Anything that seems reasonable
• Policy relevant
• Timely
• Clear Message
Researchers’ Evidence
• ‘Scientific’ (Context free)
• Proven empirically
• Theoretically driven
• As long as it takes
• Caveats and qualifications
Problem 2 - Different Notions of Evidence
www.gsr.gov.uk
Problem 2 – UK Policy Makers’ Types of Evidence
• Quantitative and statistical evidence
• Economic evidence
• Qualitative evidence
• Surveys and attitudinal evidence
• Consultation evidence
• Behavioural evidence
• Anecdotal evidence
• Hard and soft evidence
• International evidence• Experimental and Quasi-Experimental Evidence• Systematic review/meta-analytical evidence
XX
www.gsr.gov.uk
Problem 3 - Getting Research Into the Evidence Chain
Academics
Constituents, Consumers and Users
Media
Professional Associations
Lobbyists and Pressure Groups
Think Tanks/Opinion Formers
'Experts'
Special Advisers
Policy or Practice
www.gsr.gov.uk
Problem 4 – Uncertain Policy Logic/Theory of Change
Visit to aPrison byJuveniles
First HandExperience of
Prison Life
Exposure toPrison Life and
Prisoners asNegative Role
Models
Frightens orScares
Juveniles Awayfrom Crime
ReducesCrime andOffending
Theories of Change - A Policy Logic Model‘Scared Straight’ Programmes
Visit to aPrison byJuveniles
First HandExperience of
Prison Life
Exposure toPrison Life and
Prisoners asPositive Role
Models
Stimulates orAttracts
JuvenilesTowards Crime
IncreasesCrime andOffending
Programme Theory
Programme Evidence
www.gsr.gov.uk
Problem 5 – Inaccessibility of Evidence
For Policymakers Research Evidence Is Too:
• Long
• Verbose
• Detailed
• Dense
• Impenetrable
• Jargonesque
• Methodological
• Untimely
• Non-relevant/irrelevant
www.gsr.gov.uk
Some Solutions
• Integrate and plan research into policy strategically (CRAG)• Establish incentives to use evidence (PSG initiative)• Establish ownership of the evidence• Clarify the policy/practice issues with users ex ante• Establish users’ theory of change/logic model• Establish answerable questions• Establish the policy/practice timetable• Knowledge translation (establish key messages clearly)• Use appropriate formats for presentation (e.g. 1:3:25)• Persistence and Opportunism (Matthew Taylor, 2005)
www.gsr.gov.uk
[email protected] Social Research UnitCabinet OfficeAdmiralty ArchThe MallLondon SW1A 2WHEnglandTel: +44 (0)20 7276 1862Fax: +44 (0)20 7276 1450
http://www.policyhub.gov.uk