www.gsr.gov.uk

Government Social Research Unit

Impact and Insight Seminar

Overseas Development Institute

17 October 2005

Evidence-Based Policy at the Cabinet Office

Philip Davies PhDGovernment Social Research Unit

Cabinet OfficeLondon SW1A 2WH

www.gsr.gov.uk

www.gsr.gov.uk

Government Social Research Unit

Impact and Insight Seminar

Overseas Development Institute

17 October 2005

Making Public Policy More Evidence-Based

Philip Davies PhDGovernment Social Research Unit

Cabinet OfficeLondon SW1A 2WH

www.gsr.gov.uk

www.gsr.gov.uk

Outline

• Why evidence-based policy?

• Factors other the evidence

• Different types of evidence

• Some Problems

• Some solutions

www.gsr.gov.uk

Why Make Public Policy More Evidenced-Based?

• Effectiveness - ensure we do more good than harm

• Efficiency - use scarce public resources to maximum effect

• Service Orientation - meet citizen’s needs/expectations

• Accountability - transparency of what is done and why

• Democracy - enhance the democratic process

• Trust - help ensure/restore trust in government and public services

www.gsr.gov.uk

What is Evidence-Based Policy?

“Evidence-based policy helps people make well-

informed decisions about policies, programmes and

projects by putting the best available evidence from

research at the heart of policy development and

implementation”

(Davies, P.T., 1999a)

www.gsr.gov.uk

Factors Influencing Policy Making

Evidence

Experience & Expertise

Judgement

Resources

Values and Policy

Context

Habits & Tradition

Lobbyists & Pressure Groups

Pragmatics & Contingencies

www.gsr.gov.uk

Different Types of Evidence for Policy

Impact Evidence

Implementation Evidence

Descriptive Analytical Evidence

Economic and

Econometric Evidence

Ethical Evidence

ExperimentalQuasi-Experimental

Counterfactual

SurveysAdmin DataComparativeQualitative

Cost-BenefitCost-Effectiveness

Cost-UtilityEconometrics

ExperimentalQuasi-Experimental

QualitativeTheories of Change

Social EthicsPublic Consultation

Statistical Modelling

Multivariate Analysis

Attitudinal Evidence

SurveysQualitative

www.gsr.gov.uk

Evidence-Based Policy

Opinion-Based Policy

Evidence-Based Policy

Increasing Pressure (Time)

Adapted from: Muir Gray 1997

www.gsr.gov.uk

Problem 1 - Social Scientific Knowledge

• Not all research is of sufficient quality

• Uncertainty of scientific knowledge• Different status of different fields of knowledge

• Unclear objectives• Poor research design• Methodological weaknesses• Inadequate data reporting• Selective use of data• Unsupported conclusions

www.gsr.gov.uk

Evidence-Based Policy

Adapted from: Muir Gray 1997

Opinion-Based Policy

Increasing Pressure (Time)

Evidence-Based Policy

www.gsr.gov.uk

Policy Makers’

Evidence

• Colloquial (Contextual)

• Anything that seems reasonable

• Policy relevant

• Timely

• Clear Message

Researchers’ Evidence

• ‘Scientific’ (Context free)

• Proven empirically

• Theoretically driven

• As long as it takes

• Caveats and qualifications

Problem 2 - Different Notions of Evidence

www.gsr.gov.uk

Problem 2 – UK Policy Makers’ Types of Evidence

• Quantitative and statistical evidence

• Economic evidence

• Qualitative evidence

• Surveys and attitudinal evidence

• Consultation evidence

• Behavioural evidence

• Anecdotal evidence

• Hard and soft evidence

• International evidence• Experimental and Quasi-Experimental Evidence• Systematic review/meta-analytical evidence

XX

www.gsr.gov.uk

Problem 3 - Getting Research Into the Evidence Chain

Academics

Constituents, Consumers and Users

Media

Professional Associations

Lobbyists and Pressure Groups

Think Tanks/Opinion Formers

'Experts'

Special Advisers

Policy or Practice

www.gsr.gov.uk

Problem 4 – Uncertain Policy Logic/Theory of Change

Visit to aPrison byJuveniles

First HandExperience of

Prison Life

Exposure toPrison Life and

Prisoners asNegative Role

Models

Frightens orScares

Juveniles Awayfrom Crime

ReducesCrime andOffending

Theories of Change - A Policy Logic Model‘Scared Straight’ Programmes

Visit to aPrison byJuveniles

First HandExperience of

Prison Life

Exposure toPrison Life and

Prisoners asPositive Role

Models

Stimulates orAttracts

JuvenilesTowards Crime

IncreasesCrime andOffending

Programme Theory

Programme Evidence

www.gsr.gov.uk

Problem 5 – Inaccessibility of Evidence

For Policymakers Research Evidence Is Too:

• Long

• Verbose

• Detailed

• Dense

• Impenetrable

• Jargonesque

• Methodological

• Untimely

• Non-relevant/irrelevant

www.gsr.gov.uk

Some Solutions

• Integrate and plan research into policy strategically (CRAG)• Establish incentives to use evidence (PSG initiative)• Establish ownership of the evidence• Clarify the policy/practice issues with users ex ante• Establish users’ theory of change/logic model• Establish answerable questions• Establish the policy/practice timetable• Knowledge translation (establish key messages clearly)• Use appropriate formats for presentation (e.g. 1:3:25)• Persistence and Opportunism (Matthew Taylor, 2005)

www.gsr.gov.uk

Contactphil.davies@cabinet-office.x.gsi.gov.ukGovernment Social Research UnitCabinet OfficeAdmiralty ArchThe MallLondon SW1A 2WHEnglandTel: +44 (0)20 7276 1862Fax: +44 (0)20 7276 1450

http://www.policyhub.gov.uk