Louise ShaxsonAssociate, Delta Partnership

Impact & insight workshop

King’s College London

25 October 2007

Practical tools for evidence based policy making:

DEVELOPING LINES OF ARGUMENT

Acknowledgements

This presentation draws on my work for Defra, 2004-ongoing: it is my interpretation rather than any statement of Defra approach or policy. However, key reference documents include:

• Our Approach to Evidence & Innovation (from www.defra.gov.uk)• Defra’s pages on evidence-based policymaking (from www.defra.gov.uk) • Bielak et al (forthcoming, 2008) – “From science communications to knowledge brokering: the shift from ‘science push’ to ‘policy pull’”. Chapter ? in Science Communication in Social Context: Strategies for the Future. Springer.

Policy trajectory – external drivers

Complex policy goals are nebulous, contestable, open to multiple interpretations: viz ‘sustainability’

Future evidence needs are interdisciplinary, overlapping, and constantly emerging

Government’s role in fostering innovation is not well understood

Competing pressures to open up the policy process and improve analytical rigour – tendency is to projectise the former and programme the latter.

Three levels of policy work(Defra, 2007)

STRATEGY … setting the direction of travel by understanding and scoping existing and future context and challenges and developing capability and capacity to deliver desired outcomes

POLICY DEVELOPMENT … structuring choice for decision-makers that is based on robust evidence

POLICY DELIVERY … implementing policies with stakeholders to deliver measurable outputs

Evidence for policy is…

…any robust information that helps to turn a Department’s strategic priorities into something concrete, manageable and achievable. But…

…there are multiple criteria for robustness, which means that there is no single gold standard: it is negotiable and open to challenge & alternative interpretation. This in turn means that…

…robustness in the evidence base includes best practice in stakeholder engagement. Evidence from stakeholders isn’t something you use to test what you think you know.

Sustainable Consumption & Production

• One of four priority areas for action set out by Defra in 2005

• New products & services with lower environmental impacts across their lifecycle; new business models; new approaches to encouraging consumer behaviour change

• Little certainty of scale of challenge: in time or globally• SCP is an influencing, rather than a delivery programme:

others hold the policy levers

• Our question – how do we construct an evidence base for SCP policy?

Policy trajectory:internal drivers

Smaller policy core (Gershon & related processes) – constant downward pressure on central Govt budgets;

Policymakers have very limited opportunities to present their work to Ministers – complicated by Cabinet reshuffles;

How can we provide sufficient breadth of evidence to Ministerial discussions of the policy landscape & encourage rigorous analysis of its alternative interpretations when the reality is that time pressures drive us towards narrow channels of problem-specific questions?

Lines of argument Evidence for policy emerges from:

DataAnalytical evidenceViews from stakeholder engagement

Engage with stakeholders in a structured way to establish a ‘line of argument’ between: their particular goal definition the values inherent in how they define the goal the evidence they believe will validate their conviction

that this is the path policy should take

How interdisciplinary can you get?

In developing a method to draw out the lines of argument we drew from:

• Science policy (Stirling, Jasanoff, Gibbons et al…)• Stakeholder engagement (Chambers onwards…)• Political science (Rayner…)• Knowledge management (Snowden: Cynefin)• Organisational behaviour (IBM expertise)

‘Five Whys’• Why is this issue important?• Why is change happening?• Why do we need to intervene in the change process (to

change the rate or scale of change)?• Why should Government intervene in the change

process?

Summarise into:• Why do we need a policy on this issue?

Use this to underpin the question:• What evidence do we need to collect to inform policy

development on this issue?

How well did we do (1)?

We evaluated the technique internally, concluding that it:• Is a cost-effective yet powerful method of scoping the

evidence base for policy way• Allowed multiple and competing goals to co-exist, valuing

dissent & alternative interpretation• Broadened thinking about innovation• Communicated complex messages into policy• Communicated the policy questions to external

stakeholders

How well did we do (2)?

Real value in scoping the evidence base for policy: for new policy areas (little evidence) existing policy areas (surprisingly little policy-relevant evidence!) changed policy goals (how well-aligned is available evidence?) more strategic thinking about evidence needs for policy

By engendering rigorous dialogue, lines of argument: Show policymakers that challenge and alternative interpretation

is a inherent part of the process – doesn’t encourage a false consensus on the issues

Structure choice for decision-makers (see previously) Helps put the politics back into policymaking?

Do it yourself…

We would have conducted a backcasting exercise… which might or might not have given us the goal of…

Reducing the global environmental footprint of the UK’s production and consumption patterns