overseas development institute · research and policy through evidence – based policy advocacy...

1

Training of Trainers on “ Bridging Research and Policy Through Evidence – Based Policy Advocacy “

John Young, Ajoy DattaODI, London

[email protected] [email protected]

May, 11th – 15th ‘ 2009 Sari Pan Pacific, Jakarta – Indonesia

Overseas Development Institute

• Development Think Tank

• 60 researchers

• Research / Advice / Public Debate

• Rural / Humanitarian / Poverty & Aid / Economics / Policy Processes

• DFID, Parliament, WB, EC

• Civil Society

For more information see: www.odi.org.uk

RAPID Group• Promoting evidence-based

development policy & practice• Through

– Research – Advice – Public Affairs– Capacity-building

• Working with:– researchers– policymakers– parliamentarians– southern think tanks

for further information see: www.odi.org.uk/rapid / www.odi.org.uk/cspp

2

6 Lessons

6 lessonsPolicy processes are complex

Research-based evidence often plays a minor role…

…but can be very important d h bi i t

3

2

1

and have a big impact

Needs a holistic approach

Research organisations need additional skills

Research organisations need to really want to do it

6

5

4

Policy Processes

Identify the problem

Commission research

Analyse the results

Choose the best option

Establish the policy

Evaluation

Implement the policy

1 Policy processes are complex

3

Agenda Setting

Decision

Policy Formulation

Policy Processes

DonorsCabinet

Parliament

Monitoring and Evaluation

Making

Policy Implementation

Civil Society

Ministries

Private Sector

1 Policy processes are complex

Chronic Poverty in Uganda

1 Research often plays a minor role2

Policy makers are…

• Speed

• Superficiality

…practically incapable of using research-based evidence because of the 5 Ss…

• Superficiality

• Spin

• Secrecy

• Scientific Ignorance

Vincent Cable – Lib. Democrat MP & Shadow Minister of FinanceMore at: www.odi.org.uk/RAPID/Meetings/Evidence


4

Experience & Expertise

JudgementPragmatics & Contingencies

Factors influencing policy making

Evidence

Resources

Values and Policy

Context

Habits & Tradition

Lobbyists & Pressure Groups

Source: Phil Davies Impact to Insight Meeting, ODI, 20051 Research often plays a minor role2

Different Notions of Evidence

• Colloquial (Contextual)

• Anything that seems

Policy Makers’ Evidence

• ‘Scientific’ (Context free)

Researchers’ Evidence

Anything that seems reasonable

• Policy relevant

• Timely

• Clear Message

Source: Phil Davies Impact to Insight Meeting, ODI, 2005

• Proven empirically

• Theoretically driven

• As long as it takes

• Caveats and qualifications


Health Care in Tanzania

“The results of household disease surveys informed processes of health service reform which contributed to a 43

d 46 t d ti iand 46 per cent reduction in infant mortality between 2000 and 2003 in two districts in rural Tanzania.”

TEHIP Project, Tanzania: www.idrc.ca/tehip

1 Research can have a big impact2 3

5

An Analytical Framework

The political context –political and economic structures and processes, culture, institutional pressures, incremental vs radical change etc.

External Influences Socio-economic and cultural influences, donor policies etc

The evidence – credibility, the degree it challenges received wisdom, research approaches and methodology, simplicity of the message, how it is packaged etc

The links between policyand research communities –networks, relationships, power, competing discourses, trust, knowledge etc.

1 Needs a holistic approach2 3 4

A Practical Framework

External Influences political context

Politics and Policymaking

Policy analysis, & research

Campaigning, Lobbying

evidencelinks

Media, Advocacy, Networking Research,

learning & thinking

Scientific information exchange & validation


What you need to knowThe external environment:

• Who are the key actors?

• What is their agenda?

• How do they influence the political context?

The evidence:

• Is it there?

• Is it relevant?

• Is it practically useful?

• Are the concepts new?

• Does it need re-packaging?

Links:

• Who are the key actors?

• Are there existing networks?

• How best to transfer the information?

• The media?

• Campaigns?

p g g

The political context:

• Is there political interest in change?

• Is there room for manoeuvre?

• How do they perceive the problem?


6

What you need to doWhat need to know What need to do How to do it

Political Context:

Evidence

• Who are the policymakers?• Is there demand for ideas?• What is the policy process?

• Get to know the policymakers.• Identify friends and foes.• Prepare for policy

opportunities. • Look out for policy windows.

• Work with them – seek commissions

• Strategic opportunism –prepare for known events + resources for others

B ild t tiEvidence

Links

• What is the current theory?• What are the narratives?• How divergent is it?

• Who are the stakeholders?• What networks exist?• Who are the connectors,

mavens and salesmen?

• Establish credibility• Provide practical solutions• Establish legitimacy.• Present clear options• Use familiar narratives.

• Build a reputation• Action-research• Pilot projects to generate

legitimacy• Good communication

• Get to know the others• Work through existing

networks.• Build coalitions.• Build new policy networks.

• Build partnerships.• Identify key networkers,

mavens and salesmen.• Use informal contacts


Policy entrepreneurs

Storytellers Networkers

Engineers Fixers

1 Research organisations need additional skills2 3 4 5

Supply-side Tools

Overarching Tools- The RAPID Framework

- Using the Framework- The Entrepreneurship

Questionnaire

Context Assessment Tools- Stakeholder Analysis- Forcefield Analysis- Writeshops

- Policy Mapping- Political Context Mapping

Communication Tools- Communications Strategy- SWOT analysis- Message Design

- Making use of the mediaResearch Tools- Case Studies

- Episode Studies- Surveys

- Bibliometric Analysis- Focus Group Discussion

Policy Influence Tools- Influence Mapping & Power Mapping- Lobbying and Advocacy- Campaigning: A Simple Guide

- Competency self-assessment

1 Research organisations need additional skills2 3 4 5

7

IntentYou need to know what you want to do, and really want to do it.

• Clear policy objective

• Focused research

• More communication

• Right incentives

• Right systems

• Engage, engage, engage

1 Research organisations need to really want to do it2 3 4 5 6

6 Steps

6 steps

8

Agenda Setting

DecisionM ki

Policy Formulation

The Policy Cycle

Monitoring and Evaluation

Making

Policy Implementation

1 Understand the context

The policy cyclePolicy-makers use scientific information at various stages of the policy cycle

Jones, N. et al (2008) Political science? Strengthening science–policy dialogue in developing countries. ODI Working Paper 295.


The RAPID Framework






9

Policy Organisation Map

Small and Medium Scale Enterprise Policy Programme. Ministry ofFinance. Egypt.www.sme.gov.eg


Other tools• Civil Society Index (CIVICUS)

• Country Policy & Institutional Assessment (World Bank)

• Democracy and Governance Assessment (USAID)Assessment (USAID)

• Drivers of Change (DFID)

• Governance Questionnaire (GTZ)

• Governance Matters (World Bank Institute)

• Power Analysis (Sida)

• World Governance Assessment1 Understand the context

Develop enthusiasm to address

topic

Learn in partnership

High

leve

l of

men

t

The AIIM Matrix

1. Identify all stakeholders

2. Map them onto the alignment / i t t t i

Develop awareness

and enthusiasm

Challenge existing beliefs

Ge

ne

ral l

alig

nm

Low

Low

HighInterest in specific

topic1 Identify the key actors2

interest matrix

3. Identify who has power

4. Identify who you can influence

10

AIIM: some examples

1 Identify the key actors2

Focus on behaviour change

Project Team

Inputs Activities Outputs

Other Actors

Outcomes ImpactOutcomes ImpactOutcomes Impact

BehaviourChange1 Identify the necessary change2 3

The Three Stages

OUTCOME MAPPING:Building Learning and Reflection into Development ProgramsSarah Earl, Fred Carden, and Terry Smutylo

http://www.idrc.ca/en/ev-9330-201-1-DO_TOPIC.html

1 Identify the necessary change2 3

11

Strategic planning: FFA

5*

3

MEF

Time

Negative forces

4

1

Funds

CSOs

Positive forces

2

1

Your influence on the force

Plan:

Minister of Trade puts forward pro-

4

2


3

?

?

Total = ?

?

?

1

2

?

Total = ?

Public demand

?

1

?

?

forward propoor trade and complementary policy programme to the cabinet by April 2008

2

2

?

1 Develop a strategy for change2 3 4

SWOT Analysis• What type of policy

influencing skills and capacities do we have?

• In what areas have our staff used them more effectively?

• Who are our strongest

Strengths WeaknessesSkills and abilities

Funding linesCommitment to positions

Contacts and PartnersExisting activities

• Who are our strongest allies?

• When have they worked with us?

• Are there any windows of opportunity?

• What can affect our ability to influence policy?

Opportunities ThreatsOther orgs relevant to the

issueResources: financial,

technical, humanPolitical and policy space

Other groups or forces

1 Ensure you have the necessary capacity2 3 4 5

Competencies Framework

1 Ensure you have the necessary capacity2 3 4 5

12

M&E & Impact measurement1. Strategy and direction: Logframes;

Social Network Analysis; Impact Pathways; Modular Matrices

2. Management: ‘Fit for Purpose’ Reviews; ‘Lighter Touch’ Quality Audits; Horizontal Evaluation; Appreciative Inquiry

3 Outputs: Evaluating academic articles and3. Outputs: Evaluating academic articles and research reports; Evaluating policy and briefing papers; Evaluating websites; Evaluating networks; After Action Reviews

4. Uptake: Impact Logs; New Areas for Citation Analysis; User Surveys

5. Outcomes and impacts: Outcome Mapping; RAPID Outcome Assessment; Most Significant Change; Innovation Histories; Episode Studies

1 Establish an M&E and learning framework2 3 4 5 6

Groundwater in India• to maximise impact of DFID forest/

ground water research project in India

• Researchers, policy makers and activists

• Used framework to analyse factors in water sector in India

• Developed strategy for final phase:

– Less research

– More communication

– Developing champions in regional and national government

– Local, Regional & National advocacy campaign

ODI and poverty / the MDGs



Political context:• 2008 is just after mid-way point,

so opportunity to analyse and refocus through UN-sponsored Call-to-Action

• UK government, among other national governments and international actors, has strong political will towards progressing

Evidence:Links:• Research undertaken in

collaboration with many other eg the GAVI Alliance and Plan International




political will towards progressing on MDGs

• One reason for this political will is that many of the MDGs are off-track to meet targets

• UN High-level event in September 2008

and Plan International• Global call to Action

Against Poverty timed to coincide with the UN High-level Event on the MDGs in New York in September

• ODI worked with Bangladesh MoEd, UNIFEM, Danish MoFA, & UN to organise a side event.

13

Conclusions• The 6 steps approach works well if you

have a clear policy message and policy objective.

• Need to chose an appropriate approach for your own research.y

• Will be doing that today / tomorrow & trying out some of the tools:o Mapping the contexto Identifying the key stakeholderso Developing a strategyo Communication & M&E

Mapping the context

The RAPID Framework



etc.



14

Group work• Identify a policy issue you are

currently working on.

• Work through the RAPID Framework question list (not all of them).

• Capture the results on a flip chart.

• Feedback:– What are the important factors?

– Did you “discover” anything new?

– Did the tool work?

– How could it be improved?

Identifying the key stakeholders


topic


High

leve

l of

men

t

The AIIM Matrix



Develop awareness

and enthusiasm


Ge

ne

ral l

alig

nm

Low

Low



interest matrix



15

1 Identify the stakeholders

2 Identify the challenge

3 Identify the strategies

Develop a Community of Practice

Media, dissemination, information

Pilots and evaluations of projects

Tough!

16

How to do it1. List the key actors.2. Locate them on the matrix. 3. Identify the most influential actors.4. Identify the ones you can influence.y y5. Map where you’d like them to be and

what you could do to get them there.6. Feedback:

– What did you learn?


– Could it be imporved?

Developing a strategy

Developing a strategy • Identify the long term aim.

• Identify milestones along the way.

• Develop a strategy to achieve them.

17

Force field analysis• Invented by Kurt Lewin (1951) • for decision making, planning and

implementation• provides techniques for assessing

organisational contexts – the different forces acting on a potential change

– the relative strength of these forces

– the influence of change agents

• enables prioritisation • designed for use in group settings• both visual and analytical

Force Field Analysis

Plan:

Minister of Trade puts forward pro

Funds

CSOs

MEF

Time

4

1

5*

3

Positive forces Negative forces

4

2

2

1


forward pro-poor trade and complementary policy programme to the cabinet by April 2008

Public demand

?

?

?

2

?

Total = ?

3

?

?

Total = ?

2

2

?

1

?

?

How to do it• Identify something you want to

achieve.

• Do a force field analysis.

• Record the key discussion points.y p

• Write it up on a flip chart.

• Feedback:– What did you decide to do?

– Did it help?

– How did the tool work?

– Could it be improved?

18

KM & Learning, communication and M&E

KM & Learning, communication and M&E

KM & Learning for Policy Impact

ODI work on KM:• Literature review

• Developing KM in ODI

• Review of KM in Development Agencies

• Advisory work

• KM Toolkit

What is KM & Learning?

“… keeping track of people who ‘know the recipe’….

Goals ResultsActivities

Learnduring

“…every time we do something again we should do it better than the last time…”

Goals Results

Learnafter

Learnbefore

External networks; Colleagues; Information assets; Own knowledge

19

Different learning styles…

R fl t ActivistReflector

Theorist

Activist

Pragmatist

Different forms of knowledge

Implicit

Y

N

StartHas it been articulated?

Can it been articulated?

Explicit Tacit

Y N

N

…and different processes…

20

Too much information…

Tools for different processes• Different tools are good for different

processes:

– Creation of knowledge

– Mapping and identifying knowledgepp g y g g

– Sharing knowledge

– Managing and storing knowledge

– Learning

KM Toolkit• Strategy Development

• Management Techniques

• Collaboration MechanismsMechanisms

• Knowledge Sharing and Learning Processes

• Knowledge Capture and Storage

21

Starts with the attitude that someone has probably already done what I am about to do

Learning before: Peer Assist

done what I am about to do.

I wonder who?”

A peer assist is a meeting or workshop where people are invited from other groups and organisations to share their experience, insights and knowledge with a team who have requested some help early on in a piece of work

Peer Assist

• targets a specific technical or commercial challenge;

• gains assistance and insights from people outside the team;

• identifies possible approaches and new lines of inquiry;

• promotes sharing of learning with each other; and• develops strong networks amongst people involved

Peer Assist

What you know in your context "...the politics

accompanying

What’spossible?

ActionMultiplying Knowledge

What I know in my context

accompanying hierarchies hampers the free exchange of

knowledge. People are much more open

with their peers. They are much more

willing to share and to listen”

What weboth know

22

Stories of change

1. Situation

2. A change or challenge

Learning During: Stories

3. Action

4. Result

5. Lesson

An after action review asks 4 simple questions:

Learning after: AAR

• What was supposed to happen?

15 minute team debrief, conducted in a “rank-free”environment.

pp• What actually happened?• Why was there a

difference?• What can we learn from it?

Knowledge Audit• What are the core tasks?• What do the people doing them need

to know?• How is the knowledge generated?• How is it stored and accessed?• How is it stored and accessed?• Any problems?• What are the relationships between

producers and users?• How could it be improved?• Any leadership issues?• Any incentive problems?

23

Learning, communicationand M&E

Effective communicationis essential if your research is to lead to change :

• to “position” the research;

• to aid the research process itself;p

• to engage with specific groups;

• to disseminate research results;

• to provide information;

• to facilitate dialogue;

• to inform (public) discussion.

Communication is NOT:• Just about creating an output (that’s

just the beginning!).• Just about formatting publications

nicely.• Something that only happens at the• Something that only happens at the

end of research.• Something that happens overnight.• Without costs: both in terms of time

and money.• Linear and simple.• A substitute for project management.

24

Communications Tools• Planning Tools

o Social Network Analysis o Problem Tree Analysis o Communications Strategies

• Packaging Toolso Scenarios

o Storytellingo Storytelling

• Targeting Toolso Policy Papers o Communities of Practiceo Websites & Blogso TV/Radio etc

• Monitoring Toolso Most Significant Change (MSC) o Outcome Mapping

o Researcher Checklist

PlanningUnderstanding the challenge and developing a strategy. Most communication strategies include:

1. Objectives

2 AudiencesWho?

Audience

2. Audiences

3. Issues

5. Messages

6. Tools and activities

7. Resources

8. Timescales

9. Evaluation and learning

How? What?

Developed, tested and promoted new forms of livestock services in Indonesia included:

• Pilot projects with farmers & field staff

• Training and capacity

eg DELIVERI - Indonesia

p ydevelopment

• Institutional development

• Quality management

• Communication & advocacy

25

PackagingUsing research results to inspire and inform through eg scenarios or stories. But how to “make it stick”?

Simple

UMemorable

EUnexpected

Concrete

Credible

Emotional

Stories

Chip and Dan Heath (2006) “Made to Stick”

Engagement

Dialogue

Interesting

Amplification

Translation

Entertaining

Sticky

TargetingHow to inform audiences in the most effective and appropriate way.• Writing Policy

Papers • Building a CoP• Lobbying

“We have come to appreciate the true power of face-to-faceand voice-to-voicecommunication EveryLobbying

• Using Email • Websites • Blogging • Media

Engagement • Radio

communication. Every meaningful lesson or belief I’ve garnered in life came from someone I value, explaining the issue to me and involving me in the process of figuring out the solution.”

Dagron, (2001) Making Waves: Stories of Participatory Communication for Social Change : www.rockfound.org/Documents/421/makingwaves.pdf

ODI and poverty / the MDGs



Political context:• 2008 is just after mid-way point,

so opportunity to analyse and refocus through UN-sponsored Call-to-Action

• UK government, among other national governments and international actors, has strong political will towards progressing

Evidence:




political will towards progressing on MDGs

• One reason for this political will is that many of the MDGs are off-track to meet targets

• UN High-level event in September 2008

26

Policy briefs

Policy briefs

“I often read policy briefs for both my official and non‐official needs. I cannot think of going forward without consulting policy briefs. It expands my knowledge as I get an opportunity to understand what is happening around me.”

(Policy‐maker, India)

How to write them• Brief (1000-3000 words)

• Content• Title of the paper • Executive summary • Context and importance of the problem • Critique of policy option(s)• Critique of policy option(s) • Policy recommendations • Sources consulted or recommended • Appendices

• Style• Clear• Simple language• Sub-headers

www.odi.org.uk/RAPID/Tools/Toolkits/Communication/Writing_policy_papers.html

27

Communications activities:• 1:1 meetings• Workshops and seminars• Intranet• Web site & CD ROM

eg DELIVERI - Indonesia

• Web site & CD ROM• Annual reports etc• Study reports & Policy papers• Brochures, leaflets & newsletter• Radio, films, talk show• Policy & practical “toolkit”• Champions: GTZ P4K & SFDM

CREATEPromoting better monitoring & planning for education provision in Ghana:• Led by former DG Education

• Co-opted key current decision makers

• Established national reference group

• Active communication with all stakeholders

• Scholarships for key staff + CD for others

• Established education research centre

• Series of 2-page policy briefs

• Special features in newspapers + media interviews

Improved access to water and sanitation through collaborative action-research.• Established collaborative learning and

practice alliances (LPAs)

l i h di l &• + learning, exchange, dialogue & dissemination

• Three woreda, six regional, one national LPAs – identify needs & run “action-research” projects

• MoW runs national LPA + strong ownership

• Now a “virtual” LPA for Nile Basin

28

MFP IndonesiaA DFID-funded programme which worked with stakeholders to establish forestry policies which support sustainable livelihoods for the poor which included:p

• Research

• Pilot projects

• Capacity development

• Institutional development

• Communication & engagement

Initial planning workshop“To be effective, an innovative programme seeking to develop and socialise new approaches needs to spend around 50% of it resources on

i ti b t thi t icommunications, but this component is often disregarded and relegated to a small office next to the toilet.”

Felia Salim, Executive Director, The Partnership for Governance Reform, at the MFP communications strategySeminar, 13th June 2002, PKBI, Jakarta

Stakeholder map

29

Principles• Multi-directional

• Multimedia

• Multiple use of information

• Appropriate quality

• Efficiency

• Focus on key issues, but don’t lose anything

• Organic growth + periodic synthesis

• Targeted information outputs

• Leave capacity to seize opportunities

• Learn from existing projects

Key products

Schedule

30

Learning, communication and M&E

TargetingHow to inform audiences in the most effective and appropriate way.• Writing Policy

Papers • Building a CoP• Lobbying

“We have come to appreciate the true power of face-to-faceand voice-to-voicecommunication EveryLobbying

• Using Email • Websites • Blogging • Media

Engagement • Radio

communication. Every meaningful lesson or belief I’ve garnered in life came from someone I value, explaining the issue to me and involving me in the process of figuring out the solution.”

Dagron, (2001) Making Waves: Stories of Participatory Communication for Social Change : www.rockfound.org/Documents/421/makingwaves.pdf

M&E of policy research1. Strategy and direction: Logframes;

Social Network Analysis; Impact Pathways; Modular Matrices

2. Management: ‘Fit for Purpose’ Reviews; ‘Lighter Touch’ Quality Audits; Horizontal Evaluation; Appreciative Inquiry

3 Outputs: Evaluating academic articles and3. Outputs: Evaluating academic articles and research reports; Evaluating policy and briefing papers; Evaluating websites; Evaluating networks; After Action Reviews

4. Uptake: Impact Logs; New Areas for Citation Analysis; User Surveys

5. Outcomes and impacts: Outcome Mapping; RAPID Outcome Assessment; Most Significant Change; Innovation Histories; Episode Studies

www.odi.org.uk/RAPID/Publications/RAPID_WP_281.html

31

Logical frameworksGoal Indicator MOV Assumptions/Risks

Purpose Indicator MOV Assumptions/Risks

Output 1 Indicator MOV Assumptions/Risks√√

Output 2 Indicator MOV



M&E in ebpdn• Rick Davies

• Log Frame

• Matrices:

– Output:Purpose (Assumptions)

Actor:Purpose– Actor:Purpose (Researchers &ODI)

– Budget:Output (Ebpdn&ODI)

• Tools:– Annual meeting

– Annual report

– Baseline & EOP survey

– Logs & stories of change


• Log Frame

• Matrices:





– Annual report



32


• Log Frame

• Matrices:





– Annual report




• Log Frame

• Matrices:





– Annual report



Impact Log

33

Impact log• Partner’s research quoted in media or on a

blog: ebpdn research on Japanese Aid on BBC Afrique; Waldo Mendoza of CIES quoted in Géstion.

• ebpdn used to find position: MSc student in US got intern position in Uganda; a member citesgot intern position in Uganda; a member cites active membership of ebpdn in his on-line CVs (on other web sites)

• RAPID framework in academia: Garrett, J. “Improving Results for Nutrition” in the Journal of Nutrition (2008)

• RAPID approaches incorporated into organisations: CIDA research studies to make it more participatory

RSS Feeds

Webstats

34

Stories

M&E (more broadly) in ODI• In strategic review:

– Review of evaluations– Stories of change– External Peer Review

• Into the future:M&E ki– M&E working group

– Performance matrix– Standard project cycle & learning

processes (eg PAs, AARs) – External evaluations of

programmes– High-level themes (+ ETs)– Stories of change– External peer review

Case & Episode Studies• Classical case studies: how did

evidence shape policy decisions?– e.g. IFPRI & IDRC– Overestimate the role of research

• Episode studies: retrospective tracking back from policy change

www.idrc.ca/en/ev-26606-201-1-DO_TOPIC.html

www.ifpri.org/impact/impact.htm

www.odi.org.uk/RAPID/Publications/BRP_ITDG.html

www.odi.org.uk/RAPID/Projects/PPA0104/Index.html

www.gdnet.org/middle.php?oid=175

tracking back from policy change– e.g. PRSPs, SL, AHC– Underestimate the role of research

35

Social Network Analysis

www.odi.org.uk/RAPID/Tools/Toolkits/KM/Social_network_analysis.html

RAPID Outcome Mapping

www.odi.org.uk/RAPID/Publications/RAPID_WP_266.html

Strategies to Enhance Research Impact John Young, Research and Policy in Development

Programme, Overseas Development Institute, London The Overseas Development Institute is one of the UK’s leading Think Tanks on International Development. Working since 1960 on a wide range of development policy issues, the organization aims to inspire and inform policy and practice which lead to the reduction of poverty, by locking together high quality applied research, practical policy advice, and policy-focused dissemination and debate. Over the last five years, ODI’s Research and Policy in Development programme (RAPID) has been involved in research and advisory and capacity development work with a wide range of organizations throughout the developing world keen to improve the impact of their research on policy and practice, and has produced a wide range of practical guidelines and toolkits. Here’s what we’ve learned in six simple lessons. The first lesson is that policy processes are fantastically complicated. They are very rarely linear and logical. Simply presenting research results to policymakers and expecting them to put it into practice is very unlikely to work. While most policy processes do involve sequential stages from agenda setting through decision-making to implementation and evaluation, sometimes some stages take a very long time, and sometimes several stages occur more or less simultaneously. Many actors are involved: ministers, parliament, civil servants, the private sector, civil society, the media etc., and in the development sector, also the donors. All busily seeking to engage in the process directly, and trying to influence each other. While Clay and Schaeffer’s 1984 book Room for Manoeuvre1 describes “the whole life of policy as a chaos of purposes and accidents” we prefer to use the terms complex, multifactorial and non linear. The second lesson is that research-based evidence usually plays a very minor role. A recent ODI study of factors influencing chronic poverty in Uganda found that only two of twenty five were researchable issues2. In a talk on evidence-based policymaking at ODI in 2003 Vincent Cable3 said that policy makers are practically incapable of using research-based evidence because of the 5 Ss… Speed – they have to make decisions fast; Superficiality – they cover a wide brief; Spin – they have to stick to a decision, at least for a reasonable period of time; Secrecy – many policy discussions have to be held in secret; and Scientific Ignorance – few policy makers are scientists, and don’t understand the scientific concept of testing a hypothesis. At another ODI meeting, Phil Davies, then Deputy Director, Government and Social Research Unit, UK Cabinet Office, described how policy makers tend to be more heavily influenced by their own values, experience, expertise and judgement, the influence of lobbyists and pressure groups and pragmatism based on the amount of resources they have available. He went on to describe how researchers and policymakers have completely different concepts of what constitutes good evidence. Researchers only consider their results to be reliable if they are proven scientifically, underpinned by theory, are reluctant to say anything until it is, and then wrap it up in caveats and qualifications. Policy makers will take more or less anything that can help them to make a decision which seems reasonable and has a clear message and is available at the right time4. The third lesson is that it is possible. Research-based evidence can contribute to policies which can have a dramatic impact on peoples’ lives. Household disease surveys undertaken by the Tanzania Integrated Health Improvement informed processes of health service reform which contributed to a 43 and 46 per cent reduction in infant mortality in two districts in rural Tanzania between 2000 and 2003.”

The fourth lesson is that researchers need a holistic understanding of the context they are working in. While there are an infinite number of factors which affect how research-based evidence does or doesn’t influence policy, it is possible to get enough understanding to be able to make decisions about how to maximise the impact of research on policy and practice relatively easily. ODI has developed a simple analytical framework identifying four broad groups of factors5. The first group, which we call external influences, are those factors outside the context you are working in which affect what happens within it. Donor policies, for example, can be hugely influential in highly indebted countries. The second, the political context, includes the people, institutions and processes involved in policy making. The third are around the evidence itself, including the type, quality and contestability of the research and how it is communicated; and the fourth, which we call links, includes all of the other actors and mechanisms affecting how the evidence gets into the policy process. If researchers want to maximize the impact of their research or promote a particular policy they need to know about the key external actors: what is their agenda, and how do they influence the political context? They need to have a thorough understanding of the political context you are working in: is there political interest in change, is there room for manoeuvre, how do policy makers perceive the problem? They need to decide if you have enough of the right sort of evidence to convince them of the need to change, and how best to present it, and they need to know who else can help them to bring it to the attention of policy makers: who are the key organisations and individuals, are there existing networks to use? The fifth lesson is that to influence policy, researchers need additional skills. They need to be political fixers, able to understanding the politics and identify the key players. They need to be good storytellers to synthesise simple compelling stories from the results of the research. They need to be good networkers to work effectively with all the other stakeholders, and they need to be good engineers to build a programme which pulls all of this together. Or they need to work in multidisciplinary teams with others who have these skills. ODI’s RAPID programme has assembled a wide range of toolkits6 including well known and often straightforward tools to do these things. The sixth lesson is intent – researchers need to really want to do it. Turning a researcher into a policy entrepreneur, or a research institute into a policy-focused think tank is not easy. It involves a fundamental reorientation towards policy engagement rather than academic achievement; engaging much more with the policy community; developing a research agenda focusing on policy issues rather than academic interests; acquiring new skills or building multidisciplinary teams; establishing new internal systems and incentives; spending much more on communications; producing a different range of outputs; and working more in partnerships and networks. It may also involve looking at a radically different funding model. It’s not easy, but it’s not impossible, and it can make a huge difference to the lives of people in the developing world RAPID’s aim now is to make the results of this work as widely accessible as possible, and contribute to the global challenge of promoting greater use of research based evidence in development policy. All of the research, including literature reviews, theoretical work, case studies, and action-research projects, and guidelines, frameworks and toolkits are published on its web site. RAPID is also supporting the development of a global network of policy research institutes and Think Tanks worldwide who are interested in doing the same – the Evidence-Based Policy in Development Network. Vist Rapid at www.odi.org.uk/rapid, or the ebpdn at www.ebpdn.org, or contact John Young at [email protected] if you would like to know more. 1 Clay & Schaffer (1984), Room for Manoeuvre; An Exploration of Public Policy in Agricultural and Rural Development, Heineman Educational Books, London 2 Kate Bird et al, Fracture Points in Social Policies for Chronic Poverty Reduction, ODI WP242, 2004 3 Dr Vincent Cable – MP for Twickenham, in “The Political Context”, Does Evidence Matter Meeting Series, ODI, May 2003 http://www.odi.org.uk/RAPID/Events/Evidence/Presentation_3/Cable.html 4 Phil Davies in Impact to Insight Meeting, ODI, October 2005 http://www.odi.org.uk/RAPID/Events/Impact_Insight/Presentation_1/Davies.html 5 Bridging Research and Policy in International Development: An Analytical and Practical Framework. RAPID Briefing Paper 1, October 2004. http://www.odi.org.uk/RAPID/Publications/RAPID_BP_1.html 6 http://www.odi.org.uk/RAPID/Tools/Toolkits/index.html


The RAPID Programme ODI’s Research and Policy in Development (RAPID) programme aims to improve the use of research and evidence in development policy and practice through research, advice and debate. The programme has four main themes:

• The use of evidence in policy identification, development and implementation;

• Improving communication and information systems for development agencies;

• Better knowledge management to enhance the impact of development agencies;

• Promotion and capacity building for evidence-based policy.

We would like to acknowledge support for this work from: the UK Department for International Development, the Global Develop-ment Network, the Swiss Agency for Development and Cooperation, and the Merck Foundation.


ODI is the UK’s leading independent think-tank on international develop-ment and humanitarian issues.

B etter use of research-based evidence in development policy and practice can help save lives, reduce poverty and improve the quality of life. But

for this to happen more effectively researchers need to do three things:

First, they need to develop a detailed understanding of i) the policymaking process – what are the key influencing factors, and how do they relate to each other? ii) the nature of the evidence they have, or hope to get – is it credible, practical and operationally useful? and iii) all the other stakeholders involved in the policy area – who else can help to get the message across?

Second, they need to develop an overall strategy for their work – identify political supporters and opponents, keep an eye out for,

and be able to react to policy windows, ensure the evidence is credible and practically useful, and build coalitions with like-minded groups.

Third, they need to be entrepreneurial – get to know, and work with the policymakers, build long term programmes of credible research, communicate effectively, use participatory approaches, identify key networkers and sales-men and use shadow networks.

Based on over five years of theoretical and case study research, ODI’s Research and Policy in Development programme has developed a simple analytical framework and practical tools that can help researchers to do this.

Why research-policy links matter Often it seems that researchers, practitioners and policymakers live in parallel universes. Researchers cannot understand why there is resistance to policy change despite clear and convincing evidence. Policymakers bemoan the inability of many researchers to make their findings accessible and digestible in time for policy decisions. Practitioners often just get on with things.

Yet better utilisation of research and evidence in development policy and practice can help save lives, reduce poverty and improve the quality of life. For example, the results of household disease surveys in rural Tanzania informed a process of health service reforms which contributed to over 40% reductions in infant mortality between 2000 and 2003 in two districts.

Indeed, the impact of research and evidence on development policy is not only beneficial – it is crucial. The HIV/AIDS crisis has deepened

Bridging Research and Policy in International DevelopmentAn analytical and practical framework

Briefing Paper October 2004

The problemThe Policy Process: ‘The whole life of policy is a chaos of purposes and accidents. It is not at all a matter of the rational implementation of the so-called decisions through selected strategies’ – Edward Clay, 1984Relevance: ‘Most policy research on African agriculture is irrelevant to agricultural and overall economic policy in Africa’ – Steve Were Omamo, 2003Policy Uptake: policymakers ‘seem to regard “research” as the opposite of “action” rather than the opposite of “ignorance”.’ – Martin Surr, 2002Cost Effectiveness of Donor Resources: ‘Donor countries spend over US$2bn annually on development research. Is this value for money?’ – RAPID Programme, 2003

2

Briefing Paper

in some countries because of the reluctance of governments to implement effective control programmes despite clear evidence of what causes the disease and how to prevent it spreading.

What influences research to policy uptake? The RAPID frameworkOften, the link between research and policy, or evidence and practice, is viewed as a linear process, whereby a set of research findings or lessons shift from the ‘research sphere’ over to the ‘policy sphere’, and then has some impact on policymakers’ decisions and practical programmes. Reality tends to be much more dynamic and complex, with two-way processes between research, policy and practice, shaped by multiple relations and reservoirs of knowledge.

The traditional question ‘How can research be transported from the research to the policy sphere?’ has been replaced by a more complex question: ‘Why are some of the ideas that circulate in the research/policy networks picked up and acted on, while others are ignored and disappear?’.

ODI’s theoretical, case study and practical work has identified a wide range of inter-related factors, which determine whether research-based and other forms of evidence are likely to be adopted by policy-makers and practitioners. These factors can broadly be divided into three overlapping areas: the political context; the evidence; and the links between policy and research communities, within a fourth set of factors: the external context. The interplay of these four areas is laid out in Figure 1: The RAPID frame-work. The framework should be seen as a generic, perhaps ideal, model. In some cases there will not be much overlap between the different spheres; in others the overlap may vary considerably.

Political context: politics and institutionsResearch-policy links are dramatically shaped by the political context. The policy process and the

Figure 1. The RAPID framework

External InfluencesSocio-economic and cultural influences, donor policies Politics and

policymaking

Research, learning and thinking

Media, advocacy, networking

evidencelinks

political context

production of research are in themselves political processes from start to finish. Key influencing factors include: • The extent of civil and political freedoms in a

country; • Political contestation, institutional pressures and

vested interests; • The attitudes and incentives among officials,

their room for manoeuvre, local history, and power relations.

In some cases the political strategies and power relations are obvious, and are tied to specific institu-tional pressures. Ideas circulating may be discarded by the majority of staff in an organisation if those ideas elicit disapproval from the leadership.

Evidence: credibility and communication Our findings and experience suggest that the quality of the research is important for policy uptake. Policy influence is affected by topical relevance and, as importantly, the operational usefulness of an idea; it helps if a new approach has been piloted and the document can clearly demonstrate the value of a new option. A critical issue affecting uptake is whether research has provided a solution to a problem.

The other key set of issues here concern communication. The sources and conveyors of evidence, the way new messages are packaged (especially if they are couched in familiar terms) and targeted can all make a big difference. For example, marketing is based on the insight that people’s reaction to a new product or idea is often determined by the packaging rather than the content in and of itself. The key message is that communication is a very demanding process and it is best to take an interactive approach. Continuous interaction leads to greater chances of successful communication than a simple or linear approach.

Links: influence and legitimacyThird, our work emphasises the importance of links; of communities, networks and intermediaries (for example, the media and campaigning groups) in affecting policy change. Some of the current literature focuses explicitly on various types of networks, such as policy communities, epistemic communities, and advocacy coalitions. While systematic understanding remains limited, issues of trust, legitimacy, openness and the formalisation of networks have emerged as important. Existing theory stresses the role of translators and communicators. It seems that there is often an under-appreciation of the extent and ways that intermediary organisations and networks impact on formal policy guidance documents, which in turn influence officials.

External influencesFinally, a synthesis of the RAPID experience emphasises the impact of external forces and donors

3

Briefing Paper

actions on research-policy interactions. While many questions remain, key issues here include the impact of international politics and processes, as well as the impact of general donor policies and specific research-funding instruments. Broad incentives, such as EU Accession or the Poverty Reduction Strategy Paper (PRSP) process, can have a substantial impact on the demand for research by policymakers. Trends towards democratisation and liberalisation and donor support for civil society are also having an impact. Much of the research on development issues is undertaken in the North, raising concerns of relevance and beneficiaries’ access to the findings. A substantial amount of research in the poorest countries is funded by international donors, which also raises a range of issues around ownership, whose priorities, use of external consultants and perceived legitimacy. As policy processes become increasingly global, this arena will increase in importance.

However, although evidence clearly matters, there has been very limited systematic understanding of when, how and why evidence informs policy. This Briefing Paper provides a synthesis of the main conclusions of recent ODI work in this area and makes recommendations for how research can better contribute to pro-poor policy and practice.

PRSPs: a case study of research-policy linkagesIn September 1999, the World Bank and International Monetary Fund (IMF) adopted a new approach to aid – Poverty Reduction Strategy Papers (PRSPs). How did the idea of the PRSP come to be adopted? What was the role of research in this process – both ‘academic research’ in general and the ‘applied policy research’ within the World Bank and IMF? An ODI case study traces the various factors that contributed to this far-reaching policy shift.

Political Context: The most important contextual factor that shaped the PRSP initiative was the convergence of debates and controversies in the field of international development in the late 1990s. This led to a widespread sense of there being ‘a problem’ within the international development policy field even though policymakers did not agree on the exact nature of the problem. The challenges that needed to be addressed – particularly by the World Bank and the IMF – included: • The questioning of the mandates of the IMF

and World Bank – in the light of the 1997 Asia Crisis and the failure of Structural Adjustment Programmes (SAPs) to resolve Africa’s development problems;

• The 1999 Review of the Heavily Indebted Poor Countries (HIPC) Initiative and the campaign to make debt relief ‘broader, deeper, faster, better’;

• The need to operationalise the new conceptual framework for aid put forward by World Bank President James Wolfensohn’s Comprehensive Development Framework (CDF).

The PRSP initiative can be viewed as bringing together all these interlinked concerns, and providing answers or at least partial solutions to the issues that needed to be addressed. It therefore received broad-based support from many different parties.

Evidence: There were three main types of evidence that influenced the emergence of the PRSP initiative. First, academic research contributed, often indirectly, to the major shifts in international development discourse towards poverty reduction, participation, and aid effectiveness. Second, there were important pieces of applied policy research undertaken in the late 1990s, in particular the research related to the ESAF reviews, the HIPC review, the Strategic Partnership with Africa (SPA) Working Groups, and the NGO research on debt relief. This evidence focused more on providing policy recommendations and operational solutions. This was seen as particularly credible when it was commissioned by the IFIs themselves or other donors, demonstrated analytical rigour, and was communicated in a language that was accessible and relevant to World Bank and IMF staff and other donor agencies. Third, an extremely powerful demonstration effect was provided by the positive experience of Uganda in drafting the Poverty Eradication Action Plan (PEAP). This did much to convince policymakers of the feasibility and merits of the poverty reduction strategy model.

Links: The PRSP story is characterised by a multitude of links between policymakers and researchers in main institutional actors – the World Bank and IMF, Strategic Partnership with Africa (SPA), UK and US governments, and the NGO movement. As one interviewee put it, ‘none of the players is more than two handshakes away from any of the others’. The formal and informal networks contributed to the speed with which the PRSP ideas were spread and accepted in international development policy.

Examples of ODI work on research-policy linkages ODI has used this framework extensively in its research and advisory work, including:

• to analyse four major policy events: the adoption of PRSPs; the development of an ethical charter by humanitarian agencies; animal health policies in Kenya; the Sustainable Livelihoods Approach;

• to analyse 50 summary cases studies as part of Phase I of the GDN Bridging Research and Policy Project (Court and Young, 2003);

• to structure literature reviews focusing on communications issues, knowledge management, the role of Civil Society Organizations, and how networks work;

• in a study of research-policy interaction in HIV/AIDS in developing countries;• in evaluations of the impact of internal policy papers on bilateral donor

policy;• in workshops and seminars with researchers, practitioners and policymakers

in Botswana, Morocco, India, Moldova, Kenya, UK and USA.

For more information on projects, publications and lessons, please visit: www.odi.org.uk/rapid


111 Westminster Bridge Road, London SE1 7JD

Tel: +44 (0)20 7922 0300

Fax: +44 (0)20 7922 0399

Email: [email protected]

Briefing Papers present objective information on important development issues. Readers are encour-aged to quote or reproduce material from them for their own publications, but as copyright holder, ODI requests due acknowledge-ment and a copy of the publication.

This and other ODI Briefing Papers are available from www.odi.org.uk

© Overseas Development Institute 2006 ISSN 0140-8682

Briefing Paper

When does evidence influence policy?

Emerging results from this and a synthesis of the other ODI studies seems to indicate that research-based and other forms of evidence is more likely to contribute to policy if:• It fits within the political and institutional limits

and pressures of policymakers, and resonates with their assumptions, or sufficient pressure is exerted to challenge them;

• The evidence is credible and convincing, provides practical solutions to pressing policy problems, and is packaged to attract policymakers’ interest;

• Researchers and policymakers share common networks, trust each other, and communicate effectively.

But these three conditions are rarely met in practice. Although researchers and practitioners can control the credibility of their evidence and

ensure they interact with and communicate well with policymakers, they often have limited capacity to influence the political context within which they work. Resources are also limited, and researchers and practitioners need to make choices about what they do. By making more informed, strategic choices, researchers can maximise their chances of policy influence.

What can researchers do?Evidence from ODI’s work so far provides preliminary recommendations in four areas, which are laid out in Table 1.

RAPID has been testing and developing the practical applications of this framework through a series of case studies and international workshops. It is clear that the conditions of the political context, the evidence, the links and the external factors vary greatly according to the particular situation. Further information on the use of the framework in a variety of specific contexts will be presented in subsequent Briefing Papers.

Table 1: How to influence policy and practice

What researchers need to know What researchers need to do How to do it

Political Context:• Who are the policymakers? • Is there policymaker demand for new

ideas? • What are the sources / strengths of

resistance?• What is the policymaking process?• What are the opportunities and timing for

input into formal processes?

• Get to know the policymakers, their agendas and their constraints.

• Identify potential supporters and opponents.

• Keep an eye on the horizon and prepare for opportunities in regular policy processes.

• Look out for – and react to – unexpected policy windows.

• Work with the policymakers.• Seek commissions.• Line up research programmes with high-

profile policy events.• Reserve resources to be able to move

quickly to respond to policy windows. • Allow sufficient time and resources.

Evidence:• What is the current theory?• What are the prevailing narratives?• How divergent is the new evidence?• What sort of evidence will convince

policymakers?

• Establish credibility over the long term.• Provide practical solutions to problems.• Establish legitimacy.• Build a convincing case and present clear

policy options.• Package new ideas in familiar theory or

narratives.• Communicate effectively.

• Build up programmes of high-quality work.• Action-research and Pilot projects to

demonstrate benefits of new approaches.• Use participatory approaches to help with

legitimacy and implementation.• Clear strategy for communication from the

start.• Face-to-face communication.

Links:• Who are the key stakeholders?• What links and networks exist between

them?• Who are the intermediaries, and do they

have influence?• Whose side are they on?

• Get to know the other stakeholders.• Establish a presence in existing networks.• Build coalitions with like-minded

stakeholders.• Build new policy networks.

• Partnerships between researchers, policymakers and policy end-users.

• Identify key networkers and salesmen.• Use informal contacts.

External Influences:• Who are main international actors in the

policy process?• What influence do they have?• What are their aid priorities?• What are their research priorities and

mechanisms?• What are the policies of the donors funding

the research?

• Get to know the donors, their priorities and constraints.

• Identify potential supporters, key individuals and networks.

• Establish credibility. • Keep an eye on donor policy and look out

for policy windows.

• Develop extensive background on donor policies.

• Orient communications to suit donor priorities and language.

• Cooperate with donors and seek commissions.

• Contact (regularly) key individuals.

Sources and Further InformationThis Briefing Paper is based on work conducted in the RAPID Programme at ODI, and particularly draws on the book: Court, J., Hovland, I. and Young, J. (2004) Bridging Research and Policy in International Development: Evidence and the Change Process, ITDG.

For more information please contact: John Young ([email protected]), RAPID, Overseas Development Institute, 111 Westminster Bridge Road, London SE1 7JD, UK or visit www.odi.org.uk/rapid.

Working with complexity: Six steps to maximise the impact of research on policy and practice

Enrique Mendizabal and John Young, ODI, London

RAPID’s work on the research-policy interface has shown that it is complex, multi-factorial, non-linear, and highly context specific. What works in one context may not work in another. Developing effective strategies in complex environments is not straightforward. Simple tools such as cost benefit analysis, logical frameworks, traditional project management tools and others may not work as they fail to take into account the existing complexity. This paper describes how a series of tools can be used in sequence to develop strategies to maximize the impact of research-based evidence on policy and practice. It draws on concepts of complexity1, on outcome mapping tools developed by the International Development Research Centre2 and tools for policy engagement assembled and developed by the ODI Research and Policy in Development Programme3, which have been field tested through over 30 workshops and training courses worldwide. Starting from an intention to use some research-based evidence to promote a specific policy or practice, the first step is to map the policy context around that specific policy issue and identify the key factors which may influence the policy process. The RAPID framework4 provides a useful checklist of questions for this, including questions about the key external actors (What is their agenda, and how do they influence the political context?); the political context itself (Is there political interest in change, Is there room for manoeuvre, How do policy makers perceive the problem?); the research-based evidence (Do you have it? Is it credible, Is it contested?); and the other stakeholders (Who else can help to bring it to the attention of policy makers? Who are the key organisations and individuals? Are there existing networks to use?). A range of other more sophisticated context mapping tools are also available. Many of them are described in Mapping Political Context: A Toolkit for Civil Society Organisations (Hudson et al, 2006)5. The second step is to identify the key influential stakeholders. RAPID’s Alignment, Interest and Influence Matrix (AIIM) can be used to map actors along three dimensions: the degree of alignment with the proposed policy (on the y axis), their level of interest in the issue (on the x axis), and their ability to exert influence on the policy process (on the z axis – or by otherwise indicating their degree of influence on the 2-dimensional matrix). Actors who are highly interested and highly aligned should be natural allies and collaborators, actors who are highly interested but not aligned are potential obstacles, and need to be brought into alignment, or somehow prevented from creating obstacles. Stimulating enthusiasm among powerful actors who are highly aligned but not interested can increase the chance of success. Stimulating enthusiasm among actors who are not highly aligned risks creating more enemies unless they can also be brought into alignment. Their level of influence will help identify key target audiences. The third step is to identify the changes needed among the key stakeholders if they are to support the desired policy outcome. Outcome Mapping6 emphasizes that long term impact only occurs through behavior change that surpass the life-time of the project. Focusing on those actors it is possible to influence it is important to describe as precisely as possible their current behavior, the behavior that is needed if they are contribute to the required policy process (the “Outcome Challenge”) and short and medium term intermediate behaviours (or “Progress Markers”) which can be monitored to ensure

the priority stakeholders are moving in the right direction and responding to the efforts of the programme. The short term behavior change is usually a direct reaction to project activities, the medium term often demonstrate stakeholders taking the initiative, while the final behaviours should describe stakeholders influencing others. Having identified the necessary behavior changes, the fourth step is to develop a strategy to achieve the milestone changes in the process. There are many strategic planning tools that can be used for this. Force Field Analysis7 is a flexible tool that can be used to identify the forces supporting and opposing the desired change and suggest concrete responses. The forces can be ranked first according to their degree of influence over the change, and then according to the degree of control it is possible for the project team to exert over them. Activities can then be identified to reduce the high negative forces and to increase low positive forces. Sometimes it is not possible to influence actors directly and it is necessary to target others who can influence them. This might mean rethinking the priority stakeholders. A Strategy Map8 can also be a useful way of visualising the emerging strategy to identify common lines of action and facilitate coordination. The fifth step is to consider the competencies required to successfully operationalise the strategy. Complexity theory conceptualizes competence as an evolving set of systems, processes and skills to enable actors to make the right decisions and act, rather than a pre-determined set of capabilities. Focusing on strategy development, learning-oriented management techniques, collaboration mechanisms, knowledge capture and storage, knowledge sharing and learning, and resource mobilisation, the 6 competencies framework can be used to identify the required competencies, the existing competencies, the steps in between, and as a capacity monitoring tool. Strengths, weaknesses, opportunities and threats (SWOT) analysis is another useful tool to identify whether a project has the necessary resources to achieve its objectives, which also recognizes the potential impact of external influences9. The final step is to develop a monitoring and learning system, not only to track progress and make any necessary adjustments, and assess the effectiveness of the approach, but also to learn lessons for the future. Simply recording the results of using these planning steps, noting the attainment of progress markers and achievement of improved competency levels, and simple logs of unexpected events should allow the team to produce and use knowledge about policy content, context, the strategy and activities, outcomes (behaviour changes), the skills, competencies and systems necessary. A wide range of more complex tools for monitoring and evaluating the impact of research and policy are available in “Making a Difference: M&E of Policy Research (Hovland 2007)10. Crucial to the collection of knowledge is sharing it and using it. Intranet systems can be very useful, but sometimes the most basic face to face or phone to phone communications can produce the best results. Understanding how people learn is also important as learning methods need to take this into consideration. 1 Exploring the science of complexity: Ideas and implications for development and humanitarian efforts. Ben Ramalingam et al, ODI February 2008. http://www.odi.org.uk/RAPID/Publications/RAPID_WP_285.html 2 The International Development Research Centre, Canada; http://www.idrc.ca 3 The RAPID Programme at ODI: http://www.odi.org.uk/rapid 4 Bridging Research and Policy in International Development: An Analytical and Practical Framework. RAPID Briefing Paper 1, October 2004. http://www.odi.org.uk/RAPID/Publications/RAPID_BP_1.html 5 Mapping Political Context: A Toolkit for Civil Society Organisations. Robert Nash, Alan Hudson and Cecilia Luttrell, ODI July 2006. http://www.odi.org.uk/rapid/Tools/Toolkits/Mapping_Political_Context/Index.html 6 Outcome Mapping: http://www.idrc.ca/en/ev-26586-201-1-do_topic.html / http://www.outcomemapping.ca 7 Force Field Analysis: http://www.odi.org.uk/rapid/Tools/Toolkits/Policy_Impact/Forcefield_analysis.html 8 Strategy Maps: http://www.idrc.ca/en/ev-28388-201-1-DO_TOPIC.html (towards the bottom of the page) 9 SWOT Analysis: http://www.odi.org.uk/rapid/Tools/Toolkits/Policy_Impact/SWOT_analysis.html 10 Making a difference: M&E of policy research. Ingie Hovland, ODI Working Paper 281, July 2007: http://www.odi.org.uk/rapid/Publications/RAPID_WP_281.html

Tools for Knowledge and Learning

20

5. Outcome Mapping

Introduction

As development is essentially about people relating to each other and their environments, the focus of Outcome Mapping is on people. The originality of the methodology is its shift away from assessing the development impact of a programme (defined as changes in state: for example, policy relevance, poverty alleviation, or reduced conflict) and toward changes in the behaviours, relationships, actions or activities of the people, groups and organisations with which a development programme works directly. This shift significantly alters the way a programme understands its goals and assesses its performance and results. Outcome mapping establishes a vision of the human, social and environmental betterment to which the programme hopes to contribute and then focuses monitoring and evaluation on factors and actors within that programme’s direct sphere of influence. The programme’s contributions to development are planned and assessed based on its influence on the partners with whom it is working to effect change. At its essence, development is accomplished by, and for, people. This is, then, the central concept of outcome mapping. Outcome mapping does not belittle the importance of changes in state (such as cleaner water or a stronger economy) but instead argues that for each change in state there are correlating changes in behaviour.

Detailed description of the process

Intentional Design helps a programme establish consensus on the macro-level changes it will help to bring about and plan the strategies it will use. It helps answer four questions: Why? (What is the vision to which the programme wants to contribute?); Who? (Who are the programme’s boundary partners?); What? (What are the changes that are being sought?); and How? (How will the programme contribute to the change process?). Outcome and Performance Monitoring provides a framework for the ongoing monitoring of the programme’s actions and the boundary partners’ progress toward the achievement of outcomes. It is based largely on systematised self-assessment. It provides the following data collection tools for elements identified in the Intentional Design stage: an Outcome Journal (progress markers); a Strategy Journal (strategy maps); and a Performance Journal (organisational practices). Evaluation Planning helps the programme identify evaluation priorities and develop an evaluation plan. Figure 3 illustrates the three stages of outcome mapping. Figure 3: The three stages of outcome mapping

STEP 1: Vision STEP 2: Mission STEP 3: Boundary Partners STEP 4: Outcome Challenges STEP 5: Progress Markers STEP 6: Strategy Maps STEP 7: Organisational Practices

STEP 8: Monitoring Priorities STEP 9: Outcome Journals STEP 10: Strategy Journal STEP 11: Performance Journal

OUTCOME AND PERFORMANCE MONITORING

INTENTIONAL DESIGN

STEP 12: Evaluation Plan

EVALUATION PLANNING

Strategy Development

21

The process for identifying the macro-level changes and designing the monitoring framework and evaluation plan is intended to be participatory and, wherever feasible, can involve the full range of stakeholders, including boundary partners. Outcome mapping is based on principles of participation and purposefully includes those implementing the programme in the design and data collection so as to encourage ownership and use of findings. It is intended to be used as a consciousness-raising, consensus-building and empowerment tool for those working directly in the programme. Outcome mapping introduces monitoring and evaluation considerations at the planning stage of a programme, and moves away from the notion that monitoring and evaluation are done to a programme. Instead, it actively engages groups and teams in the design of a learning-oriented plan, with self-reflection as a core principle.

Key points/practical tips

Outcome mapping is a planning, monitoring and evaluation tool developed by IDRC of Canada (www.idrc.ca). It focuses on the following key points:

• Behavioural change: Outcomes are defined as changes in the behaviour, relationships, activities, or actions of the people, groups and organisations with which a programme works directly. These outcomes can be logically linked to a programme’s activities, although they are not necessarily directly caused by them.

• Boundary partners: Those individuals, groups and organisations with which the programme interacts directly and with which the programme anticipates opportunities for influence. Most activities will involve multiple outcomes because they have multiple boundary partners.

• Contributions: By using outcome mapping, a programme is not claiming the achievement of development impacts; rather, the focus is on its contributions to outcomes. These outcomes, in turn, enhance the possibility of development impacts – but the relationship is not necessarily a direct one of cause and effect.

Example: Knowledge sharing programme

For example, a knowledge sharing programme’s objective may be to provide communities with access to better information by means of an intranet system. Traditionally, the method of evaluating the results of this programme would be to count the number of potential users of the system, and to measure changes in the level of access after the system is installed. A focus on changes in behaviour begins instead from the premise that the intranet is a focal point for staff knowledge sharing behaviours, and that it will not be used without people perceiving there to be quality information available. The programme’s outcomes are therefore evaluated in terms of whether those responsible for knowledge sharing not only have, but also use, the appropriate tools, skills and knowledge to update and review information on the intranet. Outcome mapping provides a method for knowledge and learning programmes to plan for and assess the capacities that they are helping to build in people, groups and organisations. Outcome mapping does not attempt to replace the more traditional forms of planning, monitoring and evaluation, which focus on changes in condition or in the state of wellbeing. Instead, outcome mapping supplements these other forms by focusing specifically on related behavioural change.

Sources and further reading

• Earl, S., F. Carden, and T. Smutylo (2001) Outcome Mapping; Building Learning and Reflection into Development Programs, International Development Research Centre (IDRC), see: www.idrc.ca/en/ ev-9330-201-1-DO_TOPIC.html.

• Hovland, I. (2005) Successful Communication: A Toolkit for Researchers and Civil Society Organisations, ODI Working Paper 227, London: ODI.

• The website for a dedicated community of practice for users of outcome mapping worldwide is currently being finalised: www.outcomemapping.ca.

Tools for Bridging Research and Policy: The RAPID Context, Evidence, Links Framework

Introduction The RAPID framework can be used as a conceptual framework to help researchers and policy entrepreneurs understand the role that evidence based research plays, amongst other issues, in influencing policy. The four components of the framework can provide the user with in-depth and valuable information regarding policy windows, key policy actors and networks, gaps in the existing evidence, alternative means of communication and trends and changes in the external environment. Unfortunately, addressing all these issues can prove a daunting task. This tool can be used to ease the process. Thus, it presents some of the key questions that the researcher or policy entrepreneur should answer.

Detailed outline of the process This is a very flexible tool. The questions provided overleaf are only intended to guide the user in the process. It is the user who must assess whether the answers to these questions paint the whole picture of if other important questions remain unanswered. Once the questions have been answered the researcher or policy entrepreneur should consider what roles can the different policy actors (including him or herself) can play. For instance: Is there a need for more and/or different evidence? How can this new evidence be

produced? Should NGOs, grassroots or think tanks and research centres be doing things differently? If there is sufficient evidence, does it need to be re-packaged and presented differently?

Are the existing networks sufficient to carry research findings into the policy process? How can they be supported to improve their impact on policy? What new roles should these and new networks play?

Are policy makers and policy structures supportive of evidence based policy making? If not, how can they be made to be so? What capacities and skills do they need to use evidence and link with researchers? How can policymakers promote the production of more and more relevant and useful research?

How can the external forces be used to promote evidence based policymaking? Should the support networks and/or CSOs to promote the supply of evidence? Or should they work with policymakers to promote the demand of evidence?

Examples The RAPID programme has used this tool in its analysis of various policy processes. They can be seen at: http://www.odi.org.uk/rapid/Projects/R0040a/Summary.html

Further information and resources RAPID has produced a series of resources that can be accessed through its website at http://www.odi.org.uk/rapid/Index.html. RAPID’s Briefing Paper on bridging research and policy offers a good introduction into the subject. On page four, the Briefing Paper presents a table that can help move from the questions to an action strategy – it is available in English, French and Spanish (http://www.odi.org.uk/rapid/Publications/RAPID_BP_1.html). Similarly, other institutions working on similar issues can offer alternative and complementary frameworks to understand the links between research and policy (http://www.odi.org.uk/rapid/Links/Index.html).

The RAPID Framework: 28 Key Questions

Political Context

Evidence

Links

Political Context

1. Who are the key policy actors (including policymakers)? 2. Is there a demand for research and new ideas among

policymakers? 3. What are the sources of resistance to evidence based

policymaking? 4. What is the policy environment?

a. What are the policymaking structures? b. What are the policymaking processes? c. What is the relevant legal/policy framework? d. What are the opportunities and timing for input into

formal processes? 5. How do global, national and community-level political, social

and economic structures and interests affect the room for manoeuvre of policymakers?

6. Who shapes the aims and outputs of policies? 7. How do assumptions and prevailing narratives (which ones?)

influence policymaking; to what extend are decisions routine, incremental, fundamental or emergent, and who supports or resists change?

Evidence

1. What is the current theory or prevailing narratives? 2. Is there enough evidence (research based, experience

and statistics)? a. How divergent is the evidence?

3. What type of evidence exists? a. What type convinces policymakers? b. How is evidence presented?

4. Is the evidence relevant? Is it accurate, material and applicable?

5. How was the information gathered and by whom? 6. Are the evidence and the source perceived as credible

and trustworthy by policy actors? Why was the evidence produced?

7. Has any information or research been ignored and why?

External Environment

1. Who are main international actors in the policy process?

2. What influence do they have? Who influences them?

3. What are their aid priorities and policy agendas? 4. What are their research priorities and

mechanisms? 5. How do social structures and customs affect the

policy process? 6. Are there any overarching economic, political or

social processes and trends? 7. Are there exogenous shocks and trends that

affect the policy process?

Links 1. Who are the key stakeholders? 2. Who are the experts? 3. What links and networks exist between them? 4. What roles do they play? Are they intermediaries

between research and policy? 5. Whose evidence and research do they

communicate? 6. Which individuals or institutions have a significant

power to influence policy? 7. Are these policy actors and networks legitimate?

Do they have a constituency among the poor?

1

Identifying the key stakeholders


topic


High

leve

l of

men

t

The AIIM Matrix



Develop awareness

and enthusiasm


Gen

eral

lal

ignm

Low

Low



interest matrix



1 Identify the stakeholders

2

2 Identify the challenge

3 Identify the strategies

Develop a Community of Practice

Media, dissemination, information

Pilots and evaluations of projects

Tough!

How to do it1. List the key actors.2. Locate them on the matrix. 3. Identify the most influential actors.4. Identify the ones you can influence.y y5. Map where you’d like them to be and

what you could do to get them there.6. Feedback:

– What did you learn?


– Could it be imporved?

Force Field Analysis

Force field analysis was developed by Lewin (1951) and is widely used to inform decision-making, particularly in planning and implementing change management programmes in organisations. It is a powerful method for gaining a comprehensive overview of the different forces acting on a potential policy issue, and for assessing their source and strength. Detailed Outline of the Process

Force field analysis is best carried out in small group of about six to eight people using flip chart paper or overhead transparencies so that everyone can see what is going on. The first step is to agree the area of change to be discussed. This might be written as a desired policy goal or objective. All the forces in support of the change are then listed in a column to the left (driving the change forward) while all forces working against the change are listed in a column to the right (holding it back). The driving and restraining forces should be sorted around common themes and should then be scored according to their ‘magnitude’, ranging from one (weak) to five (strong). The score may well not balance on either side. The next step is to decide whether it might be possible to do anything about any of the forces. The greatest impact will be achieved by increasing weak supporting forces and decreasing strong opposing forces, but these may not be the forces over which you have the greatest control so it is sometimes also helpful to score the forces according to the degree of influence you can exert on them. The final chart might look like the following:

Throughout the process rich discussion, debate and dialogue should emerge. This is an important part of the exercise and key issues should be allowed time. Findings and ideas may well come up to do with concerns, problems, symptoms and solutions. It is useful to record these and review where there is a consensus on an action or a way forward. In policy influencing the aim is to find ways to reduce the restraining forces and to capitalise on the driving forces. Force field analysis is natural follow-on from Problem tree analysis which can often help to identify objectives for policy change. A useful next step on from Force field analysis is Stakeholder analysis in which the specific stakeholders for and against a change are identified, together with their power, influence and interests.

A Good Example

Force field analysis has been used in diverse fields ranging from participatory rural appraisal and social research to strategic planning and organisational change. As part of a DFID sponsored participatory poverty profiling in Bolangir, a drought prone district in Western Orissa, India, a team of facilitators from PRAXIS used various participatory tools to conduct a study of the poverty profile of the district. Seasonal migration poses a serious problem with the rural poor and a Force field analysis was conducted with a group of villagers to study the factors leading to migration. Drought and lack of land emerged as the most important factors contributing to migration. Among the forces inhibiting migration were emotional attachment to the village and excessive work during migration. The information generated from the analysis has been useful in that it has led to designing a livelihood project, to be implemented by the Government of Orissa, supported by DFID, India. Further Information

The case study above comes from: PLA Notes (1999), Issue 36, pp.17-23. IIED, London, at: www.worldbank.org/participation/PRSP/plna/plan_03604.pdf Another case details the use of Force field analysis in a school situation to assess the potential to change from teacher-centered methods of working to greater pupil participation in planning. See: www.crossroad.to/Quotes/brainwashing/force-field.htm For original literature of Force field analysis see: Lewin, K. (1951) Field Theory in Social Science, Harper and Row, New York. Simple step-by-step guides to carrying out force field analysis are available at:

www.mindtools.com/forcefld.html for examples of the use of force field analysis in management

www.psywww.com/mtsite/forcefld.html for examples of the use of force field analysis in psychology

For a brief overview see www.mycoted.com/creativity/techniques/forcefieldanal.php Examples of the application of force field analysis in different areas are available below:

Change management: www.accel-team.com/techniques/force_field_analysis.html

Health (MSH and UNICEF): http://erc.msh.org/quality/example/example5.cfm For computer software to conduct force field analysis see: http://www.skymark.com/resources/tools/force_field_diagram.asp


60

20. Peer Assists

Introduction

In Learning to Fly (Parcell and Collison, 2001), the learning cycle is described as made up of three elements:

• Learning before doing, or the process of learning before undertaking a task, activity or project o facilitated by Peer Assists

• Learning during doing, or the process of learning whilst undertaking a task, activity or project o facilitated by After Action Reviews (Tool 22)

• Learning after doing, or the process of learning after undertaking a task, activity or project o facilitated by Retrospects (Tool 22)

The Peer Assist is a tool which supports ‘learning before doing’ processes. Using the same principles as scientific peer review, it begins with the premise that, for any given activity, someone else has done something that is at least broadly similar. In order to use the peer assists, a team or group first needs to identify the right group of people, and then uses a systematic method to benefit from their insights/experience. If conducted effectively, peer assists can promote learning, and be used to strengthen mutual learning between people and groups within an organisation.


• Develop a clear definition of the problem to be addressed. It may be worthwhile doing some background research on whether similar issues have been, or are being, faced elsewhere. The definition should include a set of hoped-for outcomes from the peer assist process. Step 1 will help focus the peer assist meeting, and will also provide a long list of potential participants.

• Enlist participants. Participation can be generated either through open invitation or selection. In general, it is worth getting a good mix of people playing a range of roles across different locations, and from different positions in the organisational hierarchy, with the proviso that peer assist work best when there is some common ground, and scope for open honest interactions. Consider including people from outside, but only if this will not disrupt internal sharing. It may be worth bringing in outside experts after the internal process has been completed.

• Time the meeting carefully. The peer assist meeting should take place early enough to ensure that: i) the required participants are given enough notice and are available, and ii) the lessons can be applied effectively by the team calling the peer assist.

• Run the peer assist meeting. Effective peer assist meetings comprise six parts. o Part 1 – the learning team presents context, history and ideas regarding the task or issue at hand.

This should occur in an open and flexible manner to enable redefinition in the session. o Part 2 should allow the participants to consider the problem, and discuss issues of interest –

namely, what has been covered, and what hasn’t been covered. o Part 3 should be a session in which participants consider what the learning team might need to know

to address the problem and where might they find that knowledge. o Part 4 of the meeting should be for the participants to reflect on what has been learned from the

others and to examine options. Again, the learning team should not be the focus here. o In Part 5 of the meeting, the participants should present feedback to the learning team and answer

specific questions. This should be informal, and deal with what has been learned, what options there are and experiences elsewhere. Begin with the positive and then move on to options to do things differently. When presenting what has worked elsewhere, participants should be encouraged to describe rather than prescribe.

o In Part 6, the team who called the peer assist should acknowledge the contribution of the participants. There should be a commitment to a timeline for delivery of an action list of key lessons

Knowledge Sharing and Learning

61

learned, and what the learning team are going to do differently as a result. Finally, all the participants should be invited to reflect on what they learned, and how they might apply it going forward.

• Develop a set of lessons and related options to shape the learning team’s decision-making process and provide pointers to future actions. This document should be shared with the peer assist participants for final comments and suggestions, and then placed in a publicly accessible area such as an intranet.


• Ensure everyone is clear about the purpose of the peer assist and their roles: o Learning team listens in order to understand and learn; o Participants share knowledge and experience to help resolve the challenge without adding to the

workload.

• Participants should be given briefing materials in advance so they have time to prepare.

• As well as the participants themselves, an external facilitator and note-taker are essential. In particular, the facilitator should be from outside the teams concerned, in order to make sure the diverse needs of the participants are met.

• Allow time for the teams to socialise. Rapport is essential for open learning.

• Although the peer assist process is designed to provide input for a specific purpose or project, consider who else might benefit from the lessons learned. Always look out for opportunities to share and reuse knowledge and learning.

Example: ODI civil society engagement peer assist

As part of a very important proposal which had potential implications for the whole organisation, the RAPID team at ODI called a peer assist to gather together experiences of different researchers working with Southern civil society organisations. The range of people was diverse, with over 20 different thematic experts present, which was a few more than is usually considered ideal for a PPA. The session was facilitated very carefully, using the peer assist process. The outcome was a set of ideas for the substance of the proposal, as well as a list of previous projects of relevance, further internal contacts and useful external contacts. Key participants also shared documents of relevance, and an intranet page was created to store all of these in one location. A number of participants volunteered to be on a panel at a later stage to help bolster different areas of the proposal. Follow-up interviews were also conducted to capture stories of successful ODI engagement with this increasingly important group of stakeholders. Overall, it was one of the most extensive and systematic internal learning exercises conducted at ODI. The outcome was that ODI was awarded a Partnership Programme Agreement by DFID, the first time a research institute had ever done so. The ideas generated at the peer assist have since helped shape the resulting multi-year, multi-million pound Civil Society Partnerships Programme.


• NHS Guide to Peer Assists: www.nelh.nhs.uk/knowledge_management/km2/peer_assists_ toolkit.asp.

• Collison, C. and G. Parcell (2001) Learning to Fly, Oxford: Capstone Publishing.


64

22. After Action Reviews and Retrospects

Introduction

Organisational learning requires continuous assessment of organisational performance, looking at successes and failures, ensuring that learning takes place to support continuous improvement. The After Action Review is a simple tool to facilitate this assessment. It works by bringing together a team to discuss task, event, activity or project, in an open and honest fashion. The systematic application of properly conducted AARs across an organisation can help drive organisational change. As well as turning unconscious learning into tacit, it helps to build trust among team members and to overcome fear of mistakes. When applied correctly, AARs can become a key aspect of the internal system of learning and motivation.


There are many different ways to conduct AARs. The simplicity at the heart of the tool means there is much potential to experiment with the process and find the right ways that will work best with the group and the work item under review. The whole process should be kept as simple and as easy to remember as possible. The essence of the AAR is, however, to bring together the relevant group to think about a project, activity, event or task, and pose the following simple questions. Table 7: After action review questions

Question Purpose What was supposed to happen? What actually happened? Why were there differences?

These questions establish a common under-standing of the work item under review. The facilitator should encourage and promote discussion around these questions. In particular, divergences from the plan should be explored.

What worked? What didn’t? Why?

These questions generate reflection about the successes and failures during the course of the project, activity, event or task. The question ‘Why?’ generates understanding of the root causes of these successes and failures.

What would you do differently next time?

This question is intended to help identify specific actionable recommendations. The facilitator asks the team members for crisp and clear, achievable and future-oriented recommendations.

A Retrospect follows the AAR format, but involves asking the following more detailed questions:

• What did you set out to achieve?

• What was your plan to achieve this?

• How did this change as you progressed?

• What went well and why?

• What could have gone better?

• What advice would you give yourself if you were to go back to where you were at the start of the project?

• What were the two or three key lessons you would share with others?

• What next for you in terms of this project?

• Can you think of a story that summarises your experience of work on this project?

• What should we have learned from this project a year from now?

• Are there any lessons for you personally?


65


• Post the questions up on flipchart sheets prior to the session, with answers then written on the sheet as the session progresses. The completed sheets can then be stuck up around the room to serve as a reminder of the progress.

• Participants are participants, not a passive audience. The facilitator should prepare leading questions and may have to ask it of several people. The questions can be asked on an individual or a team basis. The team mechanism is ideal, but if suggestions are slow coming, the facilitator could go around the room asking each individual to express one thing that worked and one thing that did not.

• If there are issues with either openness or time, it may be worthwhile to gather ideas first and then facilitate the discussion in the group environment.

• Ideally, an uninvolved note-taker should be asked to minute the session. This will enable better capture of the learning.

• The actionable recommendations should be as specific as possible. For example, an AAR following a workshop could have the following recommendation: ‘Make more time to understand the audience.’ A better SAR would be ‘Make contact with the organising body representative and ask about the range of participants before planning the workshop.’

• Participants of an AAR should include all members of the team. A facilitator should be appointed to help create an open environment, promote discussion and draw out lessons learned.

• AARs should be carried out immediately, while the team is still available and memories are fresh. It is recommended that AARs be incorporated at key points during a project, activity, event or task in the early planning stage, although they are often completed at the end.

• AARs can be conducted almost anywhere, and will vary in length. For example, a 15-minute AAR can be conducted after a one-day workshop, or a much longer meeting could be held to reflect on the strategy development process throughout a large organisation.

Example: Joint AAR by CARE and WVI, with OXFAM GB and CRS, April 2005

This workshop was a consolidation of a number of country-level learning activities following the crisis caused by the tsunami of 26 December 2004. The AAR focused mainly on the four most affected countries: Indonesia, India, Sri Lanka and Thailand, with additional participation by staff from CARE Somalia. The primary purpose was to explore ways in which participant organisations could jointly improve their performance and quality of work by reflecting back on their activities and actions. It presented an opportunity for participants from various organisations to discover for themselves what happened and why, and how to build on strengths and improve on areas of weakness, as well as exploring ways in which they might collaborate more effectively together. During the workshop, participants discussed best practices and lessons learned in country groups and then discussed these across three themes: accountability, capacity and coordination. Of the best practices discussed over the two days, five were selected as having been most crucial to improving response time and effectiveness:

• Having existing capacity to respond;

• Making linkages at community level with local structures and community leaders;

• Having consistent leadership in the development of strategic plans;

• The existence of a longer-term planning and fundraising strategy; and

• The use of humanitarian standards such as Sphere. The top lessons learned from an interagency perspective included:

• The need for early social/economic analysis which would aid programming and programme monitoring, for joint rapid assessments;

• A central role for community consultation and participation; and


58

19. Stories

Introduction

A great deal has been written about stories in knowledge and learning strategies. Storytelling has numerous advantages over more traditional organisational communication techniques. First is that it enables articulation of emotional aspects as well as factual content, and thus allows expression of tacit knowledge that might otherwise be difficult to share. Secondly, in providing the broader context in which knowledge arises, storytelling can increase the potential for meaningful knowledge sharing. By grounding facts in a narrative structure, learning is more likely to take place, and being passed on. This guide aims to provide a set of pointers for using story telling in a workshop format, using the template developed by Sparknow Consulting (www.sparknow.net). Potential applications of narratives are:

• Team or community-building exercises;

• Breaking down barriers between multidisciplinary or multi-cultural teams;

• Workshop warm-ups;

• Trip debriefs;

• Personal project reviews;

• Entertainment and fun;

• Monitoring systems (see Most Significant Change, Tool 4).


This workshop format was developed by the innovative consultancy Sparknow, and has been used in a range of settings globally. The RAPID team has applied this in workshops in donor agency headquarters, in research study interviews, and with humanitarian aid workers returning from the field. The principle is that everyone can think of (positive/negative) changes of which they have been a part; this enables individuals, pairs and groups to learn about these in a structured fashion. Table 6: Story template for use in workshop process

• Title of story • Name of original teller • Name of listener/understander • Landscape: set the scene in time and space • Dwelling place: precise location where action occurred • Characters: cast list, descriptive attributes and roles in story • Challenge: problem or task that triggered the action • Action: sequence of events before, during and after your turning point • Turning point: the moment when the change happens • Resolution: ending, including moral, lesson learned or message • Key visual hooks: mnemonics to assist partner retelling the story

• Introduce the workshop and theme for storytelling. This could be focused on a specific theme (e.g. change in organisational management techniques), or on a range of themes. The key is to provide a context in which participants thinks about and selects the story they are going to share.

• Get participants to reflect on the change process, and details before, during and after.

• Ask participants to pair up and share their stories.

• Ask each participant to interview their partner, and write down the story, using the story template as a guide. This should enable more capture of detail.


59

• Ask the pairs to find another pair, and ask each participant in the new group of four to take turns telling their partner’s story to the larger group.

• Ask the group to identify any common points or contradictions across the stories.

• Ask each group to present back to the whole group in plenary.


This highlights key factors in using stories for change, whether social or institutional. The story:

• Needs to be simple and powerful;

• Should be in response to demand, and timed with specific opportunities;

• Should provide a solution to both immediate and broader problems;

• Should be targeted at people with the power to make decisions and change things;

• Should play to what is already in people’s minds.

Example: Knowledge management at the World Bank

After almost 20 years at the World Bank, Steve Denning used a 10-minute story to trigger change. In his own words (www.destinationkm.com/articles/default.asp?ArticleID=541):

… we were drowning in information, managing it very inefficiently, and if we cleaned it up we would save a lot of money. But it occurred to me that we’d still not be a very relevant organization. The World Bank had been a lending organization most of its life, and we were facing private-sector banks that were lending much more than we were. At that time, people were asking themselves if we had a future at all. So I started to ask myself a different question: Suppose we were to share our knowledge? We had over 50 years’ worth of know-how about what works in development and what doesn’t. Inside the organization, if you knew who knew what stuff, you could have lunch with them and find out, but if you didn’t know them you were in trouble. If you were outside, you didn’t have a prayer. But if we were to make it easy for anyone in the world to find out what we know, we could become relevant and useful …

In response to this situation, Denning told the following story:

In June 1995, a health worker in Kamana, Zambia, logged on to the Centers for Disease Control website and got the answer to a question on how to treat malaria. This story happened, not in June 2015, but in June 1995. This is not a rich country, it is Zambia, one of the least developed countries in the world. It is not even the capital of the country; it is six hundred kilometres away. But the most striking aspect of the picture is this: our organization isn’t in it. Our organization doesn’t have its know-how and expertise organized in such a way that someone like the health worker in Zambia can have access to it. But just imagine if it had! We could get ourselves organized so that professionals have access to the resources needed. Just in time and just enough.

Denning was named Programme Director and assigned the task of making the Bank a knowledge organisation. The programme caught the attention of the leadership of the Bank, and saw the Bank being re-branded ‘the Knowledge Bank’ in the 1996 inaugural speech of Bank President James Wolfensohn. This is an example of what Denning (2000) calls ‘springboard stories’.


• Examples of storytelling in the development sector and further afield, see: www.sparknow.net.

• For more on storytelling techniques and the different possible reasons for using storytelling in organisations, see Steve Denning’s website: www.stevedenning.com.

• For a look at how storytelling can be used for effective external communications, see Hovland, I. (2005) Successful Communication: A Toolkit for Researchers and Civil Society Organisations, ODI Working Paper 227, London: ODI.

Planning Tools

6. How to Write a Communications Strategy Drawing up a communications strategy is an art, not a science and there are lots of different ways of approaching the task. The advice provided below is only a guide. Whether your communications strategy is designed for a specific project or for the same period as your organisational strategy, it should establish the following:

• Objectives

• Audiences

• Messages

• Tools and activities

• Resources

• Timescales

• Evaluation and amendment Objectives Your objectives are the key to the success of your communications strategy. They should ensure that your communications strategy is organisationally driven rather than communications driven. Your communications activity is not an end in itself but should serve and hence be aligned with your organisational objectives. Ask yourself what you can do within communications to help your organisation achieve its core objectives. Aligning your communications and organisational objectives will also help to reinforce the importance and relevance of communications and thereby make a convincing case for the proper resourcing of communications activity within your organisation. Audiences You should identify those audiences with whom you need to communicate to achieve your organisational objectives. The best audiences to target in order to achieve an objective may not always be the most obvious ones, and targeting audiences such as the media may not always help achieve your objectives. Everyone would like a higher media and political profile, yet activities aiming towards this may ultimately be self-serving and only communications driven, with no wider impact. They can even have a negative effect if you dedicate resources towards this that would otherwise be put towards communicating with key stakeholders. Messages Strategic targeting and consistency are key to your organisation’s messages. Create a comprehensive case covering all the key messages, and emphasise the different elements of the case for different audiences. To maximise impact you should summarise the case in three key points which can be constantly repeated. Remember that communications is all about storytelling: use interesting narrative, human interest stories and arresting imagery.

17

Successful Communication

Tools and activities Identify the tools and activities that are most appropriate to communicating the key messages to the audiences. These will be suggested by your audiences, messages, or a combination of the two. For example, an annual report is a useful tool in corporate communications whereas an email newsletter lends itself well to internal communications. Ensure that you tailor your tools and activities to the level of time and human and financial resources available. Resources and timescales The key rules to observe are always to deliver what you promise and never over promise. Use your resources and timescales to set legitimate levels of expectations and outline the case for more dedicated resources. Evaluation and amendment Consider performing a communications audit to assess the effectiveness of your strategy with both your internal and external audiences. You should use open questions with appropriate prompts and benchmarks and, if possible, get someone independent to do the work. Consider and discuss the results carefully and use them to amend your strategy. Example audiences to consider are your staff, funders, key political targets and media. Questions you should consider asking are:

• What do you read/see/hear?

• What works/doesn’t work?

• What do you want to see more of?

• What information do you need that you are not currently supplied with?

• How often do you want us to communicate with you? While drawing up your strategy, you should involve your team, and on a smaller scale, the entire organisation. Feed the communications strategy into the organisational strategy to ensure maximum alignment and efficiency. Source

• The Media Trust, see: www.mediatrust.org/online_guides/comms_strat.html. Further resources

• The Economic and Social Research Council (ESRC) ‘How to write a communications strategy’, available at: www.esrcsocietytoday.ac.uk/ESRCInfoCentre/Support/Communications%5FToolkit/ communications%5Fstrategy/.

18

http://www.mediatrust.org/online_guides/comms_strat.html

http://www.esrcsocietytoday.ac.uk/ESRCInfoCentre/Support/Communications_Toolkit/�communications_strategy/

http://www.esrcsocietytoday.ac.uk/ESRCInfoCentre/Support/Communications_Toolkit/�communications_strategy/


The Overseas Development Institute is the UK’s leading independent think tank on international development and humanitarian issues. ODI Background Notes provide a summary or snapshot of an issue or of an area of ODI work in progress. This and other ODI Background Notes are available from www.odi.org.uk

By Nicola Jones and Cora Walsh

P olicy briefs are short documents that present the findings and recommendations of a research project to a non-specialist reader-ship. They are often recommended as a key

tool for communicating research findings to policy actors (Young and Quinn, 2007). However, there has been little systematic research in the development field about the communication needs of developing country policy-makers and how such research can be used to inform policy brief content and design. This back-ground note presents recent research by the Research and Policy in Development (RAPID) Group at ODI and the Science and Development Network (SciDev.Net) on the research communication environment involving researchers, policy-makers and development practi-tioners from the North and South in science, technol-ogy and innovation.

We begin with an overview of the theoretical litera-ture on bridging research and policy, with a focus on insights from scholars interested in the science–pol-icy interface. Drawing on an international survey and country case studies, we then highlight the barriers to, and opportunities for, strengthening communica-tion between researchers, knowledge brokers and policy-makers working in international development, and the key requisites of policy briefs to meet the challenges of this landscape.

Characterising the divide between the research and policy communitiesScholarship on the research–policy interface in recent years has done much to unpack the complexities of the uptake of research evidence into policy-making processes (Cash et al., 2003; Scott, 2006; Choi et al., 2007; Fairhead et al., 2006). There is now a growing

focus on thematic advocacy coalitions that cut across government agencies and research institutes (Buse et al., 2005) as well as innovative knowledge translation initiatives such as multi-stakeholder research partner-ships between researchers, NGOs and policy-makers (Jones and Villar, 2008) and the establishment of dedicated knowledge hubs within line ministries in some developing countries (Lavis, 2007). However, a number of key structural and professional tensions persist between researchers and policy-makers. These are presented below, with a particular emphasis on the natural science field.

Specialised research expertise vs democratised knowledgeEfforts to communicate research-based information for policy application underscore tensions between scien-tific knowledge as ‘privileged’ information and the per-ceived diluting effects that a democratised knowledge-base may introduce (Weingart, 1999). Some fear that the capacity of the current system of communication between researcher and policy communities is inad-equate to rule out excessive dilution of scientific knowl-edge (Clark and Juma, 2002). Moreover, the pluralisa-tion of knowledge in policy can, in fact, cause debate to stagnate rather than encourage it. Policy-makers, con-strained by time and overwhelmed by various sources of information, are likely to make a snap decision by selecting the ‘evidence’ most appropriate to their politi-cal leanings (Edwards, 1999). The clear warning is that, without efforts to improve these communication chan-nels, research may lose its ‘purity’ when used in the short timeframes of the political sphere.

Engagement vs objectivityA divide between ‘engaged’ and ‘objective’ researchers is highlighted in the literature concerning science com-munication in developed countries in particular, and to a lesser degree in studies on developing countries.

Policy briefs as a communication tool for development research

Background Note May 2008

2

Background Note

Figure 1: Factors influencing policy-making

Adapted from Davies (2005)

Two broad categories of researchers emerge: research-ers engaged in policy-making processes and those who separate themselves from policy. The divide often occurs between ‘strictly objective’ researchers, who believe that engaging in civic debate under-mines objectivity, and ‘citizen scientists’, who believe researchers can – and at times should – help decision-makers incorporate sound scientific knowledge into policy (Higgins et al., 2006). Debate between these camps is said to render many researchers unwilling to engage in civic discourse: some are convinced by the argument for strict objectivity, while others recog-nise that it is safer, professionally, to focus solely on research and risky to advocate on behalf of anything, even science. However, more nuanced arguments suggest that when researchers recoil too far from the policy implications of research, they leave a ‘vacuum’ that is filled by politically motivated parties who offer their own interpretations, and without credible opposi-tion, can mislead the public towards their own goals.

Researchers’ vs policy-makers’ incentive struc-tures and timescalesProblems caused by the divergent timescales and incentive structures of researchers and policy-makers lie at the heart of communication issues at the research–policy interface. On the one hand, the time-consuming nature of ‘pure’ research, not bound by time constraints, is difficult to integrate with the policy

demands of politicians who are often compelled to work under very tight deadlines to produce short-term, tangible policy results. On the other hand, policy-makers often struggle to stay apace of new scientific thinking, especially in terms of developing relevant policies and infrastructure to enable as well as regulate the implementation of scientific and technological advances (Clark and Juma, 2002).

Evidence vs contextual factors in policy decision-makingResearch findings have been responsible for many improvements in quality of life. Better use of research evidence in development policy-making can save lives through more effective policies that respond to scientific and technological advances, use resources more efficiently and better meet citizens’ needs (WHO, 2004). However, too often the linkages between research and policy-making are viewed as a linear process, in which research findings are critically analysed and the best option implemented into policy (Young and Court, 2004). In reality, the integration of evidence into policy decision-making is a complex process of multiple, frequently competing and / or intertwined sets of influences in which evidence plays just one of many roles (see Figure 1). In practice, research evidence is considered through the lens of policy-makers’ experience, expertise and judgement, contextual pragmatics, available resources and

Evidence

Resources

Habits, valuesand tradition Policy content

Judgement

Experience and expertise

Pragmatics and contingencies

Lobbyists and pressure groups

3

Background Note

the policy context, along with the habits, values and traditions of policy-makers, and the influence of lobbyists and pressure groups (Davies, 2005). Increasing the usage of evidence in policy-making therefore requires a communication approach that is informed by an understanding and engagement with these competing influences.

Research methodology

This background note is based upon the findings of a 2007 ODI/SciDev.Net international study on the research–policy interface in the field of science, technology and innovation. The study involved a systematic literature review, expert interviews, seven developing country case studies (China, Cambodia, India, Ghana, Zambia, Nicaragua and Bolivia) and an international survey with researchers, policy-makers and intermediary organisations. Research questions focused on how research information is accessed for development policy-making (particularly in developing countries), what types of communication of research evidence are most useful / effective for policy actors, and the ways in which an intermediary organisation can facilitate the communication process between researcher and policy-making communities.

This note draws primarily on the survey findings,1 as well as more in-depth qualitative work undertaken with an expert panel2 and key informant interviews in Brazil and India.3

Study findings Despite the emphasis in the literature on the polarisation between researcher and policy-maker communities, the 2007 ODI/SciDev.Net study found that greater opportunities for interaction, discussion and deliberation between researchers and policy-makers would significantly improve the uptake of research findings in policy decision-making. The survey findings underscored the large unmet need for greater communication of scientific and technological evidence for policy-makers. Some 50% of policy-makers and 65% of researchers felt that there is insufficient dissemination of research findings for policy uptake (59% of respondents overall, see Figure 2). Policy briefs were identified as a key tool for addressing this gap, with 79% of respondents from both developed and developing countries ranking policy briefs as valuable communications tools along with opinion articles written by experts, news items and discussion fora. Similarly, more in-depth interviews with sub-national developing country policy-makers confirmed that they not only read policy briefs, but often actively seek them out to inform their decision-making processes. As one Indian sub-national level policy-maker emphasised: ‘I often read policy briefs for both my official and non-official needs. I cannot think of going forward without consulting policy briefs. It expands my knowledge as I get an opportunity to understand what is happening around me’.

Figure 2: Obstacles to the uptake of scientific information in development policy-making

Other 6 Too much scientific information to be useful 15

Too little scientific information available 28

Jargon does not correspond with policy environment 30

Scientific data not perceived as credible evidence 31

Scientific research findings not relevant to policy 33

Economic and social data more relevant to policy-making 44

Lack of institutional channels for incorporation 44

Lack of incentives 56

Lack of dissemination of research findings 59

Limited openess by politicians 61

Scientific understanding by policy-makers is low 64

Percentage

4

Background Note

To be effective, our research findings emphasised the importance of a number of key ingredients. These are in line with the RAPID framework on bridging research and policy (Figure 3), which emphasises: 1) the importance of embedding an understanding of the political context within the design and communication of research, 2) the necessity of providing quality evidence and twinning this with the communication of key findings through a credible messenger, and 3) the value of fostering linkages and active engagement between researchers and policy-makers to ensure that research products are part of an ongoing dialogue. A summary is provided in Table 1.

Evidence Developing a persuasive argument Our key informants stressed the need for the purpose of a policy brief to be expressed clearly and early in the text. A statement of purpose should convey the essence of the brief, act as an enticement to read-ers and provide an overview of the contents for busy research users. Much like a newspaper article, this statement of purpose should both ‘hook’ the reader and provide a concise statement of what the policy brief will tell the reader.

As scientific evidence represents just one of many competing influences on policy-making decisions, policy briefs also need to persuade the reader of the importance of the evidence and recommendations. Policy brief reviewers in developing countries emphasised the high volume of information with which they are presented. Given this plethora of information and time constraints, a policy brief should persuade a reader that the evidence presented is important and that the recommended policy actions are necessary. To do this, effective policy briefs should develop a persuasive line of argument that maintains the scientific credibility of the information, while highlighting its relevance and urgency for policy issues. This entails distilling the complexity and nuances of research findings into clear and concise messages that the audience can easily digest and remember. The argument must also take into consideration the competing externalities that will influence decision-making, such as donor priorities, historical-political sensitivities, cultural values and timing of elections among others.

Transparency of the source of the evidence behind policy recommendations is essential to promote broader access to new scientific knowledge. Are the recommendations derived from a single study, a

Table 1: Key ingredients of effective policy briefs

Evidence

Persuasive argument • Clear purpose• Cohesive argument• Quality of evidence• Transparency of evidence underpinning policy recommendations (e.g. a single study, a synthesis of available evidence, etc.)

Authority • Messenger (individual or organisation) has credibility in eyes of policy-maker

Policy context

Audience context specificity • Addresses specific context > national and sub-national• Addresses needs of target audience > social vs economic policy

Actionable recommendations • Information linked to specific policy processes• Clear and feasible recommendations on policy steps to be taken

Engagement

Presentation of evidence-informed opinions

• Presentation of author’s own views about policy implications of research findings• But clear identification of argument components that are opinion-based

Clear language/writing style • Easily understood by educated, non-specialist

Appearance/design • Visually engaging• Presentation of information through charts, graphs, photos

Figure 3: The RAPID Framework: Context, evidence and links

External Influences International factors, economic and cultural influences, etc.

The Political Context – political structures/

processes, institutional pressures, prevailing

concepts, policy streams and windows, etc.

Links between policy makers and other stakeholders, relationships, voice

trust, networks, the media & other intermediaries,

etc.

The Evidence, credibility, methods,

relevance, use, how the message is packaged and communicated,

etc.

5

Background Note

review and synthesis of existing information, or the culmination of a programme of work? This transpar-ency can be aided by providing a short annotated list of the most important sources and publication on the topic for further reading.

Credibility of the messengerEnd-users of policy briefs emphasised that they do pay attention to who is producing the policy brief and that this influences their acceptance of the evidence and argument presented. Legitimacy stems not only from the quality of the evidence base, but also from the author of the information and / or the organisation publishing the brief.

Survey respondents identified professional scientific and international organisations as the most legitimate potential mediators between researcher and policy-maker communities. However, mediating at the science–policy interface is not necessarily part of the mandate of such organisations. This suggests that there are many undefined roles to be filled in this area by other possible knowledge brokers such as: policy advisors, donors and web-based organisations. When acting as a knowledge broker and producing policy briefs, organisations should consider partnering with authoritative research institutes so as to augment their credibility.

Context

Tailoring findings to political contextPresenting results so that they are applicable to the specific national and sub-national contexts in which policy-makers operate emerged as an important challenge. Policy-makers do not represent a homogenous group of actors, but rather have different needs, priorities and uses for information based on

their position by sector, level of government, and role in policy-making. A policy brief should, therefore, be written to address the needs of the target audience as well as in accordance with the particular point in the policy cycle that one aims to influence, whether it be agenda setting, policy formulation, implementation or evaluation.

Patterns of evidence use differ by audience segment, including country, sector, role in policy-making (e.g. legislator, minister, policy engaged NGO), level of government, etc. For example, non-science related ministries report employing scientific information primarily in the stages of policy evaluation (64%) and implementation (59%). By contrast, science-related ministries use scientific information primarily for policy conceptualisation (88%), and formulation (85%), suggesting that non-science policy-makers use scientific information to legitimate and evaluate policy decisions, whereas science-related ministries rely more heavily on scientific information to formulate policy. There is also strong demand for more regionally and locally specific policy briefs: over 50% of developing country based policy-makers prefer regionally specific information over globally applicable information. Having this information translated into local languages is also important if readership and engagement with new research is to be enhanced.

Tailoring findings to audience interestsThe purpose of a policy brief should be linked to the target audience. As shown above, the ODI/SciDev.Net survey found that the informational needs of science-ministry officials differ from those of non-science ministries. A policy brief should therefore be written to address the specific purpose for which its target audience uses information, whether it be to formulate or validate policies. As a policy-maker from Kerala State, India, explained:

Box 1: Views of developing country policy-makers

‘Policy briefs provide valuable information in an understandable format…when I read policy briefs I look for the quality of the information, adequate tables and figures, and connection of the evidence to policy processes.’ (Sub-national level policy maker, Brazil)

‘Briefs should be inspiring. They should be practical, realistic and relevant to the local contexts.’ (President of local-level government body, Kerala State, India)

‘When I read policy briefs I look for concise information that takes into account the policy process, and provides information relevant to the problems at hand.’ (Sub-national level policy-maker, Brazil)

Box 2: Country Case Study Examples

In India and Cambodia, the uptake of scientific informa-tion into policy is also closely linked to its resonance with broader national development priorities. For ex-ample, in India the framing of biotechnology research findings in pro-poor discourse (improved crop yields as a means to reduce rural poverty) has contributed to widespread policy implementation. In Cambodia as in other post-conflict societies, research messages pre-sented as part of broader socio-economic rehabilitation efforts are more likely to receive policy support. In both cases, demonstrating the complementarity of research evidence with social and economic data is often highly effective.

6

Background Note

‘Primarily, I look for applicability within my working framework. Usually, there are a hundred policy briefs on a single subject but the majority are irrelevant to local contexts and situations.’

This suggests that there may then be a need for separate tailored versions of policy briefs for different policy actors, not only according to the level of the political arena (international, national, sub-national and local) but also depending on the policy sector in which they work, and whether or not they are civil servants or elected officials. In this vein, persuading the reader to take a particular course of action based on research evidence can be enhanced by highlighting the benefits that are likely to accrue by following a particular course of action. Country case studies in the ODI/SciDev.Net study showed, for instance, that linking research evidence to socio-economic benefits in particular can be especially persuasive, due to overarching attention to poverty reduction and economic growth.

Presenting actionable recommendationsGiven the time pressures on policy-makers to deliver policies with rapid and visible impacts, recommendations must be actionable and clearly connected to specific decision-making junctures in the policy-making process. Evidence-based recommendations must provide the necessary information to differentiate between various policy options. Moreover, policy brief authors also need to take into consideration the intersection between new knowledge and complex power relations that underpin policy processes. The presentation of research evidence that challenges prevailing understandings has to tread a fine line between opening up new policy horizons while avoiding being too confrontational and alienating readers. For example, in Ghana, key informant researchers explained that they are wary of presenting evidence that is framed within a political viewpoint at odds with the politics of those in power, as their work may be disregarded. This reinforces the importance of policy briefs framing research evidence in a way that is sensitive to the political context if the messages are to be accepted and potentially acted upon.

EngagementNot shying away from opinion and value judge-ments One of the most striking findings of the study was the fact that, while policy-makers value research evidence, they do not want to be simply presented with research findings. Instead, 80% said that they value researchers’ opinions about the policy implications of

their findings. Interestingly, while those in both the North and South preferred researchers to express their opinions, the demand for opinion, value judgments and advice on policy actions was particularly high in the South, both at the national and sub-national levels.

Presenting messages in clear languageThere was a strong consensus among study participants that briefs need to be written in clear, jargon-free language, and pitched towards educated non-specialists in the topic. This is because many policy-makers are generalists and do not come from research or even strong educational backgrounds: 64% of ODI/SciDev.Net survey respondents were of the view that low levels of scientific understanding by policy-makers constituted a significant obstacle to the uptake of scientific information (Figure 2 above). Moreover, a significant number of policy-makers emphasised that much research evidence is unnecessarily verbose and dense.

Engaging audiences visually To make a significant impact on an audience, policy briefs must not only be conceptually engaging, but also visually appealing. Policy-makers have limited time to read: the ODI/SciDev.Net survey findings indicated that most policy-makers spend just 30 to 60 minutes reading information on a particular issue. Policy briefs must, therefore, draw readers’ attention and present information in a way that is easily remembered. Over 80% of respondents in the same survey found graphs or explanatory diagrams helpful, while a systematic review of policy briefs found that those that were visually stimulating were consistently rated more highly.

Conclusions

Policy briefs, if carefully designed, can be a powerful tool for communicating research findings to development policy audiences. However, the effectiveness of any tool depends upon appropriate usage. Producers of policy briefs aiming to increase uptake of scientific and technological research in development policy need to focus on, and actively address, the communication tensions at the research-policy boundary. Policy-makers operate in a complex environment of competing concerns. The provision of research information alone is not, therefore, sufficient to influence the policy agenda. The value of a policy brief needs to be viewed not only in terms of presenting quality evidence, but also in translating new knowledge into context-relevant messages and guidance for policy-makers. Most importantly,

7

Background Note

This Background Note was written by Cora Walsh and Nicola Jones and is based on work conducted in the RAPID programme at ODI, commissioned by SciDev.Net and funded by DFID. For more infor-mation contact Nicola Jones ([email protected]) or visit www.odi.org.uk/rapid. More information on science and policy can also be found at www.scidev.net

however, even with a well-crafted policy brief in hand, the research communication process has not ended but is only beginning.

To foster uptake and implementation, face-to-face and / or electronic discussion and deliberation with policy-makers about the policy brief evidence and policy guidance is critical. What is needed is active mediation and translation among knowledge producers, knowledge brokers and end users, as well as an integrated communications approach that takes into consideration individual, organisational and systemic levels. It is critical to foster close collaboration between researchers and policy-makers from the outset, rather than disseminating research results at the end of a project, to reach consensus on the key questions to be addressed and to promote understanding of research methodologies as well as ownership of findings.

Constructing an appropriate platform from which to communicate is also key, especially if research findings challenge current policy approaches. Informed by insights from literature on advocacy and user engagement, there is a growing realisation of the efficacy of promoting broad engagement and participation on an issue, and using public

engagement (e.g. global advocacy campaigns, community radio) as a platform from which to approach policy-makers and advocate for more accountable decision-making (Hovland, 2004). This approach was also strongly endorsed by over 90% of ODI/SciDev.Net survey respondents who called for more efforts to build the public’s capacity to engage in research-policy debates. Improved research communication is therefore critical, not only between researcher and policy-maker communities, but also among the broader public. Lastly, efforts to strengthen researchers’ communication and knowledge brokering skills need to be complemented by efforts to strengthen the institutional capacity of policy agencies to take up research. This includes enhancing individual capacities and skills, as well as developing institutional channels, procedures and incentive structures to promote evidence-informed policy processes.

Background Note

Overseas Development Institute, 111 Westminster Bridge Road, London SE1 7JD, Tel: +44 (0)20 7922 0300, Fax: +44 (0)20 7 922 0399, Email: [email protected]. Readers are encouraged to quote or reproduce material from ODI Background Notes for their own publications, but as copyright holder, ODI requests due acknowledgement and a copy of the publication. This and other ODI Background Notes are available from www.odi.org.uk. © Overseas Development Institute 2008. ISSN 1756-7610.

Endnotes and further resources

Endnotes1 The international online survey had a total of 617 responses,

sampling policy-makers (18.3%), intermediary communicators (34.7%) and researchers (46.7%). Most respondents (63.9%) were from developing countries. The survey results were compiled and analysed using largely descriptive statistics, disaggregating responses by sub-groups of respondents (policy-makers, intermediaries, and researchers), as well as by region. Results were then compared across these categories to discover significant patterns and differences. Large differences between groups and variables were then tested for significance using the Chi-square test.

2 An initial policy brief review panel was convened involving participants from the North and South, academia, a think tank, the NGO sector and a communication specialist. Panel participants reviewed 16 sample policy briefs across four thematic areas (Health, Technology, Environment, and Agriculture) according to set of criteria decided upon by the panel: clarity of purpose; persuasive argument with actionable recommendations; clear source of evidence; clear language /writing style; appearance / design; and authority.

3 Two case studies were coordinated by ODI and conducted by CGEE in Brazil, and PRAXIS in India to further investigate the use of policy briefs by developing country policy-makers at national and sub-national levels. Policy-makers were asked to review three example policy briefs according to the criteria employed by the international panel, and to discuss the relative importance of each criterion in affecting the usage /effectiveness of a policy brief.

Suggested further resoucesTranslating evidence for development policy:Cash, David W., William C. Clark, Frank Alcock, Nancy M.

Dickson, Noelle Eckley, David H. Guston, Jill Jager and Ronald B. Mitchell (2003) Knowledge systems for sustainable development. PNAS. (https://rapid.odinet.org.uk/rip/rap0056/rap0056shared/Process/Inception%20study/literature/Knowledge%20systems%20for%20sustainable%20development.pdf).

Clark, W. and Juma, C. (2002) Mobilizing Science and Technology for Sustainable Development. Forum on Science and Technology for Sustainability. (http://www.ksg.harvard.edu/sustsci/ists/docs/ists_cfia_rpt_final.pdf).

Court, J., Hovland, I., and Young, J. (2005) Bridging Research and Policy in Development: Evidence and the Change Process. Warwickshire, UK: ITDG.

Mediating between scientists and policymakers:Choi, B. C. K., Pang, T., Lin, V., Puska, P., Sherman, G., Goddard,

M., Ackland, M.J., Sainsbury, P., Stachenko, S., and Morrison, H. (2005) Can scientists and policy makers work together ? Journal of Epidemiology and community health 59: 632-637.

Higgins, P. A. T., Chan, K. M. A. and Porder, S. (2006) Bridge over a philosophical divide. Evidence and Policy 2(2): 249-255.

Communication toolkits:Hovland, I. (2005) Successful Communication: A Toolkit for

Researchers and Civil Society Organisations. London: ODI. (http://www.odi.org.uk/publications/rapid/tools2.pdf).

Influencing policy:Majone, Giandomenico (1989) Evidence, argument and

persuasion in the policy process. New Haven: Yale University Press.

Shaxson, L. (2007) Practical tools for evidence based policy making: developing lines of argument. Presentation at: Impact & Insight Workshop. UK: Kings College London. 25 Oct. 2007. (http://www.slideshare.net/ODI_Webmaster/lines-of-argument-presentation-at-insights-to-impact-meeting/).

WHO (2004) World Report on Knowledge for Better Health: Strengthening Health Systems. WHO: Geneva. (http://www.who.int/rpc/meetings/world_report_on_knowledge_for_better_health.pdf).

ReferencesBuse, K., Mays, N. and Walt, G. (2005) Making Health Policy.

Maidenhead, England: Open University Press.Choi, B.C.,K. (2007) Can scientists and policy makers work

together? Journal of Epidemiology and community health, 2005 59 (8).

Davies, P. (2005) Presentation: Impact to Insight Series. London: ODI.

Edwards A. (1999) Scientific expertise and policy-making: the intermediary role of the public sphere. Science and Public Policy, 26(3): 163-170. (https://ep.eur.nl/bitstream/1765/887/1/BSK077.pdf).

Higgins, P. A. T., Chan, K. M. A. and Porder, S. (2006) Bridge over a philosophical divide. Evidence and Policy 2(2).

Jones, N. with Eliana Villar (2008) ‘Situating Children in International Development Policy’. In Journal of Evidence and Policy. Vol 4., No. 1. pp 53-73.

Jones, N., Walsh, C. and Young, J. (2007) Policy Briefs for Communicating Science, Technological and Innovation Findings: What Constitutes Best Practice? Unpublished mimeo. London: ODI.

Jones, H. and Jones, N. (2007) Meeting Science/Technology Information Needs in the Policy Community of Developing Countries: Country Studies. Unpublished mimeo London: ODI.

Penn State University (2002) Preparing a policy issue brief. (http://www.courses.psu.edu/hpa/hpa301_fre1/IBInstructions_fa02.PDF).

Princeton University (2007) Thoughts on writing a policy paper. Writing Centre Web Resource Guide. (http://stokeslib.princeton.edu/writingelements.htm).

Walsh, C. and Jones, N. (2007) ODI/SciDev.Net International Survey on the Science-Development Policy Interface. Unpublished mimeo London: ODI.

Young, J. and Court, J. (2004) Bridging Research and Policy in International Development: An Analytical and Practical Framework. RAPID Briefing Paper 1, London: ODI.

Young, E. and Quinn, L. (2008) (http://www.vancouver.wsu.edu/fac/tissot/cl/esrp444/Writing%20Policy%20Briefs.pdf). Accessed on 3 February. 2008.

http://https://rapid.odinet.org.uk/rip/rap0056/rap0056shared/Process/Inception%20study/literature/Knowledge%20systems%20for%20sustainable%20development.pdf




http://http://www.ksg.harvard.edu/sustsci/ists/docs/ists_cfia_rpt_final.pdf

http://http://www.ksg.harvard.edu/sustsci/ists/docs/ists_cfia_rpt_final.pdf

http://http://www.odi.org.uk/publications/rapid/tools2.pdf

http://http://www.slideshare.net/ODI_Webmaster/lines-of-argument-presentation-at-insights-to-impact-meeting/

http://http://www.slideshare.net/ODI_Webmaster/lines-of-argument-presentation-at-insights-to-impact-meeting/

http://http://www.who.int/rpc/meetings/world_report_on_knowledge_for_better_health.pdf



http://https://ep.eur.nl/bitstream/1765/887/1/BSK077.pdf

http://https://ep.eur.nl/bitstream/1765/887/1/BSK077.pdf

http://http://www.courses.psu.edu/hpa/hpa301_fre1/IBInstructions_fa02.PDF

http://http://www.courses.psu.edu/hpa/hpa301_fre1/IBInstructions_fa02.PDF

http://http://stokeslib.princeton.edu/writingelements.htm

http://http://stokeslib.princeton.edu/writingelements.htm

http://http://www.vancouver.wsu.edu/fac/tissot/cl/esrp444/Writing%20Policy%20Briefs.pdf

http://http://www.vancouver.wsu.edu/fac/tissot/cl/esrp444/Writing%20Policy%20Briefs.pdf

Tips and Hints: Writing an ODI Briefing Paper

ODI Briefing Papers provide a short and snappy summary of the key findings and conclusions of ODI research. They’re meant to be accessible and relevant to a wide audience, including academics, policy-makers, practitioners, journalists and students. They should be easily readable by someone in a hurry – in ten minutes, say – should make sense to an intelligent non-specialist and should make a worthwhile contribution to the subject. They’re also quite tricky to write, hence this short, four-step guide.

Choosing your subject Not every research subject merits or suits a Briefing Paper: some research topics may be too complex to be tackled in this way, or too specialised to appeal to the range of audience we’re trying to reach. Your ideal subject will be:

• tightly defined (you won’t solve world poverty in 3,000 words, for example, but you might be able to say something useful about the links between HIV/AIDS and conflict in Africa);

• of relatively broad interest; • topical and relevant; and • with concrete implications for aid policy and practice.

What to put in, and what to leave out The short length of ODI Briefing Papers means that, at the outset, you need to decide what to put in, and what to leave out. Be ruthless. Ask yourself what’s really essential, and what’s not:

• What are the key components of my research? • What’s new, and what’s not? • What examples or case studies must I include to support my points? • What evidence must I present to make my conclusions and recommendations authoritative and

credible? • How much background information or context is necessary to make what I’m saying intelligible to a

non-specialist?

Organising your material Every Briefing Paper has the same basic structure:

• A title. • A byline. • A concise statement of topic and argument, which should aim both to inform your reader of what

the Briefing Paper is about, and entice them into reading it. • A body text. • A set of conclusions and/or policy recommendations.

In addition, your paper may or may not include a set of key points, boxed text, figures, images, endnotes and suggestions for further reading. Whether it does so depends on your subject, the availability of material and space. Within this, you need to plan how to organise your material. The aim is to achieve a simple, logical and progressive line of argument, leading up to a final section of conclusions or recommendations. A typical argument might begin with a statement of the problem, an explanation of why the problem exists and recommendations for addressing it, based on the findings of your research. It might help to think in terms of subheads and paragraphs:

• Use subheads to signpost key points in your text: to highlight a key research question or finding, for example, or to help people follow a change or transition in your argument. Keep subheads short, and use them sparingly.

• Use paragraphs to build your argument and give it forward movement. The ideal paragraph starts with a topic statement, like this: ‘The foregoing analysis shows that a new approach to poverty reduction is required’. It then goes on to develop the idea, in this case perhaps explaining what this new approach might be. The next paragraph then picks up the baton; in this example, the topic sentence might look like this: ‘These changes require a fundamental reform of the aid architecture’. And so on. Keep your paragraphs short, but be wary of fragmenting your text with an abundance of them.

Writing your Briefing Paper: tone and style Your tone is important. You should aim to be authoritative, credible, judicious and measured. Try to be decisive and conclusive as far as your facts allow: hedge your bets only if you feel you must, and avoid lengthy caveats to your conclusions if you can. Avoid opinion or advocacy. You’re aiming for a style that’s fresh, direct and engaging: you don’t have many words to play with, so all of them have to count. This doesn’t mean being casual, imprecise or colloquial – avoid pronouns and verbal contractions, for example, and try not to start sentences with ands or buts. Some tips:

• Remember that some of your readers will not have English as a first language, so try to write clearly, directly and simply. Avoid puns, aphorisms, acronyms and idioms like the plague.

• Keep your words and your sentences short, and avoid complex sentence structures with multiple subordinate clauses. Ideally, each sentence should convey a single idea, not many.

• Use active verbs, not passive ones. • Try to make positive statements, not negative ones: we want to know what your research tells us,

not what it doesn’t. • No jargon unless you really can’t avoid it. If you must use it, explain what it means. • Avoid references if you can. If you have to cite other works, do so in endnotes, not Harvard-like

systems.

CREATE story of national‐level policy change

Over the past several years, CREATE has worked with local Ghanaian researchers, universities and policy‐makers to increase the use of local research in the policy dialogue. This was part of a more specific attempt to shift policy discourse of the ministry from a focus on gross enrolment figures to thinking about the underlying factors that affect them. In previous instances, researcher participation in the policy process had been very weak, so CREATE has really provided the opportunity for closer dialogue and enriched the policy debate. This increased participation revolved around three senior stakeholders: the Chief Director from the Ministry of Education, Science and Sports (MOESS), the Minister of Education and the lead Ghanaian partner from CREATE, Professor Djangmah, the former director general of the Ghana Education Service.

Previously, the debate on educational access in Ghana was led by MOESS through its annual reviews of education. Usually the ministry would invite international consultants to look at the data and information provided by the Education Management System and to help analyse the data and shape the agenda. Historically however, this process hadn’t sufficiently reflected local context – analysis was strongly influenced by an international perspective and Ghanaian researchers played a limited role.

CREATE sought to tackle this disconnect by ensuring stronger linkages with in‐country research on education. In particular, CREATE’s research has emphasised the patterns of participation over the education cycle to move the agenda beyond discussions of gross enrolment figures and the influence these factors have on accessing education.

In order to influence this policy dialogue, the first action taken was the establishment of a national reference group consisting of the three RPCs in Ghana that look at quality, access and outcome, with the Chief Director for MOESS as the chair. Directors from the ministry also participated in these meetings. This group allowed for dialogue between researchers and policy‐makers – while researchers were making suggestions, policy‐makers also gave input into research findings and directions. All of the reports, for example, were sent to key stakeholders so they could review the evidence and make sure that the messages reflected the problems from the ministry’s perspective and the policy direction. Outside of this working group, CREATE has also given presentations to Ghana’s parliament on educational issues highlighted by their research.

The serious engagement with policy‐makers has led to two significant changes both in terms of policy and in terms of operation. First, the policy dialogue now recognises the importance of age as a factor affecting enrolment. Also, field work is beginning to show that, in some of the poor areas in Ghana, there is an interest in private schooling. It is counterintuitive that rural poor could afford this private education, so convincing the ministry that they need to look at this aspect has been another way CREATE has input into the policy‐making process.

Secondly, the ministry has given strong support for increasing its own capacity to understand and incorporate research. The Chief Director is currently a student at the University of Sussex (where the RPC is based), and he has helped five of his directors to enrol on a professional doctorate programme there. Beyond the $250,000 the ministry has committed to its internal capacity development, it is also supporting the establishment of a research centre on basic education at the University of Education at Winneba. The idea is for this centre to engage in research that reflects issues on basic education policy and practice in Ghana. Two researchers from the University of Winneba are currently studying for their doctorates at Sussex under the CREATE programme and will be returning to strengthen this new centre in Winneba.

In terms of influencing policy, it was recognised that to get research into policy, maintaining dialogue is essential, and it has become a key strategy. CREATE has established a series of two‐page policy briefs to help open the discussion, and having a national reference group is also an important way to keep the dialogue going both ways.

It is clear that none of this would have been possible without a receptive ministry, and in particular the Chief Director. This has been highly effective in giving the project momentum, but there are risks with relying too heavily on a single stakeholder. If a significant part of the funding and progress hinges on a single director, what happens if he changes jobs? The capacity development of other staff at the ministry is central to sustaining locally informed and developed research, but will this be enough? The Chief Director recognises this challenge and hence his commitment to send five of his senior officials at the Ministry to train at Sussex. He has already moved some of them into key leadership positions and handed over some of his responsibilities to ensure there is smooth transition into a new leadership team when he eventually leaves the ministry. According to him, giving these officials the opportunity to upgrade and improve their understanding of research and how it is used in policy will ensure that the Ministry continues to make research evidence a key part of the policy making process to improve educational quality and access.

During CREATE’s last visit to Ghana, they realised that spending 10% on communications is actually very little, especially considering the amount of funding the ministry has dedicated to the project. As more research happens, more communication support is needed, especially as momentum is gathered. They’ve done policy briefs, met in a national reference group, had several special features in a national newspaper and participated in media interviews.

Further information on this project can be found on the CREATE website, where programme policy briefs have been compiled. See specifically the Access, age and grade policy brief and the Ghanaian overview.

http://www.create-rpc.org/

http://www.create-rpc.org/publications/Policy_briefs.shtml#CAR

http://www.create-rpc.org/pdf_documents/Policy_Brief_2.pdf

http://www.create-rpc.org/pdf_documents/Ghana_CAR_policy_brief.pdf

http://www.create-rpc.org/pdf_documents/Ghana_CAR_policy_brief.pdf

RiPPLE: Making research more demand-driven Focusing on issues of water and sanitation in Ethiopia, RiPPLE has sought to make research more demand-driven through establishing collaborative Learning and Practice Alliances (LPAs). These LPAs bring together diverse stakeholders, including donors, service deliverers, practitioners, researchers and government bodies to discuss, debate and learn about research and practice in the water sector. This is one strand of a three-pronged communication strategy that involves internal learning and exchange, broader dialogue and dissemination of research findings and creating an enabling environment for the uptake of research into policy.

In Ethiopia, the lines between researchers, policy-makers and beneficiaries is traditionally distinct. This meant that the agenda of various stakeholders at best lacked coordination and at worst were in direct competition. Recognising these challenges, the International Water and Sanitation Centre (IRC), one of the main consortium partners, were interested in blurring these lines through LPAs, and pushed to make them part of the original project plan.

If the overarching goal was to bring diverse stakeholders together to work collaboratively, the main challenge was implementing this diverse engagement.

RiPPLE decided the best approach would be to work its way up from the woreda (district) and sub-national regional levels. More recently they have turned to facilitating learning across the LPAs, horizontally, vertically and ‘diagonally’, at the national level. They have also established a virtual LPA for the greater Nile River Basin region. There are currently three woreda and six regional LPAs, whose activities focus around shared experience and understanding of what the problems are in water and sanitation at each of these levels. By discussing and refining issues, they can then decide together what the research focus should be. In terms of implementation and sustainability, each LPA has a coordinator who is in charge of sustaining interest and managing LPA activities. In order to this, the focus in 2008 has been on longer-term action research projects (LARs).

Although establishing these LPAs hasn’t necessarily been easy, there is a growing recognition that they're doing something worthwhile. They have been quite effective as a device for working across the research–practice interface and for helping foster understanding among disparate stakeholders. It has proved particularly successful when there is a limited research budget, as it's a way of narrowing down and focusing through institutions and teamwork. They have also helped to improve the coordination, discussion and debate of service delivery issues. At the national level, the Ministry of Water leads the national LPA, FlOWS. This buy-in from the government has helped raise the profile and the effectiveness of the overall LPA initiatives. Although there have been some initial successes, since the project is only in the second of its five-year duration, it is difficult to gauge the ultimate impact of these initiatives, especially since they take a significant amount of time to establish.

This buy-in didn’t evolve organically: it took a lot of planning and hard work to generate interest, identify partners and support participants who are not necessarily trained researchers. For the latter, it’s important not to have mentors 'breathing down the neck' of these researchers, but rather supporting and guiding them, particularly with data analysis and interpretation. But it’s often a difficult balance to strike. It’s also often difficult to navigate around the diverse agenda and special interests that the diverse participants bring to the table.

The 10% for communications rule from DFID has helped the LPAs think about how they are using communications. Good communications between and among LPAs has been essential in generating shared understanding, ownership and sustainability of outcomes. As the LPAs have developed, this means that communications is becoming integral to research, not just in terms of uptake and dissemination, but as a support to and part of the action of research. Thinking about communications from the beginning of a project has also been a helpful motivating factor: if LPA participants understand what’s going to happen to the results, it's a lot easier to get them interested in researching. At the same time, pinning down what constitutes ‘communication’, and therefore as falling under the 10% rule, has been difficult. Including staff time to support communication, for example, quickly eats up this budget, so RiPPLE has committed the 10% to producing outputs and finding additional money for communication staff.

There is a question of how sustainable these LPAs are beyond the lifespan of RiPPLE, especially since it is clear that each LPA needs a full-time coordinator. But RiPPLE has already started exploring alternative funding opportunities by shifting focus to a local Ethiopian NGO – Hararge Catholic Secretariat (HCS), one of the stronger local NGOs in agricultural marketing and extension. The hope is that this partnership will help sustain the LPAs, as they want to help scale up the initiative across the rest of Ethiopia through their umbrella organisation.

Additional information about how these LPAs are structured and what they are working on is available on the RiPPLE website at http://www.rippleethiopia.org/page/learning-and-practice-alliances.

http://www.rippleethiopia.org/page/learning-and-practice-alliances

Successful Communication

2. Social Network Analysis A network is a simple concept. It consists of two things: nodes and links between those nodes. In social network analysis the nodes of concern are people, groups and organisations. In other areas of network analysis the nodes of concern may be pages in the World Wide Web, difference species in an ecosystem or different compounds in a cell. In social network analysis links may be social contacts, exchanges of information, political influence, money, joint membership in an organisation, joint participation in specific events or many other aspects of human relationships. The defining feature of social network analysis is the focus on the structure of relationships.

Figure 3: Social network analysis

Cooperative

Other SMEs

Other lenders/insurers

Central government

Parastatals

Businesses outside region

Local government

Suppliers/wholesalers Large/formal enterprises

National business associations

DonorKavango chamber

Retail SMEs

NGOs

Full time MEs

LCS staff

Training programme

Other business associations

SME fora

Part time MEs

Source: Biggs and Matsaert (1998) in Rick Davies (2003) It has been argued for some time that organisations are embedded in networks of larger social processes, which they influence, and which also influence them (Granovetter, 1985; 1992). Recognising this can help us bridge links between different levels of analyses, relating to different types of organisational entities within development aid: projects, country programmes, and government policies. Within the aid agencies themselves the structuring of relationships between staff is another set of relationship choices with direct consequences for how local projects and national policies relate to each other or not. Structure can link strategies at different levels, or not (Davies, 2003). Further resources

• Davies, Rick (2003) Network Perspectives in the Evaluation of Development Interventions: More Than a Metaphor, see: www.mande.co.uk/docs/nape.pdf.

• The International Network of SNA, with a comprehensive list of resources, see: www.ire.org/sna.

10

http://www.mande.co.uk/docs/nape.pdf

Planning Tools

• Ramalingam, Ben (2005) ‘Tools for Knowledge and Learning’, RAPID Toolkit, London: ODI, see: www.odi.org.uk/Rapid/Publications/Documents/KM_toolkit_web.pdf. (Especially for further resources on networks and on the different connecting roles that people can play)

• Kincaid, Larry (2000) ‘Social networks, ideation, and contraceptive behaviour in Bangladesh: a longitudinal analysis’, Social Science & Medicine 50(2): 215-31. (On the use of a social network approach to family planning communication in Bangladesh)

• Perkin, Emily and Julius Court (2005) ‘Networks and Policy Processes in International Development: A Literature Review’ ODI Working Paper 252, see: www.odi.org.uk/Rapid/Publications/Documents/ WP252.pdf).

• Castells, Manuel (2004) ‘Why networks matter’, published as an afterword to the recent Demos collection of essays: McCarthy et al. (2004) Network Logic: Who governs in an interconnected world?

• Boonyabancha, Somsook (1999) Citizen Networks to Address Urban Poverty Experiences of Urban Community Development Office, Thailand UCDO, Asian Coalition for Housing Rights.

11

http://www.odi.org.uk/Rapid/Publications/Documents/KM_toolkit_web.pdf

http://www.odi.org.uk/Rapid/Publications/Documents/�WP252.pdf

http://www.odi.org.uk/Rapid/Publications/Documents/�WP252.pdf

An Approach to Understanding the Role of

Research in Policy Change Processes

Project Objective To identify and institutionalise innovative R&D approaches that lead to pro-poor policy outcomes.

Activities 1. Methodology development to evaluate the role of research and other factors in policy

processes.

2. Case studies to identify innovative R&D approaches that lead to pro-poor policy outcomes.

3. Use lessons from case studies to develop framework to guide design of new R&D projects.

4. Communicate this framework to international and national partner organisations to facilitate mainstreaming and institutionalisation of the identified approaches.

Case studies 1. Kenya: Dairy marketing policy: Smallholder Dairy Project and its partners.

2. Uganda: Development of New City Ordinances on Urban Agriculture in Kampala.

3. Ethiopia: Changes in policies/institutions on communal natural resource in Tigray.

The RAPID Outcome Assessment (ROA) This approach was developed by ODI’s Research and Policy in Development (RAPID) programme, and has been adapted for use in this project. The ROA approach combines elements from three approaches described in Box 1. In doing this, it utilises the best elements and reduces the limitations of each approach, as well as enabling triangulation of findings.

The ROA • is a visual tool combining the Outcome Mapping concept within a case study and

episode study approach; • uses a systematic approach to collecting information about changes in behaviour of

key project partners that contributed to the policy change; • can be used to assess the contribution of a project (programme, strategy, etc.) to

observed changes in behaviour – and ultimately to a policy change . N.B. the ROA is not designed as a way of capturing economic impact of research.

The ROA: What is actually done? 1. Background assessment

• Gathering information on targeted policy change, actors and events; • Base on literature review and key actor interviews, guided by ‘Context-Evidence-

Links’ (CEL) framework; • Includes ‘Case study’ or project-based information (‘supply-focus’?) and ‘Episode

study’ or policy-change focused information (‘demand-focus’?), based on range of actors interviewed and literature sources.

2. Workshops with key policy actors To map behavioural changes in key actors related to the policy change and build a map of influences. The stages are:

• Defining policy environment at start and end of project/period; • Identifying key actors – including ‘boundary partners’ of project if appropriate; • Characterising actors’ behaviour (i) now and (ii) at given start point in the policy

process; • Mapping key behaviour changes along the timeline; • Mapping (i) key project activities/changes, and (ii) external influences along the

same timeline; • Determining influence on actor behaviour change caused by identified

events/changes, including project activities, external influences and other actors’ behaviour change.

3. Triangulate and refine conclusions

• Key informant interviews and further literature review; • Analyse findings using the ‘Context-Evidence-Links’ (CEL) framework; • Report writing.

Box 1: Approaches that the ROA draws on 1. Case study approach: A systematic inquiry into an event or a set of related events which aims to describe and explain the phenomenon of interest.

• Can describe policy influencing process itself, and context in which research has influenced policy;

• Can explore situations where policy intervention being evaluated has no clear set of outcomes; • Tends to overemphasise role of research compared with other influences – ‘supply perspective’.

2. Episode study: A study that focuses on a clear policy change and tracks back to assess what impact research had among the variety of issues that led to the policy change.

• Process of working backwards in time can capture complex, multi-layered aspects of policy processes;

• Difficult to isolate research impact from other factors – may underemphasise research role; • Needs an actual policy change to start with; • Actors may ‘re-write history’ and may need much cross-checking.

3. Mapping actor behaviour changes:

• Based on Outcome Mapping – an integrated PM&E tool that focuses on changes in the behaviour, relationships or actions of partners (as outcomes);

• Behaviour focus allows recognition of outcomes within a project’s (or actor’s) sphere of influence, as stages in a process towards policy change;

• Captures many complex elements of policy processes, as all ultimately depend on people’s behaviour;

• Captures elements of policy implementation, rather than focusing on ‘paper’ policy.

BP 0 1 2 3 4 5 6 7 8 9 10

Project

EE

Polic

y En

viro

nmen

t

Policy Change

BP 0 1 2 3 4 5 6 7 8 9 10

Project

EE

Polic

y En

viro

nmen

t

Policy Change

Before TodayBP 0 1 2 3 4 5 6 7 8 9 10

BP1

BP2

BP3

BP4

BP5

BP6

BP7

Project

EE

Polic

y En

viro

nmen

t

year/month

Policy Change

Before TodayBP 0 1 2 3 4 5 6 7 8 9 10

BP1 0 1 2 3,4 5

BP2 0 1 2,3 4

BP3 0 1 2 3 4,5,6 7 8

BP4 0 1 2 3 4 5 6 7

BP5 0 1 2 3 4

BP6 0 1,2 3,4,5 6,7,8 9

BP7 0 1, 2 3

Project

EE

Polic

y En

viro

nmen

t

year/monthPolicy C

hange

Before TodayBP 0 1 2 3 4 5 6 7 8 9 10

BP1 0 1 2 3,4 5

BP2 0 1 2,3 4

BP3 0 1 2 3 4,5,6 7 8

BP4 0 1 2 3 4 5 6 7

BP5 0 1 2 3 4

BP6 0 1,2 3,4,5 6,7,8 9

BP7 0 1, 2 3

Project 0 1 2,3 4,5 6 7 8 9 10

EE 0 1 2 3,4 5 6 7 8 9 10 11

Polic

y En

viro

nmen

t

year/month

Policy Change

Before TodayBP 0 1 2 3 4 5 6 7 8 9 10

BP1 0 1 2 3,4 5

BP2 0 1 2,3 4

BP3 0 1 2 3 4,5,6 7 8

BP4 0 1 2 3 4 5 6 7

BP5 0 1 2 3 4

BP6 0 1,2 3,4,5 6,7,8 9

BP7 0 1, 2 3

Project 0 1 2,3 4,5 6 7 8 9 10

EE 0 1 2 3,4 5 6 7 8 9 10 11

Polic

y En

viro

nmen

t

year/month

Policy Change

Before TodayBP 0 1 2 3 4 5 6 7 8 9 10

BP1 0 5

BP2 0 4

BP3 0 8

BP4 0 7

BP5 0 4

BP6 0 9

BP7 0 3

Project

EE

Polic

y En

viro

nmen

t

year/month

Policy Change

Before TodayBP 0 1 2 3 4 5 6 7 8 9 10

BP1 0 5

BP2 0 4

BP3 0 8

BP4 0 7

BP5 0 4

BP6 0 9

BP7 0 3

Project

EE

Polic

y En

viro

nmen

t

year/month

Policy Change

Key stages of the RAPID Outcome Assessment (ROA) Actor Behaviour Workshop Step 1: Describe the policy environment at the end and beginning of the timescale.

Step 4: Map the key changes in behaviour for each of the key actors/boundary partners from the start of the timescale (or project).

Step 2: Identify key policy actors and ‘boundary partners’ that were considered influential in the process.

Step 5: Map (i) the key changes in the project including organisational changes, outputs and changes in behaviour, and (ii) external influences including actions of strategic partners and other exogenous partners, during the same period.

Step 3: Describe (i) the behaviour of the key actors/boundary partners that contributed to the change in the policy environment or policy, and (ii) the behaviour at the beginning of the timescale or project.

Step 6: Determine level of impact/influence of (i) the project on the changes in behaviour of the key actors/partners and (ii) of external influences on the changes in behaviour of the key actors/partners and the project.

Before Today

BP 0 1 2 3 4 5 6 7 8 9 10

BP1 0 1 2 3,4 5

BP2 0 1 2,3 4

BP3 0 1 2 3 4,5,6 7 8

BP4 0 1 2 3 4 5 6 7

BP5 0 1 2 3 4

BP6 0 1,2 3,4,5 6,7,8 9

BP7 0 1, 2 3

Project 0 1 2,3 4,5 6 7 8 9 10 11

EE 0 1 2 3,4 5 6 7 8 9 10 11

Polic

y En

viro

nmen

t

year/month

Policy Change

Before TodayBP 0 1 2 3 4 5 6 7 8 9 10

BP1 0 1 2 3,4 5

BP2 0 1 2,3 4

BP3 0 1 2 3 4,5,6 7 8

BP4 0 1 2 3 4 5 6 7

BP5 0 1 2 3 4

BP6 0 1,2 3,4,5 6,7,8 9

BP7 0 1, 2 3

Project 0 1 2,3 4,5 6 7 8 9 10 11

EE 0 1 2 3,4 5 6 7 8 9 10 11

Polic

y En

viro

nmen

t

year/month

Policy Change

Findings refined through in-depth interviews and further literature review, to assess the real contribution of the project on the policy change.

Influences: Direct Indirect External

The ROA in use [See accompanying brief on initial case studies and lessons]

Information and Contacts

Project website: www.pppppc.org RAPID website: www.odi.org.uk/rapid

Nick Hooton: [email protected] Dannie Romney: [email protected] John Young: [email protected]

ILRI role in dairy marketing policy change in Kenya (first application of the ROA)

CIP – Urban Harvest role in new city Ordinances on urban agriculture in Kampala (ROA approach modified)

INCREASING AWARENESS OF IMPORTANCE OF URBAN AGRICULTURE

DecentralisationLocal Gov’tAct

Research on Economic Importance of UA and Child Nutrition by Maxwell

Elected politicians hear/see about importance of U.A. from theirconstituencies

KCC Technical staff feed reports on UA up to sectoral committees

1992 1994 1996 1998 2000 2002 2004 2006

Review of outdated Ordinances

Major collaboration Research on Urban Agriculture in Kampala CIP/URBAN HARVEST & National partners incl KCC

Funding –DFID - EA

Forums on UA Ordinances

KCC Council approves Revisions

UA ordinances passed

High-level political support for UA laws

Field visits for Mayor and councillors (EA)

MORE POSITIVE VIEW OF UA & NEED FOR SUPPORTIVE LAWS

Re-Draft of Ordinances

NGO activities in UA

1992 1994 1996 1998 2000 2002 2004 2006

INCREASING AWARENESS OF IMPORTANCE OF URBAN AGRICULTUREINCREASING AWARENESS OF IMPORTANCE OF URBAN AGRICULTURE

DecentralisationLocal Gov’tAct

Research on Economic Importance of UA and Child Nutrition by Maxwell

Elected politicians hear/see about importance of U.A. from theirconstituencies

KCC Technical staff feed reports on UA up to sectoral committees

1992 1994 1996 1998 2000 2002 2004 20061992 1994 1996 1998 2000 2002 2004 2006

Review of outdated Ordinances

Major collaboration Research on Urban Agriculture in Kampala CIP/URBAN HARVEST & National partners incl KCC

Funding –DFID - EA

Forums on UA Ordinances

KCC Council approves Revisions

UA ordinances passed

High-level political support for UA laws

Field visits for Mayor and councillors (EA)

MORE POSITIVE VIEW OF UA & NEED FOR SUPPORTIVE LAWSMORE POSITIVE VIEW OF UA & NEED FOR SUPPORTIVE LAWS

Re-Draft of Ordinances

NGO activities in UA

1992 1994 1996 1998 2000 2002 2004 20061992 1994 1996 1998 2000 2002 2004 2006

External Influences political context

evidencelinks

Politics andPolicymaking

Media,Advocacy, Networking Research,

learning &thinking

Scientific information exchange & validation

Policy analysis, &research

Campaigning, Lobbying

External Influences External Influences political context

evidencelinks

Politics andPolicymaking

Media,Advocacy, Networking Research,

learning &thinking

Scientificinformationexchange &validation

Scientificinformationexchange &validation

Policy analysis, &researchPolicy analysis, &research

Campaigning,LobbyingCampaigning,Lobbying

See: www.odi.org.uk/RAPID/Tools/Toolkits/RAPID_Framework.html

External influences: Factors outside a country which affect policy processes – e.g. socio-economic or cultural influences, donor policies etc. Political context: People/institutions/processes involved in policy making. Political/economic structures and processes, culture, institutional pressures, rate of change etc. Evidence: Type and quality of evidence, including research – its credibility, whether it is contested, methodology, and how it is packaged and communicated. Links: Mechanisms affecting whether/how evidence gets into the policy process – the links between policy, research and other actors – networks, relationships and trust, power.

Box 2: The ‘Context-Evidence-Links’ framework

RAPID developed the CEL framework following extensive literature review and case studies. As an analytical framework, it simplifies the complexity of how evidence contributes to policy processes. For this project, case study findings using the ROA approach are analysed using this CEL framework.

53

Appendix 1: Research and Policy in Development Outcome Assessment (ROA)

ROA was designed as a learning methodology to assess the contribution of a project’s actions and research on a particular change in policy or the policy environment. It is a flexible, visual tool that can be used to map changes in the project and its environment, and it can be used in conjunction with other evaluation tools and methods to evaluate a particular project or programme. The key steps of the ROA were originally designed as follows: 1. Describe the policy environment at the end of the project; 2. Describe the policy environment at the beginning of the project; 3. Identify the key policy actors or agents of change; 4. Within the agents of change, identify the boundary partners that are conducive to the change or

that influence the policy environment; 5. Describe the behaviours of the boundary partners that are conducive to a change in the policy

environment or policy; 6. Describe the behaviours of the boundary partners at the beginning of the project; 7. Map the key changes in behaviour for each boundary partner from the start of the project; 8. Map the key changes in the internal environment of the project including organisational changes,

outputs and changes in behaviour during the same period; 9. Map the external influences including the actions of strategic partners and other exogenous factors

during the same period; 10. Determine the level of impact/influence of the project on the changes in behaviour of the boundary

partners; 11. Determine the level of impact/influence of external influences on the changes in behaviour of the

boundary partners and the project; 12. Refine the conclusions with in-depth interviews and assess the real contribution of the project to

the policy environment; 13. Write report. The intention was that steps 1 to 11 could be covered in a workshop with key stakeholders to produce a table similar to Figure 4 below. However, after some discussions with SDP researchers, concerns were raised about the ability of some workshop participants less exposed to training methodologies to understand the ROA methodology and to follow the discussion. It was decided that the initial workshop with SDP related staff and advocacy CSOs would not follow the steps described above rigidly. Instead, various participatory training techniques were used to collect the information needed to complete the expected output of ROA.

54

Figure 4: Example of Research and Policy in Development Outcome Assessment (ROA) Output

In the SDP workshop, the following steps were actually carried out: 1. Description of the policy environment at the beginning and the end of the project. Participants were

asked to write on cards what they thought had changed in the smallholder dairy sector in Kenya in the last few years. The cards were then organised into following categories: organisational change, policy change, practical change and behavioural change.

2. An introduction to the Smallholder Dairy Project. A presentation about the history, organisation and activities of the SDP was given by the original project manager from MoLFD.

3. Identification of SDP project time line, policy context and external factors. Participants were divided into groups: o Group 1 (SDP Staff) developed a project time-line and identified the key events and changes; o Group 2 (Other participants) were asked to develop a ‘rich description’ of the policy context for

smallholder dairy production in Kenya currently and in 1996; and to identify any key external events which influenced how the policy context for smallholder dairy production has changed.

4. Identification of key players. Participants were asked to name all the key players in the development of the smallholder dairy sector in Kenya. They were then asked to select the three that they thought had had the most impact on the smallholder dairy sector, and to write the reasons why on a coloured card – yellow for their first choice, green for their second choice and blue for the third choice. A score was calculated for each stakeholder where a yellow card equalled three points, a green card two points and blue cards one point.

5. Description of key actor behaviour. Participants were divided into groups to consider specific groups of stakeholders and wrote on cards their behaviour now, their behaviour in 1996, and key points when their ‘behaviour and attitudes’ changed in between. On the back of those cards, they wrote why they thought that behaviour change happened, and what impact they thought that the change had had on others. The cards were then pinned to the wall.

6. Participants were then asked to look at all the factors on the wall and see if they could see any links between them. Each link identified was connected using a piece of string and the reason was noted. A copy of the cards and links was projected via PowerPoint onto another wall. There was not

55

enough time to complete this activity on the second day; a smaller group completed the map the following day.

7. Identification of issues to be followed-up in the interviews with key players. In the end, the workshop was able to produce a table similar to the example above and important information about the project was gathered. From the process of applying ROA in the workshops, some useful findings and experiences should be noted:

• It is very important to know in advance exactly who will be participating in the workshop, their background, their expectations, their relationship with the project and their agendas (if any) as this will help smooth the facilitation process and enable better planning of workshop activities.

• The participants should remain the same throughout the workshop – people who are not available to attend for the whole duration of the workshop should not be invited as it will delay the progress of the workshop and disrupt the small group discussions.

• It is important that all participants understand the concepts of policy, behaviour and attitude to be able to accurately identify changes in policies. The facilitator should ensure the participants understanding before moving on to the next activity.

• Involving the project staff in the planning of the workshop and also involving them in the facilitation process is very useful for focusing the activities better and to gain ‘insights’ into participants statements and comments, thus enabling a better understanding of the project.

• Better workshop planning needs to be done to avoid a long plenary discussion, especially in identifying links between key events. The background of each participant should be collected so they can be grouped into different criteria of key actors and asked to discuss the links in small groups. It would be useful for one group to discuss the link between the project and external environment with the key events, and the other group to discuss links between other key actors and the key events. Another idea is to give three sets of strings to each participant: blue for direct effects from the project; green for effects from the key actors; and red for effects from the external environment. Participants are then asked to use the strings to link two key events and note why they think there should be a link. The result should then be discussed in a plenary session.

• Writing a daily report and having it available for the next day of the workshop is very useful in focusing the discussion on the second day because the participants can refer to the results of the discussions from the day one.

Through this case study ROA was proven to be beneficial for:

• Developing a project time line

• Identifying the key actors and the role they played in specific key events and their interaction with SDP and other key players

• Identifying changes overtime

• Identifying external influences

• Identifying the project behaviour

• Describing behaviour changes

• Identifying ‘hotspots’ (a key event which leads to many other key events or when various key events added up to one major event)

• Establishing cause and effect between various key events (to certain extent)

• Identifying issues to follow up However, the ROA was not effective for:

• Describing what actually happen within each of the key events

• Identification of the environment surrounding a specific key event

Figure 2: Causal factors