Simon Batchelor

Interim Impact and Learning Team manager

Date:29/3/10

Monitoring and Evaluation

Some thoughts on Theory of Change and Outcome Mapping

2

Theory of Change

Simply put its about writing down what change you expect to happen?

And how that change might come about?

3

Theory of Change (TOC) vs. Logical Framework (LF)!

Some people will be familiar with Logical Frameworks

Donors tend to ask for Logical Framework (LF)

Is this the same as a Theory of Change?

Not quite.

Both are suppose to help project design

The idea of “Theory of Change” is growing

People talk about TOCs and LFs but dont mean the same thing

Limited knowledge on how to use TOC

TOC and LFs can “blend” into each other

4

Theories of Change

Outcomes-based

Causal model

Articulate underlying assumptions

“Logical Frameworks graphically illustrate program components, and

creating one helps stakeholders clearly identify outcomes, inputs and

activities”

“Theories of Change link outcomes and activities to explain HOW and WHY

the desired change is expected to come about”– Ref Clark and Anderson 2004

5

Long-termOutcome

Necessary Pre-

condition

Necessary Pre-

condition

Necessary Pre-

condition

Necessary Pre-

condition

Necessary Pre-

condition

All outcomes that must be achieved BEFORE long-term

Explain WHY here

Show activities here also

Ref Clark and Anderson 2004

6

So explain the HOW and the WHY

Explain how your work will affect people and make a change

Explain why your work will affect people and create a change

7

Clarifying the difference“Logic Models usually start with a program and illustrate its components

Theories of Change may start with a program, but are best when starting with a goal, before deciding what programmatic approaches are needed

Logic Models require identifying program components, so you can see at a glance if outcomes are out of sync with inputs and activities, but they don’t show WHY activities are expected to produce outcomes

Theories of Change also require justifications at each step – you have to articulate the hypothesis about why something will cause something else (it’s a causal model, remember!)”

– Ref Clark and Anderson 2004

8

Explaining the outcome

How often do students need to attend the after-school program in order for their reading tests scores to improve?

A Logic Model would tell you that the after school program is an activity and improved reading scores is an outcome. It might tell you that attendance at the after school program is an intermediate outcome.

But it wouldn’t tell you that:

“students need to attend after-school programs at least 3 days per week for a minimum of 60 days, and the curricula must focus on love of reading and literacy, IN ORDER FOR test scores to rise”

– Ref Clark and Anderson 2004

9

When to Use?

Logical Frameworks are great when you need to:

• Show someone something they can understand at a glance

• Demonstrate you have identified the basic inputs, outputs and outcomes for your work

• Summarize a complex theory into basic categories

Theories of Change are best when you need to:

• Design a complex initiative and want to have a rigorous plan for success

• Evaluate appropriate outcomes at the right time and the right sequence

• Explain why an initiative worked or did not work, and what exactly went wrongRef Clark and Anderson 2004

10

Summary

Logical Frameworks Theories of Change

Representation

List of Components

Descriptive

Critical Thinking

Pathway of Change

Explanatory

Ref Clark and Anderson 2004

11

So how does this apply to I K Mediary Network?

In IDS we found this exercise useful.

You can find our thinking on our own theory of change.

(Note a Google search on Theory of Change will lead you to the “superwoman” project - It’s a good example to start with.)

12

At IDS we said

Access to information cannot be viewed as an end in itself but as a contributor to development processes and social change.

Individual stories from users of the MK4D programme have helped to develop a framework to guide our work and improve planning, evaluation, analysis and targeting of services.

The resulting theory of change sets out how the programme contributes to information use in development and the model will evolve as it is tested, debated and used in planning, evaluation and research.

13

At its most simplistic, we see our influence as having a number of layers:

14

15

16

TOC maps out your initiative through 5 stages

Identifying long-term goals and the assumptions behind them

Backwards mapping and connect the preconditions or requirements necessary to achieve that goal.

Identifying the interventions that your initiative will perform to create your desired change.

Developing indicators to measure your outcomes to assess the performance of your initiative.

Writing a narrative to explain the logic of your initiative.

17

Or to put it another way….

What are you trying to achieve?

What changes in people do you expect to see?

How will these changes come about?

18

Theory of Change led to talking about Outcomes

Changes in the behaviour, relationships, activities and/or actions of our target groups that can be logically linked to our programme (although they are not necessarily directly and solely caused by it).

Outcomes are changes we would like to see in other people, and are thus not entirely within our control. We have broadly divided our theory of change into immediate, intermediate and longer term outcomes.

Our intermediate outcomes around understanding and influence sit between access to information and the end use of that information. It is about both the understanding and knowledge that results from engaging with information, and the influence that the kinds of information we promote can bring.

Higher level outcomes are the changes in behaviours of development actors in which we hope to see information influence development processes and interventions.

19

And thinking about Outcomes led us to Outcome mapping

Championed by IDRC, Outcome mapping focuses on outcomes and fits with the theory of change process.

It challenged the direct causality of the logic model

It recognises the world is a complex place!

And it therefore focuses on a range of possible outcomes (hence the term map rather than route).

20

Last time on these sessions you discussed Policy Mapping – lots of questions

What policy, practice or discourse are you seeking to influence by communicating to this audience?

What attitudes or beliefs might impede influence?

What kind of knowledge can most effectively influence this audience?

Which are the best mediums for reaching this audience?

What is your objective in engaging with policy?

What kind of policy actors would be most likely to use your resources?

21

It talked about mapping who talked to who

22

Key messages of Outcome mapping

Seeing yourself as a part of a interconnected web of relationships and systems

Recognizing that change is Continuous, Complex, Non-linear, Multidirectional, Not controllable

Keeping your eyes wide open - Being attentive along the journey is as important as the destination

Contribution not attribution - your influence on a better world - you can influence but not control change in your partners

23

In other words……

Whatever you are trying to achieve, you have to work with people?

It would be great to know who those people are – but its not simple.

You cant control those people – but you might influence them.

24

Focuses on the who

Key concept is « boundary partners »

The individuals, groups, and organizations you work with directly and anticipate opportunities for influence

boundary partners have boundary partners

Therefore network mapping comes into play

25

Boundary partners are those that engaged with you

ExamplePortal specialising in NGO issues ?

26

Boundary partners are those that engaged with you

ExamplePortal specialising in NGO issues ?

Boundary actors for the portal

27

Boundary partners are those that engaged with you

ExamplePortal specialising in NGO issues ?

Boundary actors for the NGOs engaged in policy process

28

In other words…..

You engage with some people (Boundary partners)

They engage with others

Ideas flow from one to the other hopefully creating change (outcomes)

29

Outcome mapping talks about “progress markers”

A graduated set of statements describing a progression of changed behaviours in the boundary partner

Describe changes in actions, activities and relationships leading to the ideal outcome

Shows story of change by articulating the complexity of the change process

Can be monitored & observed

Permit on-going assessment of partner’s progress (including unintended results)

Talks about an Outcome Journal

– Story of change and reasons for change

– Unexpected changes

– The actors and factors that contributed to that change

– How we know the change occurred

– Learnings (what? how? why?)

30

Or to put it another way….

What are you trying to achieve and in who?

What changes do you expect to see in those people (and how might you “see” them)?

And who might they tell about the new ideas and what changes might you expect in them.

31

How can Outcome mapping help?

focus more on contribution to outcomes, rather than attribution!

understanding the boundary partners and strategic partners attitudes and approach is essential

move from boundary partners to outcome challenges to progress markers to strategy maps in a participatory fashion

use Organisational Practices systematically to ensure strategies are tailored to existing organisational contexts

use journals to address the crucial M&E gap

use the flexibility of OM to combine with other compatible methodologies

– Ben Ramalingam 2005

32

Questions for discussion

So do you think you can apply Theory of Change to your work?

Outcome mapping relies on knowing your Boundary partners – do you know your users, and do you know what change you expect in them?

User surveys can seek the outcome stories of change – do your existing surveys ask about how users used the information you gave them?

33

Acknowledgements

This was a short training session, not a publication.

For the opening section I rely heavily on Clark and Anderson 2004, which can be found at

www.theoryofchange.org/ppt/TOCs_and_Logic_Models_forAEA.ppt

I am grateful for their succinct summary of the Logic/TOC difference.

And Ben Ramalingam 2005 can be found at

http://www.idrc.ca/uploads/user-S/11335475941OMandKM_Toronto.pps