outcomemapping.ca

Complexity and Outcome Harvesting

Most recentlyIn June the World Bank published 10 case studies

and a toolkit for using Outcome Harvesting

2http://www.outcomemapping.ca/resource/resource.php?id=452

http://wbi.worldbank.org/wbi/document/cases-outcome-harvesting

e Harvesting

outcomemapping.ca

e Harvesting

4

Inspired and informed by the Outcome Mapping methodology.

A tool for practitioners operating in dynamic, uncertain situations to monitor and evaluate the social change results they are achieving.

Outcome HarvestingSix steps:

1.Design the harvest — users => uses => evaluation questions => data to be collected

2.Review documentation and draft outcomes => intervention’s contribution

3.Engage with informants — generally within the intervention

4.Substantiate with independent but knowledgeable third parties

5.Analyse, interpret — What? + So What? but not Now What?

6.Support use of findings

TRIA

NG

ULA

TIO

N

outcomemapping.ca

International social change networks

6

outcomemapping.ca

International development funders

7

outcomemapping.ca

What did all of these organisations have in common?

COMPLEXITY! COMPLEXITY!

outcomemapping.ca

Relationships of cause and effect are KNOWN

Simple

9

Annual polio vaccination campaign

ACTIVITIES

OUTCOMES

IMPACT

INPUTS

Inspired by Jeff Conklin, cognexus.org

Time

OUTPUTS

M&E of a simple intervention

10

outcomemapping.ca

Relationships of cause and effect are UNKNOWN

Complex

11

OUTPUT

OUTCOME

INPUTS

ACTIVITY

INPUTSACTIVITY

INPUTS

ACTIVITY

INPUTS

OUTPUT

OUTPUT

ACTIVITY

OUTPUT

OUTPUT

OUTCOME

OUTCOME

OUTCOME

OUTCOME

OUTCOME

M&E of a complex intervention

Time

Strengthening nonviolent responses to communal conflict

in the Horn of Africa

12

outcomemapping.ca

OUTPUT

OUTCOME

INPUTS

ACTIVITY

INPUTSACTIVITY

INPUTS

ACTIVITY

INPUTS

OUTPUT

OUTPUT

ACTIVITY

OUTPUT

OUTPUT

OUTCOME

OUTCOME

OUTCOME

OUTCOME

OUTCOME

Time

Dimensions in which the relationships of cause and effect are known

NOT EITHER OR

13

outcomemapping.ca

OUTPUT

OUTCOME

INPUTS

ACTIVITY

INPUTSACTIVITY

INPUTS

ACTIVITY

INPUTS

OUTPUT

OUTPUT

ACTIVITY

OUTPUT

OUTPUT

OUTCOME

OUTCOME

OUTCOME

OUTCOME

OUTCOME

Time

COMPLEX

Results are substantially unforeseeable.

Unknown relations of cause and effect dominate.

NOT EITHER OR

14

outcomemapping.ca

Outcome Harvesting and complexity

The greater the:

1. Disagreement about what is the development challenge

2. Disagreement about what is its solution

3. Uncertainty about what will be the results of your actions to solve the development challenge

The more Outcome Harvesting may be useful

CO

MP

LEX

IME

TE

R

High

Low

15

16

Formative evaluation

Summative evaluation

Developmental evaluation

Developmental Evaluation is not, however, the same as

evaluation of development.

outcomemapping.ca

When do you evaluate?

Time

SummativeDevelopmental Formative

Progress of intervention

17

outcomemapping.ca

RESULTS BASED M&E

STEP 1: Vision

INTENTIONAL DESIGN

STEP 2: Mission

STEP 3: Boundary Partners

STEP 4: Outcome Challenges

STEP 5: Progress Markers

STEP 6: Strategy Maps

outcomemapping.ca

STEP 1: Vision

STEP 2: Mission

STEP 3: Boundary Partners

STEP 4: Outcome Challenges

STEP 5: Progress Markers

STEP 6: Strategy Maps

Indicators

Indicators

GLOBAL

PARTNERSHIP FOR

THE PREVENTION OF

ARMED CONFLICT

STEP 1: Vision

STEP 2: Mission

STEP 3: Boundary Partners

STEP 4: Outcome Challenges

STEP 5: Progress Markers

STEP 6: Strategy Maps

Indicators

Indicators

ICT4D in Asia PAN and Africa ACACIA Nigeria Evidence based Health System Initiative‐ EcoHealth Fieldbuilding Leadership Initiative Consorcio por la Salud, Ambiente y Desarrollo

Regional Peacebuilding Programme in the Horn

of Africa

outcomemapping.ca

STEP 1: Vision

STEP 2: Mission

STEP 3: Boundary Partners

STEP 4: Outcome Challenges

STEP 5: Progress Markers

STEP 6: Strategy Maps

Indicators

Indicators

Art and Culture Programme in Central America

outcomemapping.ca

In sum

Outcome Harvesting is “an evaluation approach that does not measure progress towards predetermined outcomes, but rather collects evidence of what has been achieved, and works backward to determine whether and how the project or intervention contributed to the change.” - UNDP

22

https://undp.unteamworks.org/node/370238

outcomemapping.ca

Discussion

Have any of you found other ways to apply the principles of Outcome Mapping to evaluate projects, programmes or organisations when the elements of intentional design have been missing?