marcia brown thursday

8/14/2019 Marcia Brown Thursday

1/59

Results Chains: A Tool forMeasuring Effectiveness

and Attributing Change to

Conservation Programs

Environmental Evaluators Forum

Washington D.C.June 12, 2008


2/59

This Presentation

1. Background on the Use of Evaluation &Adaptive Management in the Field of

Biodiversity Conservation

2. What Is a Results Chain3. How to Develop Results Chains

4. How to Use Results Chains as a

Framework for Attributing Change toConservation Programs

5. Build Example Chains


3/59


4/59

Common Questions

Are we achieving an impact?

Are we doing the right things?

Are we doing them well?


5/59

To Achieve Success, We Need To:

Develop sound theories of change Measure the results of and improve

effectiveness of actions

Assess impact on ecosystems and species Convince managers, donors, andsupporters that results are credible


6/59


7/59

What is Adaptive Management?

The integration of design, management, andmonitoring to provide a framework for:

Testing assumptions

Adaptation Learning


8/59

Foundations of Success

Our Mission

To improve the practice of conservation through

adaptive management working withpractitioners to systematically test assumptions,

adapt, and learn.


9/59

Foundations of Success

Our Goals

Our goals involve achieving the threefoundations of success:

Foundation #1: Define clear goals andpractical measures of success. Foundation #2: Determine guiding

principles for using conservation strategies

and tools. Foundation #3: Develop and strengthen

practitioners ability to do AM


10/59

The Conservation Measures Partnership:

Leading Conservation Organizations

Core Members:

Collaborating Members:

Funding Support:


11/59

The CMP Open Standards

for the Practice of Conservation

www.ConservationMeasures.org

[email protected]


12/59


13/59

CMP Open Standards


14/59

Results Chains A Tool for

Implementing the CMP Open Standards

Resultschains


15/59

How are the CMP Open Standards

Structured?

Steps

Sub-stepsDescription

Outputs

References


16/59

Step 1: Conceptualize

Sub-step 1: Complete Situation Analysis

Description: This standard asks you to complete the description ofthe context within which your project takes place

Outputs:

Identification and analysis of indirect threats and opportunities. Assessment of stakeholders. Initial conceptual model that illustrates cause and effect

relationships among factors operating at your site.

Ground-truthing and revision of your model.

References

Example from the CMP Standards


17/59

Interchange Between CMP and Orgs

Initial team

Scope & vision Targets

Context & stakeholders

1. Define

Action plan: goals,objectives & activities

Monitoring plan Operational plan

2. Design

Workplans & budgets Fund raising Capacity building Partnerships

3. Implement

Incoming data Results & assumptions Operational functions Plans & budgets

4. Analyze/Adapt

Lessons Formal products

Feedback & evaluation Learning culture

5. ShareWWFs

ConservationProject/Programme

Cycle

CI

WWFWCS

TNC

Implement Actionsand Measure

EffectivenessWork-plans and Budgets

Implement actionsMonitor progress

Define thecontext

Vision and scope

Threats mappingStakeholders

Design Approach andMeasures of Success

Select targetsCreate conceptual models

Build conservation landscapesMonitoring frameworks

Review Progress andRevise Approach

Evaluate impactsAdapt to changes

Share lessons learned

2. HeartlandSelection

1. Priority

Setting

AWF Heartland Conservation Process

4. Heartland Strategy

Implementation,Evaluation, and

Adaptation

Performance

And Impact

Assessment

Implementation

of PriorityInterventions

Learning and

Adaptive

Management

3. Heartland/Landscape-level Planning

Implementation

Planning

ConservationTarget and

Goal setting

Threats and

OpportunityAnalysis

Socio-economicAnalysis

5. Scale-down Heartland

operations

Initial Scoping



1. Priority

Setting

1. Priority

Setting

AWF Heartland Conservation Process



Adaptation

Performance

And Impact

Assessment

Implementation


Learning and

Adaptive

Management



Adaptation

Performance

And Impact

Assessment

Implementation




Adaptation

Performance

And Impact

Assessment

Implementation


Learning and

Adaptive

Management


Implementation

Planning


Goal setting

Threats and

OpportunityAnalysis



Implementation

Planning


Goal setting

Threats and

OpportunityAnalysis


5. Scale-down Heartland

operations

Initial Scoping

AWF

CMP


18/59


19/59

A tool that clarifies assumptions about howconservation strategies contribute to

reducing threats and achieving the

conservation of targets

What is a Results Chain?


20/59

The Basic Components of a Results Chain:

Strategy



21/59


Strategy

Impact on

Target

Goal



22/59


Result(Direct Threat )

Result

Strategy

Objective Objective

Impact on

Target

Goal



23/59

Results Chain Terminology

Outcomesinterim results(objectives)

achieved byoutputs

Strategyset of actions

undertaken by a

project

Outputsimmediateproducts of

project activities

Outcomesinterim results(objectives)

achieved byoutputs

Inputsstaff, time,

money, other

resources

ResultsProcess

Impactsdesired end goals

of the project

Logic Model

ImpactsOutcomesOutputsActivitiesInputs

Results Chain


24/59


A diagram of a series of ifthen causalstatements that:

Defines how a project team thinks a

strategy will contribute to reducing a threatand conserving a target

Focuses on the achievement of results

not the implementation of activities Is composed of assumptions that can be

tested


25/59

Results Chains

Implicit Assumptions:

StrategyConservation

target improved?

?


26/59

Achieving Success

Accurate

Results Chain

Well executed

projectDesired results

used to

design

leads to

Accurate

Results Chain

Poorly

executed

project

Desired resultsdoes not

lead to

Inaccurate

Results Chain

Well executed

project Desired resultsdoes not

lead to

Inaccurate

Results Chain

Poorly

executed

project

Desired resultsdoes not

lead to

SUCCESS!

THEORY FAILURE

PROGRAM FAILURE

TOTAL FAILURE!!!

used to

design

used to

design

used to

design


27/59


28/59

How to Develop a Results Chain

1. Construct an initial results chain2. Complete the links in the results chain

3. Verify that your results chain meets

criteria of a good results chain

A Chain From the Model Allows You


29/59


toScope:

Volga River

watershed

Clearing fornew home

construction

Dams

Harvestingfor caviar

Pollution

from domesticsewage

Demand for

caviar

High price forcaviar

Rapid

urbanization

Need for

electricity

Global warming

Traditions &consumer

preferences

Need to

generate

income

Inadequate

zoning

regulations

Limited govt

capacity for landuse planning

Population

growth

Demand forsecond homes

Failure of rural

economyCheap land

Govt policiesfavorable tourban dev.

Riparian

forest

Forest

corridors

Volga River

& tributaries

Sturgeon



30/59


toScope:

Volga River

watershed

Clearing fornew home

construction

Dams

Harvestingfor caviar

Pollution

from domestic

sewage

Demand for

caviar

High price for

caviar

Rapid

urbanization

Need for

electricity

Global warming

Traditions &

consumer

preferences

Need to

generate

income

Inadequate

zoning

regulations

Limited govt

capacity for land

use planning

Population

growth

Demand for

second homes

Failure of rural

economyCheap land

Govt policiesfavorable to

urban dev.

Media

campaign

Riparian

forest

Forest

corridors

Volga River

& tributaries

Sturgeon


31/59

1. Construct an Initial Results Chain

Healthy

sturgeon

population


32/59


Reduction inharvesting for

caviar

Healthy

sturgeon

population


33/59


Media

campaign


caviar

Healthy

sturgeon

population?


34/59

2 Complete the Links in the Results


35/59

2. Complete the Links in the Results

Chain

Consumersprefer not toeat caviar

Media

campaign


caviar

Decreaseddemand for

caviar? Healthysturgeon

population



36/59


Chain


Media

campaign


caviar

Decreaseddemand for

caviar

Increased

knowledge of

importance of

sturgeon? Healthysturgeon

population



37/59


Chain


Media

campaign


caviar

Decreaseddemand for

caviar

Increased

knowledge of

importance of

sturgeon

Healthysturgeon

population

Increased

interest in

sturgeon

conservation

3 Review the Criteria for Good Results


38/59

3. Review the Criteria for Good Results

Chains


Mediacampaign


caviar

Decreaseddemand for

caviar

Increased

knowledge ofimportance of

sturgeon

Healthysturgeon

population

Increased

interest insturgeon

conservation

Results oriented:Boxes contain desired results(e.g., reduction of hunting), and not activities

(e.g., conduct a study). Connected in a causal manner:There are

clear connections of ifthen between each pairof successive boxes.



39/59


Chains


Mediacampaign


caviar

Decreaseddemand for

caviar

Increased


sturgeon

Healthysturgeon

population

Increased

interest insturgeon

conservation

Demonstrates changes:Each box describeshow you hope the relevant factor will change

(e.g., improve, increase, or decrease).



40/59


Chains


Mediacampaign


caviar

Decreaseddemand for

caviar

Increased


sturgeon

Healthysturgeon

population

Increased

interest insturgeon

conservation

Relatively complete:There are sufficient boxesto construct logical connections but not so many

that the chain becomes overly complex.

Simple:There is only one result per box.


41/59

How to Develop a Results Chain

1. Construct an Initial Results Chain Basedon Your Conceptual Model

2. Complete the Links in the Results Chain

3. Verify that Your Results Chain MeetsCriteria of a Good Results Chain


42/59


A diagram of a series of ifthenstatements (causal) that:

Defines how a project team thinks a

strategy will contribute to reducing a threatand conserving a target

Focuses on the achievement of results

not the execution of activities Is composed of assumptions that can be

tested


43/59

What is NOT a Results Chain?

It is not an implementation flow diagram

Identify target

audience

Media

campaign

Distribute

educational

materials

Produce

educational

materials

Monitor &

evaluate the

campaignseffectiveness

Healthy

sturgeon

population

Your Turn: Which of the Following is NOT a


44/59

Your Turn: Which of the Following is NOT a

Results Chain?

Community

capacity building

for forest resource

management

Greater

indigenous

knowledge

about rights

More control

of & vigilance

over external

actors

More illegal

wood

confiscated

Less illegal

selective logging

in indigenous

communities

Primary

forest

conserved

Increasedpermanence

of agricultural

occupation

Farmers implement

sustainable

agriculture methodsMore

permanent

crops

Increased

yields

Promotion ofsustainableagriculture

Coastal

forests

conserved

Miombo

woodland

conserved

Lobbying ofgovernment for

strongerregulations

Jaguar

populations

increased

Identify key

decision

makers

Educate

decision

makers

Decision

makers pass

laws

Research &

develop

regulations

No wildlife

trade

Lessconversion of

forest to

agriculture

A.

B.

C.


45/59

This Presentation

Background on the Use of Evaluation &Adaptive Management in the Field of


What Is a Results Chain How to Develop Results Chains

How to Use Results Chains as a


Build Example Chains

S C


46/59

Steps to Attributing Change to Programs

1. Define the Programs Theory of Change2. Develop Key Results into Good

Objectives

3. Define Indicators for Objectives andGoals

4. Measure Indicators

5. Define What Results are Directly vs.Indirectly Attributable to Program

E l f th M A i R f


47/59

Example from the Meso-American Reef

Mexico

HondurasGuatemala

Belize

1 D fi h P Th f Ch


48/59

1. Define the Programs Theory of Change

1 D fi th P Th f Ch


49/59


Coral

reefs

Lessagrochemical

contamination inmarine waters &

organisms

KEY

ThreatReduction

ResultResult TargetStrategy

Promote

Agricultural Best

Mgmt Practices

(BMPs)

Obj Ag6, Ag7Man-

groves

Sea-

grasses

Littoral

zone

1 D fi th P Th f Ch


50/59


Coral

reefs

Lessagrochemical


organisms

Companies

sign MOU

Companies agree

to participate in

program to reduce

agrochemicals

Companies

accept BMP

plan

KEY

ThreatReduction


Promote

Agricultural Best

Mgmt Practices

(BMPs)

Bioaccumulation

study completed

& shared with

agroindustry

Obj Ag1 Obj Ag2 Obj Ag3

Obj Ag6, Ag7Man-

groves

Sea-

grasses

Littoral

zone

1 D fi th P Th f Ch


51/59


Coral

reefs

Lessagrochemical


organisms

Companies

sign MOU

Companies agree

to participate in

program to reduce

agrochemicals

Companies

implement

BMPs

Companies

accept BMP

plan

KEY

ThreatReduction


Promote

Agricultural Best

Mgmt Practices

(BMPs)

Bioaccumulation

study completed

& shared with

agroindustry


Obj Ag6, Ag7Man-

groves

Sea-

grasses

Littoral

zone

Less

agrochemical

toxicity in

plantations

Less toxic

agrochemical

contamination in

streams & rivers

Obj Ag4, Ag5


52/59

2 D l K R lt I t G d Obj ti


53/59

2. Develop Key Results Into Good Objectives

Coral

reefs

Lessagrochemical


organisms

Companies

sign MOU

Other

companies

accept BMPs

Companies agree

to participate in

program to reduce

agrochemicals

Industry

adopts

BMPs

Companies

implement

BMPs

Companies

accept BMP

plan

KEY

ThreatReduction


Promote

Agricultural Best

Mgmt Practices

(BMPs)

Bioaccumulation

study completed

& shared with

agroindustry


Obj Ag6, Ag7Man-

groves

Sea-

grasses

Littoral

zone

Less

agrochemical

toxicity in

plantations

Less toxic

agrochemical

contamination in

streams & rivers

Obj Ag4, Ag5

Ag5: By the end of 2012, reduce by 40% the

total pesticide toxicity from fungicides,

insecticides, and herbicides used onbanana, citrus and sugarcane in high run-off

Mesoamerican Reef watershed areas

3 D fi I di t f Obj ti d G l


54/59

3. Define Indicators for Objectives and Goals

Coral

reefs

Lessagrochemical


organisms

Companies

sign MOU

Other

companies

accept BMPs

Companies agree

to participate in

program to reduce

agrochemicals

Industry

adopts

BMPs

Companies

implement

BMPs

Companies

accept BMP

plan

KEY

ThreatReduction


Promote

Agricultural Best

Mgmt Practices

(BMPs)

Bioaccumulation

study completed

& shared with

agroindustry


Obj Ag6, Ag7Man-

groves

Sea-

grasses

Littoral

zone

Less

agrochemical

toxicity in

plantations

Less toxic

agrochemical

contamination in

streams & rivers

Obj Ag4, Ag5

Ag5: By the end of 2012, reduce by 40% the

total pesticide toxicity from fungicides,

insecticides, and herbicides used onbanana, citrus and sugarcane in high run-off

Mesoamerican Reef watershed areas

Indicator: total pesticide toxicity from

fungicides, insecticides, and herbicides used on

banana, citrus and sugarcane

3 D fi I di t f Obj ti d G l


55/59

3. Define Indicators for Objectives and Goals

Coral

reefs

Lessagrochemical


organisms

Companies

sign MOU

Other

companies

accept BMPs

Companies agree

to participate in

program to reduce

agrochemicals

Industry

adopts

BMPs

Companies

implement

BMPs

Companies

accept BMP

plan

KEY

ThreatReduction


Promote

Agricultural Best

Mgmt Practices

(BMPs)

Bioaccumulation

study completed

& shared with

agroindustry


Obj Ag6, Ag7Man-

groves

Sea-

grasses

Littoral

zone

Less

agrochemical

toxicity in

plantations

Less toxic

agrochemical

contamination in

streams & rivers

Obj Ag4, Ag5

Ag1: By the end of 2008, develop a list of

agrochemicals to reduce

Indicator: list

developed

Ag4: By the end of 2012, at least 16

companies are implementing BMP

programs

Indicator: # of

companies implementing

BMP programs

Goal for Coral Reefs: Goal for Coral Reefs:

By 2017, at least 25% of coral reefs in the

MAR (including all reef types) are

ecologically viable.*

* ecologically viable = accretion rate >/= X,

live coral coverage of 7-20%, healthy

population of herbivores, balanced species

composition, etc.

4 M I di t


56/59

4. Measure Indicators


57/59

5. Define What Results are Directly vs. Indirectly


58/59

Coral

reefs

Lessagrochemical


organisms

Companies

sign MOU

Other

companies

accept BMPs

Companies agree

to participate in

program to reduce

agrochemicals

Industry

adopts

BMPs

Companies

implement

BMPs

Companies

accept BMP

plan

KEY

ThreatReduction


Promote

Agricultural Best

Mgmt Practices

(BMPs)

Bioaccumulation

study completed

& shared with

agroindustry


Obj Ag6, Ag7Man-

groves

Sea-

grasses

Littoral

zone

Less

agrochemical

toxicity in

plantations

Less toxic

agrochemical

contamination in

streams & rivers

Obj Ag4, Ag5

y y

Attributable to Program

Results directlyattributable to

WWF

Resultsindirectly

attributable toWWF

Impossible

to attributeresults

solely toWWF

This Presentation


59/59

This Presentation

Background on the Use of Evaluation &Adaptive Management in the Field of


What Is a Results Chain How to Develop Results Chains

How to Use Results Chains as a


Build Example Chains

marcia brown thursday

Documents