developing evaluation capacity jean langlois, msc, mba

Developing Evaluation Capacity

Jean Langlois, MSc, MBA

www.naturecanada.ca

Outline

Intro Nature of the work NGO funding model Motivating people Leading questions

www.naturecanada.ca

Introduction

The world according to Jean Langlois…

Challenges in developing evaluation capacity at an ENGO Nature of the work NGO funding model Motivating people

www.naturecanada.ca

Nature of the work - Challenges

Advocacy: Diffuse causality Causality attribution

Direct Conservation: Measurable vs.

Important biases Durability of

outcomes Defining outcomes

and impacts

www.naturecanada.ca

Nature of the work

What works: Invest time in the

logframe at the outset

Articulate and track tangible results

Articulate, don’t track, intangibles

What’s needed: Shared comfort with

unmeasured intangible benefits

www.naturecanada.ca

Nature of the work - Questions

How do we combat the biases of the measurable vs. the important?

Is it useful to articulate intangible benefits that will not be measured?

For advocacy campaigns, is there a better model than the dotted-line-logframe?

www.naturecanada.ca

NGO funding model - Challenges

Not enough money, ever www.naturecanada.ca

Reporting to multiple funders Multiple evaluation frameworks Varying degrees of buy-in to rigorous evaluation

Propensity to provide “positive” reports

www.naturecanada.ca

NGO funding model

What works: Invest time in

logframe at the outset

Consistent deliverables across grant proposals

What’s needed: Shared expectations Comfort reporting

and learning from failure

www.naturecanada.ca

NGO funding model - Questions

How do we create an environment where people are comfortable sharing and learning from “failure”?

www.naturecanada.ca

Motivating people - Challenges

“The effectiveness of our program will be measured by the degree to which the outputs and outcomes identified in the logical framework are achieved.”

- NOT Martin Luther King Jr

www.naturecanada.ca

Motivating people - Challenges

“Evaluation is a time drain” “It’s not my job” “I’d get more work done if I didn’t have to

report on everything I do”

- Anonymous Colleagues

www.naturecanada.ca

Motivating people

What works: Adapt the language

to the audience Always relate to

mission Tough love: invest in

logframe at the outset

What’s needed: Positive experiences

with evaluation Project-to-project

application of lessons learned

www.naturecanada.ca

Motivating people - Questions

How can we create short-term wins (i.e. positive experiences with evaluation)?

How can we design evaluation approaches that produce useful results that are easily applicable to multiple future projects?

www.naturecanada.ca

Leading questions How do we combat the biases of the measurable vs.

the important? Is it useful to articulate intangible benefits that will

not be measured? For advocacy campaigns, is there a better model

than the dotted-line-logframe? How do we create an environment where people are

comfortable sharing and learning from “failure”? How can we create short-term wins (i.e. positive

experiences with evaluation)? How can we design evaluation approaches that

produce useful results that are easily applicable to multiple future projects?

Merci