developing evaluation capacity jean langlois, msc, mba
TRANSCRIPT
Developing Evaluation Capacity
Jean Langlois, MSc, MBA
www.naturecanada.ca
Outline
Intro Nature of the work NGO funding model Motivating people Leading questions
www.naturecanada.ca
Introduction
The world according to Jean Langlois…
Challenges in developing evaluation capacity at an ENGO Nature of the work NGO funding model Motivating people
www.naturecanada.ca
Nature of the work - Challenges
Advocacy: Diffuse causality Causality attribution
Direct Conservation: Measurable vs.
Important biases Durability of
outcomes Defining outcomes
and impacts
www.naturecanada.ca
Nature of the work
What works: Invest time in the
logframe at the outset
Articulate and track tangible results
Articulate, don’t track, intangibles
What’s needed: Shared comfort with
unmeasured intangible benefits
www.naturecanada.ca
Nature of the work - Questions
How do we combat the biases of the measurable vs. the important?
Is it useful to articulate intangible benefits that will not be measured?
For advocacy campaigns, is there a better model than the dotted-line-logframe?
www.naturecanada.ca
NGO funding model - Challenges
Not enough money, ever www.naturecanada.ca
Reporting to multiple funders Multiple evaluation frameworks Varying degrees of buy-in to rigorous evaluation
Propensity to provide “positive” reports
www.naturecanada.ca
NGO funding model
What works: Invest time in
logframe at the outset
Consistent deliverables across grant proposals
What’s needed: Shared expectations Comfort reporting
and learning from failure
www.naturecanada.ca
NGO funding model - Questions
How do we create an environment where people are comfortable sharing and learning from “failure”?
www.naturecanada.ca
Motivating people - Challenges
“The effectiveness of our program will be measured by the degree to which the outputs and outcomes identified in the logical framework are achieved.”
- NOT Martin Luther King Jr
www.naturecanada.ca
Motivating people - Challenges
“Evaluation is a time drain” “It’s not my job” “I’d get more work done if I didn’t have to
report on everything I do”
- Anonymous Colleagues
www.naturecanada.ca
Motivating people
What works: Adapt the language
to the audience Always relate to
mission Tough love: invest in
logframe at the outset
What’s needed: Positive experiences
with evaluation Project-to-project
application of lessons learned
www.naturecanada.ca
Motivating people - Questions
How can we create short-term wins (i.e. positive experiences with evaluation)?
How can we design evaluation approaches that produce useful results that are easily applicable to multiple future projects?
www.naturecanada.ca
Leading questions How do we combat the biases of the measurable vs.
the important? Is it useful to articulate intangible benefits that will
not be measured? For advocacy campaigns, is there a better model
than the dotted-line-logframe? How do we create an environment where people are
comfortable sharing and learning from “failure”? How can we create short-term wins (i.e. positive
experiences with evaluation)? How can we design evaluation approaches that
produce useful results that are easily applicable to multiple future projects?
Merci