Evaluation Engaging Science Conference

April 2016

Derrick Jones, Economist

Contents

About Pro Bono Economics

How we work

Evaluation: typical issues and things to think about

Some references/examples

2© Pro Bono Economics

Our objectives

PBE matches professional economists with charities;

providing pro bono help to measure performance and understand charity impact; and

fostering a culture of volunteering within the economics profession

3© Pro Bono Economics

© Pro Bono Economics

Scope

Analyse

Quality Assure

Publish

Data on outcomes

Control group

Monetary values

Transparent Conservative Rigorous Economic analysis

Assess & Match

PBE team

Volunteers

HOW WHO WHAT

Typical questions from funders to charities…

1) Who are you aiming to help? (target group)

2) What is your project trying to achieve for your clients? (the outcomes question)

3) What is your project doing with the clients? (activities)

4) What outcomes are you achieving for your clients? (monitoring)

5) How do you know it’s your project rather than other factors causing those outcomes? (the attribution issue)

6) What’s the economic value/benefit of your impact?

7) What does the project cost?

8) Is it value for money..?

5

Some key issues we have come across…

• Lack of clarity about Objectives, Outputs and Outcomes

• Lack of a clear Theory of Change, linking Activities and Resources to Outputs and Outcomes

• (Plans often focus on monitoring throughput of projects)

• Lack of reasonable data on

• Size of target group (and hence take-up)

• Inputs/resources (£ and in-kind)

• Effect-size evidence (to predict outputs and outcomes)

• Need for a control group or counter-factual

6© Pro Bono Economics

Some References…

• Pro Bono Economics website http://www.probonoeconomics.com

http://www.probonoeconomics.com/content/5-tips-charities

Some PBE evaluation projects with charities…

http://www.britishscienceassociation.org/crestsilver-report

http://www.probonoeconomics.com/news/publications/looking-viewer-engagement-royal-institutions-online-science-videos

http://www.probonoeconomics.com/news/publications/music-education-and-academic-

achievement-london-music-masters-bridge-project

7© Pro Bono Economics