Theories of change

Session 3PMAP 8521: Program evaluation

Andrew Young School of Policy Studies1 / 26

Plan for today

Reproducibility

Program theories

Logic models & results chains

2 / 26

Reproducibility

3 / 26

Why am I making you learn R? 

More powerful

Free and open source

Reproducibility

4 / 26

Debt:GDP ratio 90%+ → −0.1% growth Paul Ryan's 2013 House budget resolution

Austerity and Excel

5 / 26

Thomas Herndon From Paul Krugman, "The Excel Depression"

Austerity and Excel

6 / 26

Austerity and Excel

Debt:GDP ratio = 90%+ → 2.2% growth (!!)7 / 26

Septin 2 Membrane-Associated RingFinger (C3HC4) 1

2310009E13

Genes and Excel

20% of genetics papers between 2005–2015 (!!!)8 / 26

General guidelines

Don't touch the raw dataIf you do, explain what you did!

Use self-documenting, reproducible codeR Markdown!

Use open formatsUse .csv, not .xlsx

9 / 26

Airbnb, ggplot, and rmarkdown

 

The UK's reproducible analysis pipeline

R Markdown in real life

10 / 26

Program theories

11 / 26

InputsThings that go into an activity; money, people,

time, etc.

ActivitiesActions that convert inputs to outputs; things

that the program does

OutputsTangible goods and services produced by

activities; you have control over these

OutcomesWhat happens when the target population usesthe outputs; you don't have control over these

Elements of a program

Inputs → Activities → Outputs → Outcomes

12 / 26

Program theory

How and why an intervention causes change

A sequence of events that connects inputs to activities to outputs to outcomes

Why you think inputs → activities → outputs → outcomes

13 / 26

Impact theory

Causes (activities) linked to effects (outcomes)

14 / 26

One Laptop Per Child (OLPC)

15 / 26

Nov 16, 2005: The One Laptop perNov 16, 2005: The One Laptop per……

Adi Robertson, "OLPC's $100 laptop was going to change the world—then it all went wrong"

OLPC 

16 / 26

PlayPumps

17 / 26

Implicit theoryWhat program designers think or

assume is going to happen, and why

Assumed

Articulated theoryWhat program designers officially

claim and predict, and why

Written down

Why theorize?

Should all social programs be rooted in explicit articulated theory?

18 / 26

19 / 26

20 / 26

Impact theoryEnsure that the theory linking activities to the outcomes is sound!

21 / 26

Logic models & results chains

22 / 26

23 / 26

24 / 26

 

Impact theory vs. logic model

25 / 26

MPA/MPP at GSU

26 / 26