research and public policy evans school of public affairs april 30, 2013 annie pennucci associate...
TRANSCRIPT
Research and Public Policy
Evans School of Public Affairs
April 30, 2013
Annie Pennucci
Associate DirectorWashington State Institute for Public Policy
www.wsipp.wa.gov
UpcomingExamplesThe ResearchBackground
Washington State Institute for Public PolicyWashington State Institute for Public Policy
Created by the 1983 Legislature
Mission: carry out non–partisan research on projects assigned by the Legislature or the Institute’s Board of Directors
Senator Mike CarrellSenator Karen FraserSenator Jeanne Kohl-Welles Representative Cary CondottaHouse Republican VacancyHouse Democratic VacancyKen Conte, House StaffRichard Rodger, Senate Staff
David Schumacher, OFM Director Gubernatorial Appointee VacancySandra Archibald, Univ. of WALes Purce, The Evergreen State Col.Robert Rosenman, WA State Univ.Rodolfo Arévalo, Eastern WA Univ.
Board of DirectorsSenator Mark Schoesler, Co-Chair
House Democratic Vacancy, Co-Chair
Slide 2 of 22
UpcomingExamplesThe ResearchBackground
WSIPP
Capitol
Washington State Institute for Public Policy
Slide 3 of 22
UpcomingExamplesThe ResearchBackground
1983 legislative intent
Apply expertise of academia to needs of state policymakers
Connect The Evergreen State College (based in Olympia) with state government
Goal: better informed policy making
Washington State Institute for Public Policy
Slide 4 of 22
UpcomingExamplesThe ResearchBackground
Current role
Applied research
Legislature assigns and funds most studies
Reports are easy to understand and accessible to policymakers
Non-partisan staff
Washington State Institute for Public Policy
Slide 5 of 22
UpcomingExamplesBackground The Research
Three Types of Legislative Assignments to WSIPP
1. What Works? Benefit-Cost Analysis (research reviews)
2. Outcome evaluations of specific Washington programs (primary research)
3. Miscellaneous
Are there evidence-based policy options that improve public outcomes, at less cost?
Slide 6 of 22
UpcomingExamplesBackground The Research
We systematically analyze all, credible evaluations of real world ways to improve key public outcomes.
1a. What Works? What Doesn’t?
CrimeEducation, Early Ed. Child Abuse & Neglect Substance AbuseMental HealthDevelopmental Disabilities
Teen BirthsEmploymentPublic AssistancePublic HealthHousing
What does the weight of the evidence tell us?
Slide 7 of 22
UpcomingExamplesBackground The Research
1b. What’s Cost-Beneficial?
For programs and policies determined to “work”
How much does it cost?
What are the long-term benefits in monetary/fiscal terms?
Which are the best investments from the state’s perspective?
What is the risk of success/failure?
Slide 8 of 22
UpcomingExamplesBackground The Research
2. Outcome Evaluations
Do Washington programs & policies achieve their intended effects?
Comparison group research
Administrative and survey data
Outcomes-focused
Slide 9 of 22
UpcomingExamplesBackground The Research
3. Miscellaneous
Often qualitative (e.g., describe a program or process or survey stakeholders regarding their views)
Review other states’ policies in comparison with Washington’s
Key Institute role = independence, objectivity
“Other duties as assigned”
Slide 10 of 22
UpcomingThe ResearchBackground Examples
Slide 11 of 22
Example #1: How Does Class Size Impact Student Outcomes?
Report to the 2013 Legislature
Test scores and high school graduation
Literature review (meta-analysis)
• Supplemented by WSIPP analysis of state-level data
Cost estimation
Risk analysis
UpcomingThe ResearchBackground Examples
Slide 12 of 22
How to find studies:
Keywords in combination (class size, evaluation, K-12, outcomes, public schools, effective, and so on)
Databases (ProQuest and other academic journal sources, Google Scholar)
Previously published reviews/meta-analyses. Comb reference lists.
Organization databases (e.g., the Campbell Collaboration, What Works Clearinghouse)
Literature Search
UpcomingThe ResearchBackground Examples
Slide 13 of 22
Literature Review
Screening studies:
Is the study an evaluation or descriptive?
Does the evaluation address the research question?
Is there a valid comparison group?
What outcomes are measured?
UpcomingThe ResearchBackground Examples
Slide 14 of 22
An effect size (or in the case of class size, an “elasticity”)
Represents the difference in outcomes between the experimental and comparison groups
Tells us how much change we might expect in an outcome if a program or policy is implemented
Must be interpreted within the context of the policy environment
Meta-analysis
UpcomingThe ResearchBackground Examples
Slide 15 of 22
0 1 2 3 4 5 6 7 8 9 10 11 12-0.5%
+0.0%
+0.5%
+1.0%
+1.5%
+2.0%Individual StudiesInstitute Summary
School Grade Where Class Size Is Reduced
Ch
ang
e in
Ou
tco
mes
*The measured outcomes include student test score gains, high school graduation, and dropout rates.The chart plots the 77 credible effects we found in the research literature. Our summary line is a weighted average.
How a 10% Decrease in Class SizeAffects Student Outcomes*
UpcomingThe ResearchBackground Examples
Slide 16 of 22
Per-Student Benefits and Costs of Reducing Class Size by One Student
Grade Costs Avg. Benefits Avg. NetValue
K $198 $2,302 $2,1041 $198 $1,218 $1,0212 $198 $725 $5283 $198 $578 $3814 $179 $422 $2435 $179 $366 $1876 $179 $347 $1687 $162 $358 $1968 $162 $336 $1759 $160 $306 $146
10 $160 $301 $14111 $160 $378 $21812 $160 $353 $193
UpcomingThe ResearchBackground Examples
Slide 17 of 22
Chance that benefits outweigh costs
0%
50%
100%
K 1 2 3 4 5 6 7 8 9 10 11 12Grade Where Class Size Is Reduced
1st grade: 94%
Close to 50%
Economics & Risk: Reducing Class Size by One Student
UpcomingThe ResearchBackground Examples
Slide 18 of 22
Benefits Per Individual
Crime reduction $5,485 Lower CJ / victim costs
Main Source of Benefits
Test scores & graduation rates $13,266 Increased earnings
Special education reduction $1,081 Lower K-12 costs
Lower K-12 costsGrade repetition reduction $307
Lower CW / victim costs Fewer out-of-home placements $557 Lower health care system costs $756 Lower Medicaid enroll.
Total Benefits Per Individual $22,693
Cost Per Individual
Benefits Per Dollar of Cost
$7,489
$3.06
Nearly 100% chance
of benefits exceeding costs
Less child abuse & neglect $1,240 Lower CW / victim costs
Example #2: Early Childhood Education (low income)2011 dollars
UpcomingThe ResearchBackground Examples
Slide 19 of 22
WSIPP “Consumer Reports” ListsEvidence-based policy options ranked by return on investment
ExampleThe ResearchBackground Upcoming
Slide 20 of 22
Current Legislative Proposals
In the 2013 session:
Reduce class size in K-3 (or K-1)
Expand early childhood education for low-income children by 10%
Assign WSIPP additional reviews to identify evidence-based options for K-12 education funding increases
ExampleThe ResearchBackground Upcoming
Slide 21 of 22
Current WSIPP Projects
Innovative K-12 Public Schools in Washington State
State Need Grant (Student Outcomes)
Outcomes of Juveniles Tried in Adult Courts
I-502 Evaluation: Costs and Benefits of the Legalization of Cannabis in Washington State
Impacts on Crime from Different Approaches to Policing
ExampleThe ResearchBackground Upcoming
Slide 22 of 22
Potential* Upcoming WSIPP Projects
How can the state control Medicaid costs?
Which components of early childhood education programs are evidence-based?
Develop a risk assessment tool for individuals involuntarily committed for mental illness
Compile an inventory of evidence-based approaches to substance abuse intervention
*Depends on 2013 legislative session