Education Evaluation: Where Research and Policy Intersect

Jonathan Plucker, Director

April 8, 2009

2

Topics for Discussion

• Brief history of CEEP

• Some Recent Evaluation Projects

• Lessons Learned

3

CEEP History

• Formed in January 2004 by merging two smaller centers– Indiana Education Policy Center– Indiana Center for Evaluation

• Longest year and a half of my professional life

4

The Center Mission is to …

• Improve education by providing nonpartisan information, research, and evaluation on education issues to policymakers and other education stakeholders.

• Encourage rigorous program evaluation across a variety of settings by providing evaluation expertise and services to diverse agencies, organizations, and businesses.

• Expand knowledge of effective strategies in evaluation and policy research by developing, modeling, and disseminating innovative approaches to program evaluation and policy research.

5

Current Capabilities

• People– ~100 people, 70 FTE (+ “highly involved” faculty)

– 25-30 graduate assistantships, dozens hourly

• Organization– Director, 2 assoc directors, asst director

• Facilities– Parts of three floors in Eigenmann

6

Externally-Funded Projects

32

39

47 49

62

68

0

10

20

30

40

50

60

70

FY04

FY05

FY06

FY07

FY08

FY09

7

Current and Recent Activities

• KY and IN 21st CCLC projects• Lev Tech• US ED Technical Asssitance• Alternative school evals and tech assistance

– IDOE & GADOE charters, SyF ERC evaluation, alternative education, Cleveland voucher study, homeschooling

• IES evaluation

Some Current Projects

9

SYF ERC Evaluation

• SYF facilitates alternative high schools

• Comprehensive evaluation of most ERCs

• Site visits, analysis of existing data

• Report to BoD well-received, recommendations generally followed

• Bumpy ride at times …

10

Georgia Charter Evaluation

• Longshot RFP

• Great kickoff meeting …

• … but no follow-through

• Yearlong project compressed to a month

• Analysis of existing data

• Many GADOE staff changes

• Directly led to legislative changes

11

Indiana Charter Evaluation

• Read about it in state budget

• Bumpy kickoff …– Warning from professional association

• 2-year project compressed to 4 months

• Analysis of existing data

• Both foes and advocates deliberately distorted results …– … but that was expected (to a point)

12

SOE Principal Evaluation

• Study of principal assessments of SOE teacher education graduates

• Follow-up to graduate survey

• Phone, e-mail, and on-line components

• Internal politics considerable but manageable– However, they complicate the methodological

challenges

What Have We Learned?

1. Evaluation is so much messier than you can possibly imagine …

15

How Research and Policy are Related (in reality)

Previous governor, with record surplus, introduces FDK bill with state supt support

Ed Roundtablesupports FDK in

P-16 plan

Efforts failR gov candidatesupports FDK

Time passes(1999-2003)

D govproposes FDK

Everyonesupports FDK!

Good news, right?

Record deficit

Public opinion 45-45

FDK and fundingSeparated in D

controlled House

House ways andmeans hearing

House educationhearing

Full House votes

Action moves to heavily R senate

Big, closely contestedelection later in year

Several IUfaculty testify

I testify again(ouch)

D gov and R IDOEget it worse Committee sends

funding tostudy committee

Education billsfrom both

houses get killed

Results:•Big, closely contested election•Record deficit•PO: 60+ to 40-•Lots of angry people•Several good, stalled bills•No FDK

Conf commto reconcile S and

H actions

Webcamgate

House Ds aren’tvery healthy

FDK passes,funding bill doesn’t

Lots of mediacoverage

Public opinion60-40

Senateeducationhearing

Everyonesupports FDK!

Good news,right?

Things get ugly(or uglier)

A few education groupsdecide to oppose billfor financial reasons

2. … but so much more important.

3. Ethics are a constant concern.

60+ grants and contracts + 20+ self-funded projects + ~100 people + dozens of clients + clients’ internal and external political issues = lots of ethical quandries

4. Faculty, esp. senior faculty, are hard to get on board.

I don’t blame them at all: Empire building, lack of credit, promotion concerns, etc.

5. Making valuable intellectual contributions in the current context

is very difficult …

… but hardly impossible.

6. Life would be simpler yetmore exciting if everyone

used logic models