prof. eric bettinger stanford university, nber postsecondary remediation: how can we serve the needs...
TRANSCRIPT
PROF. ERIC BETTINGERSTANFORD UNIVERSITY, NBER
Postsecondary Remediation:Postsecondary Remediation:
How can we serve the needs of How can we serve the needs of Underprepared Students?Underprepared Students?
The Academic Gatekeeper: Remedial & Developmental
Postsecondary Courses2
Lack of academic preparation is a significant barrier to college success – How should we address this problem?
Nationally approx. 40% of 1st year students are placed into college remediation (55-60% at CCs)
The bulk of remediation is provided at non-selective publics colleges & universities, the point of entry for 80% of 4yr students and 99% of 2yr students
The remediation placement exam taken once arriving on campus has become the key gatekeeper to a college-level education (often a surprise to students)
3
CUNY phased remediation out of its 4yrs (1999)“CUNY university system currently devotes far too much money and effort to teaching skills that students should have learned in high school.”
Debates about Remediation
CSU: Fall 2001 “kicked out more than 2,200 students – nearly 7% of the freshman class – for failing to master basic English and math skills.”
Proponents: opportunities for underprepared, many of whom did not have the chance to take in HS
Critics: Provides disincentives for high school students; Double dipping; Not the appropriate place
$1 Billion each year at publics (conservative estimate)
The Continuing Debates about Remediation…
4
CT and AZ do not allow remediation at publics
8 states, incl. FL and IL, restrict remediation to 2yrs
VA: Some High Schools “guarantee” their diplomas – pay remedial expenses of their students
FL: Legislature elected to have students pay the full (unsubsidized) cost of their remediation (4 times the regular tuition)
Limits on funding of remediation: CA, TX, TN, & UT
Difficulties in Studying Remediation5
Comparisons are Different. How do you compare remedial to non-remedial students? Historically, researchers focused on descriptive
studies often without looking at college outcomes.
Policies are Often Inconsistent True over time and true across schools
Heterogeneity in Remediation Among students Among class offerings
Lack of Data
Why Don’t Raw Comparisons Work?6
Dramatic Differences Between Remedial and Non-Remedial Students
Average ACT Scores Ohio 4-Year Colleges Math: 23.3 for Non-Remedial , 17.4 for Remedial English: 22.8 vs. 15.8 Reading: 23.8 vs. 16.9
Average HS GPA Math: 3.3 vs. 2.5; English: 3.4 vs. 2.8
5-year Dropout Rate 31 Percent for Non-Remedial; 66 Percent for Math
Remediation; 67 Percent for English Remediation
How Inconsistent are the Policies?7
Ohio Non-Selective Colleges
SELECT IVE UNIVERSIT IES
Freq
uenc
y
ACT Math Score10 15 20 25 30 35 36
0
.120192
Diff
ere
nce
in L
og
from
Max
ACT Math Score10 15 20 25 30 35 36
0
100
NON-SELECT IVE AND UB UNIVERSIT IES
Freq
uenc
y
ACT Math Score10 15 20 25 30 35 36
0
.226811
Diff
ere
nce
in L
og
from
Max
ACT Math Score10 15 20 25 30 35 36
0
100
Ohio Non-Selective Colleges
How Inconsistent are the Policies?8
Ohio Non-Selective Colleges
SELECT IVE UNIVERSIT IES
Freq
uenc
y
ACT Math Score10 15 20 25 30 35 36
0
.120192
Diff
ere
nce
in L
og
from
Max
ACT Math Score10 15 20 25 30 35 36
0
100
NON-SELECT IVE AND UB UNIVERSIT IES
Freq
uenc
y
ACT Math Score10 15 20 25 30 35 36
0
.226811
Diff
ere
nce
in L
og
from
Max
ACT Math Score10 15 20 25 30 35 36
0
100
Ohio Non-Selective Colleges
How Inconsistent are the Policies?9
Ohio 2-Year Colleges
COMMUNIT Y COLLEGES
Fre
qu
en
cy
ACT Math Score10 15 20 25 30 35 36
0
.256623
Dif
fere
nce
in
Lo
g f
rom
Ma
x
ACT Math Score10 15 20 25 30 35 36
0
100
How Heterogeneous are the Students?
10
0.0
05.0
1.0
15.0
2.0
25D
ens
ity
0 20 40 60 80National Percentile
kernel = epanechnikov, bandwidth = 3.3164
Kernel density estimate
Heterogeneity Among Students11
Recommended CurriculaBottom Half:
practice and apply estimation and computation using whole numbers and decimals
choose the appropriate method of computation to solve multistep problems (e.g., calculator, mental, or pencil and paper)
calculate length of a lineTop Half:
solve routine arithmetic problems that involve rates, proportions, and percents
do multistep computations with rational numbers calculate area and perimeter of triangles and rectangles;
use geometric formulas
Heterogeneity in Delivery12
Almost half taught by Adjunct (31%) or Graduate Students (17%)
63% taught by non-tenure track faculty42% taught by faculty with doctorates; 31%
have masters; remaining 27% by other degrees (mostly bachelors)
Lack of Data13
National data has low sample sizesSystem-wide data is important
15 percent “transfer down” 16 percent “transfer up” All of these students would look like dropouts
State Administrative Data to the Rescue
14
System-wide data from Florida, Texas, and Ohio have helped develop new strategies
Identify “bubble” students Discontinuities Similar students across campuses
15
Florida Administrative Dataset Sample: All first-time, degree-seeking CC
students who began in Fall 1997, 1998, 1999, or 2000 (over 100,000 records)
Term-by-term transcript data through Spr 2006
All test scores (CPT, SAT, ACT) plus other controls
Outcomes: passing Algebra/English 101, retention, certificate, associate degree, transfer to state 4-year college, credits earned (remedial and non-remedial)
RD Design: The IntuitionBeth and Becky are observationally
similar
Compare the outcomes of Beth & BeckyCompare the outcomes of Beth & Becky
Beth and Becky both take the College Placement Test (CPT)
Beth scores just above the cut-off score
Becky scores just below the cut-off score
Beth to college-level courses Becky to remediation
Crossover: Beth takes
remediation anyway
No Show: Becky never
enrolls in remediation
RD Design: The IntuitionBeth and Becky are observationally
similar
Compare the outcomes of Beth & BeckyCompare the outcomes of Beth & Becky
Beth and Becky both take the College Placement Test (CPT)
Beth scores just above the cut-off score
Becky scores just below the cut-off score
Beth to college-level courses Becky to remediation
Endogenous Sorting:
Beth retests to place out of remediation
Crossover: Beth takes
remediation anyway
No Show: Becky never
enrolls in remediation
18
Endogenous Sorting around the Cutoff
Density of Reading CPT for Institution E
0.0
1.0
2.0
3.0
4
-50 -40 -30 -20 -10 0 10 20 30
Expect to see larger number who barely exceed the cutoff than those who barely failed
discontinuity of the conditional density at the threshold
19
Passing First College-Level Course
(negative)
0.2
.4.6
.8
-50 -40 -30 -20 -10 0 10 20 30CPT Score Relative to Reading Cutoff
Estimated Discontinuity = -0.066(0.008)
Passing First College-Level Course
0.2
.4
-50 -40 -30 -20 -10 0 10 20 30CPT Score Relative to Reading Cutoff
Estimated Discontinuity = -0.025(0.004)
2 yr Degree Completion
3035
4045
-50 -40 -30 -20 -10 0 10 20 30CPT Score Relative to Reading Cutoff
Estimated Discontinuity = 1.527(0.447)
Total Credits Earned
0.2
.4.6
.8
-50 -40 -30 -20 -10 0 10 20 30CPT Score Relative to Reading Cutoff
Estimated Discontinuity = -0.009(0.008)
Fall-to-Fall Retention
0.2
.4
-50 -40 -30 -20 -10 0 10 20 30CPT Score Relative to Reading Cutoff
Estimated Discontinuity = -0.016(0.004)
Transfer to 4 yr
2025
3035
40
-50 -40 -30 -20 -10 0 10 20 30CPT Score Relative to Reading Cutoff
Estimated Discontinuity = -1.751(0.467)
Total College-Level Credits Earned
Figure 4: Outcome by Reading CPT Score and Estimated Discontinuity
20
Total Credits Earned(positive)
0.2
.4.6
.8
-50 -40 -30 -20 -10 0 10 20 30CPT Score Relative to Reading Cutoff
Estimated Discontinuity = -0.066(0.008)
Passing First College-Level Course
0.2
.4
-50 -40 -30 -20 -10 0 10 20 30CPT Score Relative to Reading Cutoff
Estimated Discontinuity = -0.025(0.004)
2 yr Degree Completion
3035
4045
-50 -40 -30 -20 -10 0 10 20 30CPT Score Relative to Reading Cutoff
Estimated Discontinuity = 1.527(0.447)
Total Credits Earned
0.2
.4.6
.8
-50 -40 -30 -20 -10 0 10 20 30CPT Score Relative to Reading Cutoff
Estimated Discontinuity = -0.009(0.008)
Fall-to-Fall Retention
0.2
.4
-50 -40 -30 -20 -10 0 10 20 30CPT Score Relative to Reading Cutoff
Estimated Discontinuity = -0.016(0.004)
Transfer to 4 yr
2025
3035
40
-50 -40 -30 -20 -10 0 10 20 30CPT Score Relative to Reading Cutoff
Estimated Discontinuity = -1.751(0.467)
Total College-Level Credits Earned
Figure 4: Outcome by Reading CPT Score and Estimated Discontinuity
The Florida Remediation StudyConclusions – Overall Sample
21
Being assigned to remediation appears to increase the total number of credits completed for students on the margin of passing out of the requirement…
But it does not increase the completion of college-level credits or eventual degree completion.
Remediation might promote early persistence in college, but it does not necessarily help students on the margin of passing the cutoff to make long-term progress toward a degree
How do these results compare to other studies?
Texas Study
22
State with a single cutoffs and placement exam RD methodology similar to Florida study
Sample: students who took all three placement exams (math, reading, and writing) and passed the writing section
Remediation appears to have little effect on a wide range of educational and labor market outcomes. The estimates are small and statistically insignificant but suggest that students are neither harmed nor greatly benefited by remediation
The Ohio Study23
Nearly 66,000 first-time freshman in Fall 1998 (FT, traditional age, 4yr degree intent) for 6 years
Compares observationally-similar students: one placed into remediation because his nearby college has a stringent policy while the other student does not because his school has a lax policy
Students in remediation had better subsequent outcomes – Reduced the likelihood of dropping out and increases the likelihood of completing a degree
Discouragement effect from certain Majors
Reconciling the Results?24
Single placement exam and cutoff versus autonomy
Different locations of the cutoff (where should it be?)
Different student samples (all versus traditional-age, degree seeking) and institutions (only CCs vs. 2yrs and 4yrs)
Could the effects of remediation differ by type of student?
Does remediation work for students with far less preparation?
25
The FL, OH and TX studies focus on students on the margin of needing remediation -- they do not investigate the effects of remediation on students who are extremely under-prepared (i.e. don’t have an appropriate control group)
The Tennessee Case Remediation at two-years and four-years
Multiple cutoffs and changes in placement policy over time
Can investigate the effects of different types of remediation for students of different abilities
26
Policy Implications and Remaining Questions
Costs of remediation should be given careful consideration in light of the limited benefits.
Explore noncompliance and retesting practices and consider potential consequences
What is the best way to offer remediation? Characteristics of strong remedial programs?
What are the effects of limitations states impose on remedial course-taking (e.g., only at CCs, time limits)?
Early Placement Testing – a preventative measure?