2009 sheeo/nces network conference and ipeds workshop may 21, 2009 j. keith brown
DESCRIPTION
A Better Way to Measure Community College Performance: An Achieving the Dream Cross-State Data Initiative. 2009 SHEEO/NCES Network Conference and IPEDS Workshop May 21, 2009 J. Keith Brown North Carolina Community College System. Presentation Summary :. - PowerPoint PPT PresentationTRANSCRIPT
Success is what counts.Success is what counts.
A Better Way to Measure Community College Performance: An Achieving the Dream Cross-State Data Initiative
2009 SHEEO/NCES Network Conference and IPEDS WorkshopMay 21, 2009
J. Keith BrownNorth Carolina Community College System
Success is what counts.
Presentation Summary: Overview of Achieving the Dream and the Cross-
State Data Workgroup
Recommendations for an Alternative Set of Measures for Community College Performance
Introduction of Intermediate Milestones and Final Measures of Student Performance
Implications for Policy and Practice
Success is what counts.
Achieving the Dream Overview
National initiative to help more community college students succeed (earn degrees, earn certificates, or transfer)
Particularly concerned about student groups that have faced the most significant barriers to success, including low-income students and students of color
Success is what counts.
82 Institutions in 15 StatesAR, CT, FL, HI, MA, MI, NC, NM, OH, OK, PA, SC, TX, VA, WA
Success is what counts.
Achieving the Dream Values
Student-centered
Equity and excellence
Culture of evidence, inquiry, accountability, and shared responsibility
Success is what counts.
Cross-State Data Workgroup
Initial StatesConnecticutFloridaNorth CarolinaOhioTexasVirginia
States JoiningArkansasMassachusettsNew MexicoOklahomaSouth CarolinaWashington
Success is what counts.
Data Workgroup Goals
Develop a set of indicators to:
More effectively track student performance
Evaluate the effectiveness of interventions
Learn from the strengths of other community college systems
Success is what counts.Success is what counts.
Test Drive: Six States Pilot Better Ways to Measure and Compare Community College Performance
Success is what counts.
Recommended Changes:Prior EnrollmentCurrent Federal Method ATD Work Group Method
First-time-in-college students only Same as federal method
Intent at time of EnrollmentCurrent Federal Method ATD Work Group Method
Only students seeking a certificate or degree
Same as federal method
Enrollment StatusCurrent Federal Method ATD Work Group Method
Full-time students only Full-time and part-time students
Success is what counts.
TimeframeCurrent Federal Method ATD Work Group Method
Three years (150% of “normal time” to completion)
Six years
Recommended Changes:
Success Outcomes
Current Federal Method ATD Work Group Method
Earned degree or certificate - Earned degree or certificate- Transferred without an award- Enrolled in year 6 with at least 30 hrs
Success is what counts.
Tracking transfer students within the 2-year sector
Current Federal Method ATD Work Group Method
Reporting is based on individual college; doesn’t track outcomes of students who transfer to another college; simply reported as transferred out
Reporting is based on statewide community college system; tracks outcomes within the system (therefore across colleges)
Recommended Changes:
Success is what counts.
Controlling for factors associated with success
Current Federal Method ATD Work Group Method
Part-time students excluded from analysis; no disaggregation of results by age at initial enrollment
Disaggregated results by part-time and full-time status and age at initial enrollment
Recommended Changes:
Success is what counts.
Results of Cross-State Comparison
Success is what counts.
AGE DISTRIBUTION OF ENROLLMENT
74% 74%
51%
75%83%
76%
23% 23%
43%
22%15%
22%
2% 3% 7% 2% 2% 2%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Connecticut Florida North Carolina Ohio Texas Virginia
PERC
ENT
SHAR
E
22< 23-45 46>
Results of Cross-State Comparison
Success is what counts.
GENDER DISTRIBUTION OF ENROLLMENT
45% 46% 44% 47% 46% 44%
55% 54% 56% 53% 54% 56%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Connecticut Florida North Carolina Ohio Texas Virginia
PERC
ENT
SHAR
E
Male Female
Results of Cross-State Comparison
Success is what counts.
Results of Cross-State Comparison
Success is what counts.
Distribution of Awards and Transfer w/ o Award Outcomes (all students)
1%
6%
10%
2%5%
3%
9%
22%
16%
23%
12%
19%
14%
7%
14%
6%
25%
13%
0%
10%
20%
30%
40%
50%
CT FL NC OH TX VA
Award of Less Than AA Degree Associate's Degree Transfer w/ o Award
Results of Cross-State Comparison
Success is what counts.
Students Who Earned an Award or Transferred OR Who Are Still Enrolled with 30+ Credits (all students)
23%36% 40%
30%42%
35%
9%
8%8% 9%
3%5%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
CT FL NC OH TX VA
Total Earned Award or Transfer Still Enrolled w/o Award with 30+ Credits
Results of Cross-State Comparison
Success is what counts.CT FL NC OH TX VA0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
25%
39% 40%35%
45%38%
10%
8%10%
5%
10%
4%
Younger Students (> or = 22) Who Earned an Award or Transferred and Who Are Still Enrolled with 30+ Credits
Total Earned Award or Transfer Still Enrolled w/o Award with 30+ Credits
Perc
enta
ge
Results of Cross-State Comparison
Success is what counts.
PERCENTAGE OF STUDENTS EARNING AN AWARD WHO SUBSEQUENTLY TRANSFER TO A 4-YEAR INSTITUTION
(FULL-TIME STUDENTS)
17%12%
5% 7% 7%
15%
47%
72%
36%
54% 54%
22%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Connecticut Florida NorthCarolina
Ohio Texas Virginia
PERC
ENTA
GE
% of Certificate Earners % of Associate's Degree Earners
Results of Cross-State Comparison
Success is what counts.
PERCENTAGE OF STUDENTS EARNING AN AWARD WHO SUBSEQUENTLY TRANSFER TO A 4-YEAR INSTITUTION
(PART-TIME STUDENTS)
8% 8% 6% 7% 4% 3%
34%
59%
29%23%
43% 43%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Connecticut Florida NorthCarolina
Ohio Texas Virginia
PERC
ENTA
GE
% of Certificate Earners % of Associate's Degree Earners
Results of Cross-State Comparison
Success is what counts.Success is what counts.
Next Steps: Developing Intermediate Benchmarks to Measure Student Progress
Success is what counts.
Intermediate milestones to track studentsFirst-Year Milestones• Persisted fall to spring• Passed 80% or more of attempted hours• Earned 24 or more hours
Second-Year Milestones• Persisted fall to fall• Completed developmental math by year 2• Earned 48 or more hours
Third-Year Milestones• Passed gatekeeper English or higher by year 3• Passed gatekeeper math or higher by year 3
Success is what counts.
First-Year Milestones
Returned spring semester 16% increase in final success outcomes for
full-time students; 28% for part-time students
Earned 24 credits (full-time) or 18 credits (part-time) by the end of the first year 25% increase in final success outcomes for
full-time students; 66% for part-time students
Passed 80% of credits attempted 12% increase in final success outcomes for
full-time students; 46% for part-time students
Success is what counts.
Second-Year Milestones
Returned in fall of second year 23% increase in final success outcomes for full-
time students; 53% for part-time students
Returned and earned 42 credits (full-time) or 24 credits (part-time) by the end of the second year 32% increase in final success outcomes for full-
time students; 49% increase for part-time students
Success is what counts.
Second-Year Milestones
Passed developmental mathematics course by the end of the second year 84% increase in final success outcomes for full-
time students; 110% increase for part-time students
Passed developmental English course by the end of the third year 17% increase in final success outcomes for full-
time students; 39% increase for part-time students
Success is what counts.
Third-Year Milestones
Passed “gatekeeper” mathematics course by the end of the third year 45% increase in final success outcomes for full-
time students; 147% increase for part-time students
Passed “gatekeeper” English course by the end of the third year 17% increase in final success outcomes for full-
time students; 59% increase for part-time students
Success is what counts.
Tracking toward final success measuresFourth- and Sixth-Year Measures• Award of less than associate’s degree w/o transfer
• Award of associate’s degree or higher w/o transfer
• Award of less than associate’s degree and transferred
• Award of associate’s degree or higher and transferred
• Transferred w/o an award
• Still enrolled with 30 or more college hours
• Total success rate
Success is what counts.
Next Steps…finishing data runs• Run benchmarks at state and institutional
levels• Disaggregate and analyze performance by:o academic readinesso incomeo ethnicityo gender
• Identify and document promising interventions
Success is what counts.Success is what counts.
What does all this mean and what are the policy implications?
Success is what counts.
Context of the Data Reflects the mission of colleges/state
systems Examples: age distribution, award
distribution Reflects differences in state
priorities/policies Example: college transfer policy
Not all measures pertain to all students
Example: gatekeeper math course
Success is what counts.
AGE DISTRIBUTION OF ENROLLMENT
74% 74%
51%
75%83%
76%
23% 23%
43%
22%15%
22%
2% 3% 7% 2% 2% 2%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Connecticut Florida North Carolina Ohio Texas Virginia
PERC
ENT
SHAR
E
22< 23-45 46>
Success is what counts.
Distribution of Awards and Transfer w/ o Award Outcomes (all students)
1%
6%
10%
2%5%
3%
9%
22%
16%
23%
12%
19%
14%
7%
14%
6%
25%
13%
0%
10%
20%
30%
40%
50%
CT FL NC OH TX VA
Award of Less Than AA Degree Associate's Degree Transfer w/ o Award
Success is what counts.
Context of the Data Reflects the mission of colleges/state
systems Examples: age distribution, award
distribution Reflects differences in state
priorities/policies Example: college transfer policy
Not all measures pertain to all students
Example: gatekeeper math course
Success is what counts.
Impact of state policy on outcomesDifferences in transfer patterns reflect policy differences
o Encouraging transfer after earning a degree• e.g. FL – 69% transfer after degree & 7% before
o Encouraging transfer without a degree• e.g. TX – 25% transfer without degree
o Absence of strong transfer policies• e.g. OH – 22% transfer after degree & 6% before
o Balanced approach to transfer• e.g. NC – 16% transfer after degree & 14% before
Success is what counts.
Context of the Data Reflects the mission of colleges/state
systems Examples: age distribution, award
distribution Reflects differences in state
priorities/policies Example: college transfer policy
Not all measures pertain to all students
Example: gatekeeper math course
Success is what counts.
Implications of the Data: College Perspective Identification of at-risk students Student advising Review of policies/practices
Examples: Course taking sequence/timing Drop/add policies
Success is what counts.
Implications of the Data: State/System Perspective Policy development to improve student
success Review of regulations Performance indicators Development/refinement of student
database Benchmarking
Success is what counts.
Implications of the Data: National Perspective More appropriate measure of student
success: accountability Financial aid policy Expand the body of knowledge on
successful community colleges/practices
Success is what counts.Success is what counts.
Acknowledgements
The following individuals are gratefully acknowledged for their contributions to this presentation:
Chris Baldwin: Jobs for the FuturePat Windham: FloridaDonna Jovanovich: VirginiaCorby Coperthwaite: Connecticut
Success is what counts.Success is what counts.
Achieving the DreamSuccess is what counts.ww.achievingthedream.org