the potential of transcript-based placement
DESCRIPTION
The Potential of Transcript-Based Placement . Terrence Willett, RP Group. CCCCIO 2013 Spring Conference, San Francisco April 4, 2013. A brief overview. The Student Transcript-Enhanced Placement Project (STEPS). Pride. Time. Money. Validation. What Are We Trying to Do?. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: The Potential of Transcript-Based Placement](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816241550346895dd27b57/html5/thumbnails/1.jpg)
Terrence Willett, RP Group
The Potential of Transcript-Based Placement
CCCCIO 2013 Spring Conference, San FranciscoApril 4, 2013
![Page 2: The Potential of Transcript-Based Placement](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816241550346895dd27b57/html5/thumbnails/2.jpg)
2
A BRIEF OVERVIEW
The Student Transcript-Enhanced Placement Project (STEPS)
The Potential of Transcript-Based Placement: CCCCIO, April 2013
![Page 3: The Potential of Transcript-Based Placement](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816241550346895dd27b57/html5/thumbnails/3.jpg)
3
𝑦= 𝑓 (𝑥 )The Potential of Transcript-Based Placement: CCCCIO, April 2013
![Page 4: The Potential of Transcript-Based Placement](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816241550346895dd27b57/html5/thumbnails/4.jpg)
4
Pride
The Potential of Transcript-Based Placement: CCCCIO, April 2013
![Page 5: The Potential of Transcript-Based Placement](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816241550346895dd27b57/html5/thumbnails/5.jpg)
5
Time
The Potential of Transcript-Based Placement: CCCCIO, April 2013
![Page 6: The Potential of Transcript-Based Placement](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816241550346895dd27b57/html5/thumbnails/6.jpg)
6
Money
The Potential of Transcript-Based Placement: CCCCIO, April 2013
![Page 7: The Potential of Transcript-Based Placement](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816241550346895dd27b57/html5/thumbnails/7.jpg)
7
Validation
The Potential of Transcript-Based Placement: CCCCIO, April 2013
![Page 8: The Potential of Transcript-Based Placement](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816241550346895dd27b57/html5/thumbnails/8.jpg)
8
What Are We Trying to Do?• Examine the value of using transcripts as part of the
assessment process• Funded by the Walter S. Johnson and Gilbert Foundations
with support from the California Partnership for Achieving Student Success (Cal-PASS)
• Create predictive models using Cal-PASS data to study students who had already taken community college courses
• Use models to analyze how well transcript data predicts the first English & math courses students take and how well they do in them
• Recruit colleges to do local analyses to generate campus-specific insights and trigger CCC/K-12 conversations
The Potential of Transcript-Based Placement: CCCCIO, April 2013
![Page 9: The Potential of Transcript-Based Placement](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816241550346895dd27b57/html5/thumbnails/9.jpg)
9
Related Work• Brown & Conely. (2007). Comparing state high school
assessments to standards for success in entry-level university courses.
• Brown & Niemi. (2007). Investigating the alignment of high school and community college assessments in California.
• Belfield & Crosta. (2012). Predicting Success in College: The Importance of Placement Tests and High School Transcripts.
• Willett, Hayward, & Dahlstrom. (2007). Leveraging the CSTs.• Wurtz. (2008). Relations among high school math performance
and college placements and achievement.• Fuenmayor, Hetts, & Rothstein. (2011). Assessing Assessment:
Evaluating Models of Assessment and Placement.• English Curriculum Alignment Project (ECAP) between
Grossmont College and Grossmont UHSD.
The Potential of Transcript-Based Placement: CCCCIO, April 2013
![Page 10: The Potential of Transcript-Based Placement](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816241550346895dd27b57/html5/thumbnails/10.jpg)
The Potential of Transcript-Based Placement: CCCCIO, April 2013
Source: Willett, Terrence; Hayward, Craig; Dahlstrom, Eden. (2008). An Early Alert System for Remediation Needs of Entering Community College Students: Leveraging the California Standards Test. California Partnership for Achieving Student Success Report 2007036, Encinitas, CA.
![Page 11: The Potential of Transcript-Based Placement](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816241550346895dd27b57/html5/thumbnails/11.jpg)
11The Potential of Transcript-Based Placement: CCCCIO, April 2013
Source: Willett, Terrence; Hayward, Craig; Dahlstrom, Eden. (2008).
![Page 12: The Potential of Transcript-Based Placement](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816241550346895dd27b57/html5/thumbnails/12.jpg)
12
What Is Happening Now?• Missing data weakened the first set of
predictive models (this highlights the value of strengthening data sharing among segments)
• Some colleges are still working on their analyses
• Updated file management module and analysis scripts has been released
• Bottom line: So far, high school performance is partially predictive of college performance
The Potential of Transcript-Based Placement: CCCCIO, April 2013
![Page 13: The Potential of Transcript-Based Placement](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816241550346895dd27b57/html5/thumbnails/13.jpg)
13
Cox & Snell pseudo R-square ~ 0.35
The Potential of Transcript-Based Placement: CCCCIO, April 2013
0123456789
1011
CST's A-GCourses
HSCourseLevel
HSCourseGrade
HS GPA*
Coun
t of C
olle
ges
Show
ing
Sign
ifica
nce
Predictor Variable Category
Predicting College English Level
Weakest
Intermediate
Strongest
![Page 14: The Potential of Transcript-Based Placement](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816241550346895dd27b57/html5/thumbnails/14.jpg)
14
In Englishtests
predict tests,grades and courses
matter but vary by college
![Page 15: The Potential of Transcript-Based Placement](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816241550346895dd27b57/html5/thumbnails/15.jpg)
15
Cox & Snell pseudo R-square ~ 0.50
The Potential of Transcript-Based Placement: CCCCIO, April 2013
0123456789
1011
CST's A-GCourses
HSCourseLevel
HSCourseGrade
HS GPA*
Coun
t of C
olle
ges
Show
ing
Sign
ifica
nce
Predictor Variable Category
Predicting College Math Level
Weakest
Intermediate
Strongest
![Page 16: The Potential of Transcript-Based Placement](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816241550346895dd27b57/html5/thumbnails/16.jpg)
16
In math, thetests
predict testsmost consistently among colleges but high school levelalso strongly predictive
![Page 17: The Potential of Transcript-Based Placement](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816241550346895dd27b57/html5/thumbnails/17.jpg)
17
Cox & Snell pseudo R-square ~ 0.20
The Potential of Transcript-Based Placement: CCCCIO, April 2013
0123456789
1011
CST's A-GCourses
HSCourseLevel
HSCourseGrade
HSGPA*
CollegeCourseLevel
Coun
t of C
olle
ges
Show
ing
Sign
ifica
nce
Predictor Variable Category
Predicting College English Success
Weakest
Intermediate
Strongest
![Page 18: The Potential of Transcript-Based Placement](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816241550346895dd27b57/html5/thumbnails/18.jpg)
18
In English,
gradespredict
grades
![Page 19: The Potential of Transcript-Based Placement](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816241550346895dd27b57/html5/thumbnails/19.jpg)
19
Cox & Snell pseudo R-square ~ 0.20
The Potential of Transcript-Based Placement: CCCCIO, April 2013
0123456789
1011
CST's A-GCourses
HSCourseLevel
HSCourseGrade
HSGPA*
CollegeCourseLevel
Coun
t of C
olle
ges
Show
ing
Sign
ifica
nce
Predictor Variable Category
Predicting College Math Success
Weakest
Intermediate
Strongest
![Page 20: The Potential of Transcript-Based Placement](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816241550346895dd27b57/html5/thumbnails/20.jpg)
20
In math, success
predictors vary by
college
![Page 21: The Potential of Transcript-Based Placement](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816241550346895dd27b57/html5/thumbnails/21.jpg)
21
What’s Going on Here?CST tests for English and math are different• English is less complex to examine as there is
generally only one 11th grade CST test form and most students are taking the same types of classes
• In math, CST tests are based on the specific course you took (e.g., Algebra II, Pre-Calc) and so analyses are more complex
The Potential of Transcript-Based Placement: CCCCIO, April 2013
![Page 22: The Potential of Transcript-Based Placement](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816241550346895dd27b57/html5/thumbnails/22.jpg)
22
What’s Going on Here?Math: It’s a question of articulation
• It’s more straightforward to articulate math than English because there is a clear sequence of courses—the question is whether we are examining the math level that students reached—that’s why high school math level is most important
The Potential of Transcript-Based Placement: CCCCIO, April 2013
𝐴=𝜋 𝑟2
𝑎2+𝑏2=𝑐2
![Page 23: The Potential of Transcript-Based Placement](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816241550346895dd27b57/html5/thumbnails/23.jpg)
23
What’s Going on Here?English: It’s a different question of articulation
• For English, there is a disconnect between the literary emphasis in high school and the expository emphasis in college—non-English high school GPA is more predictive than grades in high school English of college success, which implies that other skills are supporting positive outcomes
The Potential of Transcript-Based Placement: CCCCIO, April 2013
![Page 24: The Potential of Transcript-Based Placement](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816241550346895dd27b57/html5/thumbnails/24.jpg)
24
What Happens Next?• Colleges beginning to incorporate findings into
policy and planning discussions• Logistics of managing transcripts an early concern• Articulation and multiple measures
• More colleges working on their analysis• Large scale analysis underway• New research questions have emerged about
using transcripts for assessment• For how long are transcripts valid?• What is the predictive value EAP level and other tests?• What is the influence of student level characteristics?
The Potential of Transcript-Based Placement: CCCCIO, April 2013
![Page 25: The Potential of Transcript-Based Placement](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816241550346895dd27b57/html5/thumbnails/25.jpg)
Discussion• For those of you that have reviewed transcript data,
have you used this information in your placement process?
• How did you get this information?• What could colleges do to increase their access to
transcript information?• What factors seem most important for successfully
using transcripts?• How important would it be at your institution to link
shifts in course offerings with changes in assessment processes?
25The Potential of Transcript-Based Placement: CCCCIO, April 2013
![Page 26: The Potential of Transcript-Based Placement](https://reader035.vdocuments.us/reader035/viewer/2022062520/56816241550346895dd27b57/html5/thumbnails/26.jpg)
26
Find Out MorePrimary Contact:Terrence Willett, RP [email protected] , (831) 461-4586http://www.rpgroup.org/projects/STEPS
Team MembersAll participating collegesAndrew Fuenmayor, Long Beach City College Developer/ResearcherJohn Hetts, Long Beach City College ResearcherCraig Hayward, RP ResearcherDan Lamoree, Lead RP DeveloperEmily Lawrence, CalPASSplus ResearcherNathan Pellegrin, RP ResearcherKaren Rothstein, Long Beach City College ResearcherKen Sorey, CalPASSplus ManagerNick Wade, CalPASSplus DeveloperAlex Zakharenkov, CalPASSplus Developer
Special Thanks to Dustin Tamashiro of Pasadena City College