admissions testing: predicting college success

Admissions Testing: Predicting College Success

CRESST September 2002

Wayne J. Camara

The College Board

Overview Presentation focuses on…

• Admission Models and Practices• Predictors/Criterion of College

Success• Validity and Subgroup Differences• SAT I and SAT II• Changes to SAT in 2005

Importance of Admisson Factors – 4 year Public

1

1.5

2

2.5

3

3.5

4

4.5

1979 1985 1992 2000

HSGPA/rank

Admissions test

HS Courses

Letters

Essays

Portfolio, Auditions

Interviews

Importance of Admisson Factors – 4 year Private

1

1.5

2

2.5

3

3.5

4

4.5

1979 1985 1992 2000

HSGPA/rank

Admissions test

HS Courses

Letters

Essays

Portfolio, Auditions

Interviews

Preference in Admissions Test (4-year institutions)

• 82% of all colleges require test in 99 vs 86% in 89-90

• For non-open colleges, (5%) 91% required test in 89-90 vs 87% in 99

• Fairtest errors – only 173 of the 390 institutions listed are truly SAT/ACT optional.

Annual Survey of Colleges (College Board)

0102030405060708090

89-

90

94-

95

97-

98

98-

99

Non-Open Colleges

1 test

Either

none

Predictor-Criterion Relationship: CriteriaCriterion - What outcome do you want to

measure? Is it measurable? • Predict college success:

• completion (persistence, graduation)• academic (FGPA, Cum GPA)

• Life success: • occupational (salary, job title, degrees)• skills (team work, creativity, leadership)• community service

• Benefit: • social / economic mobility/ growth

Average Predictability of Cumulative GPA By Year (Kunkel et. al. 2002)

0.650.61

0.720.68

0.65

0.76

0.680.64

0.75

0.670.64

0.74

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

HS VM VMH

Year 1

Year 2

Year 3

Year 4

Four Years of Course Grades by Combined SAT Score

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

700 andbelow

710-800 810-900 910-1000 1010-1100

1110-1200

1210-1300

1310-1400

1410-1500

1510-1600

A (3.90-4.30)

A- (3.60-3.89)

B+ (3.30-3.59)

B (2.90-3.29)

B- (2.60-2.89)

C+ (2.30-2.59)

C (1.90-2.29)

C- (1.60-1.89)

D (0.60-1.59)

F (below 0.59)

Graduation

0%

20%

40%

60%

80%

100%

700 700-

849

850-

999

1000-

1149

1150-

1299

> 1300

A

B

C+

SAT V + M (Astin et al, 1996)

365 institutions, 76,000 students

Added value beyond high school grades

SAT HSGPA Both IncrementAll .57 .61 .68 .08 (Bridgeman, et. Al., 2000)

Male .56 .58 .61 .07Female .62 .61 .71 .10Cumulative .36 .42 .52 .10 (Burton & Ramist 2002)

Persistence (all correlations are uncorrected)Non-minority.25 .34 -Minority .35 .44 -Graduation .29 .33 .36 .03

Added value beyond high school grades

• Increment of SAT is about .10 above HSGPA and greatest for African American students

• HS grades overpredict College grades for Hispanic (.19) and African American males (.22) most. They underpredict grades for white females (-.09)

NELS Proficiency Levels x Ethnicity and Income

0 20 40 60

Inc

om

e

Percent

Math 5

Math 4 or 5

Reading 3

Same Group Differences in HS Grades and Class Rank

MeanHigh School Grades

Mean High School Rank

Subgroup A B C BelowC

90th 80th 60th Below60th

African-American

18.9 53.2 26.8 1.1 11.9 18.8 28.7 40.6

Asian-American

47.5 42.7 7.0 0.0 27.8 24.9 25.7 21.7

Hispanic 30.0 53.4 16.1 0.5 16.5 21.5 28.5 33.5White 40.3 47.8 11.7 0.3 23.2 23.2 27.3 26.3

Differences between HS & College Grades

Subgroup HS GPA FGPA Difference

African American

3.18 2.14 1.04

Asian 3.58 2.80 .78

Hispanic 3.33 2.37 1.16

White 3.40 2.66 .74

Parental Ed and HSGPA

0

10

20

30

40

50

60

<HS

HS Graduate

Some College

College G

raduate

Some Grad. S

chool

A

B

C or below

Graduation

Six-year graduation rates:

• African Americans (38%)

• Hispanics (45%)• Asians (63%)• Whites (56%)

• SAT scores have only slightly less weight in predicting graduation than HS grades.

• Actual effects are under estimated as students transfer and attrite who are in good standing.

Impact of SAT - Test/Grade Discrepant Score Analysis

• 68% of students have SAT scores that are highly consistent with their pattern of grades.

• The remaining 32% of students have discrepant SAT/grades. Half with higher grades. Unadjusted correlation of FGPA with SAT/HSGPA

SAT =HSGPA

HSGPAHigher

SATHigher

SATV+M

.457 .379 .452

HSGPA .462 .356 .464

N = 48,410, 23 colleges

SAT I vs SAT II

FGPA Course Grade

SAT I .60 .59

HSGPA .63 .57

SAT II .62 .57

SAT I + HSGPA .71 .71

SAT I + II .63 .63

SAT II + HSGPA .72 .69

SAT I, II+ HSGPA .72 .73

42,985 students entering 38 colleges in 82 and 85. Corrected for shrinkage, restriction of range and crit. unrel.

SAT I vs SAT II• SAT II more selective test takers • Mean SAT I and II for CA Students taking both:

V+M Writing Math 1c Math 2c 3 SAT II’s white 1200 593 574 646 1764 Latino 1018 (182) 506 (87) 500 (74) 572 (74) 1630 (134) Black 1016 (184) 509 (84) 487 (87) 576 (70) 1503 (261) Asian A 1150 (50) 540 (53) 581 (-7) 656 (-10) 1730 (34)

• Correlations:• SAT I V (SAT II W .79) (SAT II Lit .83)• SAT I M (SAT II M1C .84) (SAT II M2C .79)

SAT I vs SAT IIStandardized Differences

SAT I SAT II Writing/ Math IC

SAT II Writing, Math and 3rd

Asian Amer .27 .50 / -.11 .12

African Amer .98 .84 / .93 .95

Latino .97 .84 / .78 .50

K/C/J Spanish Other 3rd test

Latino - .31 (4,440) .69 (3,896)

Asian Amer -.47 (3,581) - .27 (14,359)

K/C/J = Korean, Chinese, Japanese

SAT I and ACT

• SAT I 138 questions vs ACT 210 questions in similar time

• SAT two scores (V, M), ACT one score (C, and four subscores)

• Concordance Study by CB/ETS/ACT (1997)• SAT V+M and ACT Composite r=.92• SAT M and ACT M r =.89• SAT V and ACT Reading or English r=.83• SAT V and SAT M r = .66

• Correlations between SAT I and SAT II are lower than Correlation between SAT I and ACT

New SAT – Related to College Skills (March/2005)

• All students take same test – equivalent to current SAT I

• Three scores (200 – 800 scale) - Critical Reading, Math Reasoning, & Writing

• Writing includes a 25-minute essay and MC• Distributed reading and essay available on web

• Math more curriculum-related, retain open ended items (qc items removed)

• Critical Reading – more passages based on curriculum (analogies removed)