chapter 4 alignment among secondary and … · ... (ninth grade language and composition and...
TRANSCRIPT
63
Chapter 4 Alignment Among Secondary and Postsecondary Assessments
in Georgia
The Georgia Assessment Environment
Over the past 10 years, Georgia’s testing program has undergone many changes.
In 1991, the Georgia Assembly established a new set of tests, the Georgia High School
Graduation Tests (GHSGTs), which replaced the previous graduation exam, the
minimum competency Basic Skills Test. The GHSGTs are intended to measure the
learning objectives specified in the state standards (i.e., Quality Core Curriculum), and to
determine whether a student should receive a high school diploma. The new GHSGTs
differed from the previously required Basic Skills Test in that they assessed achievement
in social studies and science (in addition to reading, mathematics, and writing). State law
also mandated that the GHSGTs be a more rigorous test than the Basic Skills Test, and
extended the range of item difficulty found on the GHSGTs.
The GHSGTs graduation tests were phased in gradually. The first group of
students to be affected was those who entered the ninth grade between July 1991 and July
1993 (these would have been students expected to graduate in spring 1995 and 1996).
These students had to pass the English/language arts, mathematics, and writing tests to be
eligible to receive a diploma. The next group, students who entered the ninth grade
between July 1993 and July 1994 (the expected class of '97), was required to pass the
social studies test. And finally, the science test was required for those students who
entered the ninth grade after July 1994 (expected class of '98).
Students have five opportunities to take each of the tests before the end of the
12th grade. There are two testing periods during the junior year. The Writing test is
offered in the early fall semester and again in the spring semester for those students who
need to retake it. Content-specific exams are administered only during the spring
semester of the junior year. After that, the students have three more opportunities to pass
the writing test (summer, fall and spring of their senior year) and the content-specific
exams (summer, fall and spring of their senior year). Students who do not pass all the
required tests but have met all other graduation requirements are eligible to receive a
Certificate of Performance or a Special Education Diploma. These students may then
64
return to take any graduation test(s) necessary in order to qualify for a high school
diploma. There is no limit for these students on the number of times they may retake any
of the tests.
Currently, there are plans to replace the GHSGTs with new assessments
developed as part of the End-of-Course Testing (EOCT) program. Slated for full
implementation in 2003, the EOCT will be multiple-choice, end-of-course exams
assessing student knowledge of course standards in eight core courses, specifically math
(algebra and geometry), ELA (ninth grade language and composition and American
language and composition), social studies (U.S. history and economics/business), and
science (biology and physical science). Although there are plans to make passing some
or all of the end-of-course tests a condition for graduation, there has not been a final
decision regarding how the ECOT will affect graduation requirements.
Georgia Assessments Included in this Study
For this study, we did not include any EOCT assessments because they were not
available when this study was initiated. However, we were able to examine the reading,
editing, math, and writing sections of the GHSGTs. The reading, editing, and math
sections of the GHSGTs assess student knowledge with multiple-choice items, whereas
the writing section requires a written composition. Although each section of the test has
a recommended time limit, the time limit may be extended at the discretion of the test
proctor if students appear to be making progress and need additional time.
In addition to the GHSGTs, students applying to a college in Georgia are often
required to take placement tests in math and/or English. Because the kinds of placement
tests given are likely to vary by the selectivity of the institution, we obtained placement
tests from both a highly selective university (e.g., University of Georgia) and a less
selective institution (e.g. community colleges). Many of the community colleges in
Georgia administer the COMPASS, which is a system of computer-adaptive assessments
that measure student readiness for a broad range of math and English courses.1 Unlike
traditional paper-and-pencil tests, which administer the same set of items to all
examinees, computer-adaptive measures administer different items to examinees based
1 We examined a set of secured items from the COMPASS item pool. COMPASS is published by ACT.
65
on their demonstrated proficiency level. Because students at different proficiency levels
may encounter differ item content mixes, findings about content coverage or cognitive
demands should be interpreted with caution.
For ELA, we examined assessments in writing skills and reading. For math, we
examined tests in numerical skills/prealgebra and algebra. The COMPASS Writing
Skills, COMPASS Reading, and COMPASS Numerical Skills/Prealgebra tests are used
to determine whether students need to enroll in remedial courses, but the COMPASS
Algebra can be used more broadly to place students into an appropriate math course up to
trigonometry.
The University of Georgia does not use a commercial placement test, but instead
administers their own departmentally-created post-admittance exams; we include their
math and English placement tests. At the University of Georgia, all incoming freshmen,
except those with AP Calculus credit or those who have transferred college-level calculus
credit from another postsecondary institution, are required to take a 45-minutes, 26-item
multiple-choice exam, which is used to place students in an appropriate mathematics
course (hereafter this test will be referred to as the UOG Math Placement Test). The
analogous English test, referred to hereafter as the UOG ELA Placement Test, consists of
two sessions: a morning session in which students are given 55 minutes to complete 60
multiple-choice items, and an afternoon session in which students are to compose an a
writing sample in one hour. The writing session is required only of students whose
performance during the morning session is below a predetermined cut-off score.
High school students may also choose to take the Georgia Early Math Placement
Test (GEMPT) in their junior year. The purpose of this measure is to assess students’
current mathematics proficiency level in relation to the skills required by college-level
mathematics courses. Students are made aware of deficiencies that would require them to
take a remedial mathematics course in college, and are encouraged to take the appropriate
preparatory courses during their senior year in high school so that the students do not
have to take the remedial course at the postsecondary level.
Tables 4.1 and 4.2, organized by test function, list these testing programs and the
type of information we were able to obtain for this study. For most tests, we used a
single form from a recent administration or a full-length, published sample test. In a few
66
instances where full-length forms were unavailable, we used published sets of sample
items. This was the case for the GHSGTs. For the ELA tests, Table 4.2 specifies
whether the test includes each of three possible skills: reading, editing, and writing
67
Tab
le 4
.1
Tec
hnic
al C
hara
cter
isti
cs o
f th
e M
athe
mat
ics
Ass
essm
ents
T
est
Tes
t Typ
e M
ater
ials
E
xam
ined
T
ime
Lim
it
Num
ber
and
Typ
e of
Ite
ms
Too
ls
Pur
pose
C
onte
nt a
s S
peci
fied
in T
est
Spe
cifi
cati
ons
Geo
rgia
Hig
h S
choo
l Gra
duat
ion
Tes
t (G
HSG
T)
Sta
te a
chie
vem
ent
Sam
ple
item
s 18
0 m
inut
es
60 M
C
Cal
cula
tor
Mon
itor
stu
dent
ac
hiev
emen
t to
war
d st
ate-
appr
oved
con
tent
st
anda
rds
Num
ber
and
Com
puta
tion
(17
-19
%),
Dat
a A
naly
sis
(19-
21%
),
Mea
sure
men
t and
Geo
met
ry (
32-
34%
), A
lgeb
ra (
28-3
0%)
AC
T
Col
lege
ad
mis
sion
s Fu
ll s
ampl
e fo
rm
60 m
inut
es
60 M
C
Cal
cula
tor
Sel
ecti
on o
f st
uden
ts f
or h
ighe
r ed
ucat
ion
Pre
alge
bra
(23%
), e
lem
enta
ry
alge
bra
(17%
), in
term
edia
te
alge
bra
(15%
), c
oord
inat
e ge
omet
ry (
15%
), p
lane
geo
met
ry
(23%
) an
d tr
igon
omet
ry (
7%)
SA
T I
C
olle
ge
adm
issi
ons
Full
sam
ple
form
75
min
utes
35
MC
15 Q
C
10 G
R
Cal
cula
tor
Sel
ecti
on o
f st
uden
ts f
or h
ighe
r ed
ucat
ion
Ari
thm
etic
(13
%),
alg
ebra
(3
5%),
geo
met
ry, (
26%
), a
nd
othe
r (2
6%)
SAT
II
Mat
hem
atic
s L
evel
IC
Col
lege
ad
mis
sion
s Fu
ll s
ampl
e fo
rm
60 m
inut
es
50 M
C
Cal
cula
tor
S
elec
tion
of
stud
ents
for
hig
her
educ
atio
n
Alg
ebra
(30
%),
geo
met
ry (
38%
,
spec
ific
ally
pla
ne E
ucli
dean
(2
0%),
coo
rdin
ate
(12%
), a
nd
thre
e-di
men
sion
al (
6%))
, tr
igon
omet
ry (
8%),
fun
ctio
ns
(12%
), s
tati
stic
s an
d pr
obab
ilit
y (6
%),
and
mis
cell
aneo
us (
6%)
SAT
II
Mat
hem
atic
s L
evel
II
C
Col
lege
ad
mis
sion
s Fu
ll s
ampl
e fo
rm
60 m
inut
es
50 M
C
Cal
cula
tor
Sel
ecti
on o
f st
uden
ts f
or h
ighe
r ed
ucat
ion
Alg
ebra
(18
%),
geo
met
ry (
20%
,
spec
ific
ally
coo
rdin
ate
(12%
) an
d th
ree-
dim
ensi
onal
(8%
)),
trig
onom
etry
(20
%),
fun
ctio
ns
(24%
), s
tati
stic
s an
d pr
obab
ilit
y (6
%),
and
mis
cell
aneo
us (
12%
)
Tab
le c
onti
nues
67
68
Tes
t T
est T
ype
Mat
eria
ls
Exa
min
ed
Tim
e L
imit
N
umbe
r an
d T
ype
of I
tem
s T
ools
P
urpo
se
Con
tent
as
Spe
cifi
ed in
Tes
t S
peci
fica
tion
s
CO
MP
AS
S
Alg
ebra
C
olle
ge p
lace
men
t
Full
sam
ple
form
N
o ti
me
lim
it
Num
ber
of M
C
item
s gi
ven
depe
nds
on
abil
ity
Non
e A
sses
s st
uden
t pr
ofic
ienc
y in
al
gebr
a,
inte
rmed
iate
al
gebr
a, a
nd
coor
dina
te
geom
etry
Alg
ebra
, int
erm
edia
te a
lgeb
ra,
coor
dina
te g
eom
etry
CO
MP
AS
S
Num
eric
al
Ski
lls/
Pre
alge
bra
Col
lege
pla
cem
ent
Full
sam
ple
form
N
o ti
me
lim
it
Num
ber
of M
C
item
s gi
ven
depe
nds
on
abil
ity
Non
e A
sses
s pr
ofic
ienc
y in
pre
alge
bra
Pre
alge
bra
Geo
rgia
Ear
ly
Mat
h P
lace
men
t T
est (
GE
MP
T)
Col
lege
pla
cem
ent
Ful
l sam
ple
form
45
min
utes
32
MC
C
alcu
lato
r A
sses
s cu
rren
t m
athe
mat
ics
prof
icie
ncy
leve
l in
rela
tion
to c
olle
ge-
leve
l mat
hem
atic
s co
urse
s
Pre
alge
bra,
ele
men
tary
alg
ebra
, ge
omet
ry, i
nter
med
iate
alg
ebra
Uni
vers
ity
of
Geo
rgia
Mat
h P
lace
men
t Tes
t (U
OG
Mat
h P
lace
men
t Tes
t)
Col
lege
pla
cem
ent
Ful
l sam
ple
form
45
min
utes
26
MC
C
alcu
lato
r P
lace
men
t of
stud
ents
into
ap
prop
riat
e m
ath
cour
se
Ari
thm
etic
, alg
ebra
, geo
met
ry,
trig
onom
etry
(se
e ht
tp:/
/ww
w.m
ath.
uga.
edu/
~cur
r/P
lace
men
tTop
ics.
htm
l for
a
com
plet
e li
st o
f to
pics
)
Not
es.
MC
= m
ultip
le-c
hoic
e O
E =
ope
n-en
ded
G
R =
gri
d-in
Q
C =
qua
ntita
tive
com
pari
son
68
69
Tab
le 4
.2
Tec
hnic
al C
hara
cter
isti
cs o
f th
e E
nglis
h/L
angu
age
Art
s A
sses
smen
ts
Tes
t T
est F
unct
ion
Mat
eria
ls
Exa
min
ed
Tim
e L
imit
N
umbe
r an
d T
ype
of I
tem
s P
urpo
se
Rea
ding
S
ecti
on?
Edi
ting
S
ecti
on?
Wri
ting
S
ecti
on?
Geo
rgia
Hig
h S
choo
l G
radu
atio
n T
est
(GH
SGT
) E
ngli
sh/L
angu
age
Art
s
Sta
te
achi
evem
ent
Sam
ple
item
s 18
0 m
inut
es f
or
both
rea
ding
and
ed
itin
g
Add
itio
nal t
ime
allo
wed
if
stud
ent i
s m
akin
g pr
ogre
ss
50 M
C f
or b
oth
read
ing
and
edit
ing
Mon
itor
stu
dent
ac
hiev
emen
t tow
ard
stat
e st
anda
rds,
pro
vide
hig
h sc
hool
dip
lom
a
Y
Y
N
Geo
rgia
Hig
h S
choo
l W
riti
ng T
est
(GH
SWT
)
Sta
te
achi
evem
ent
Sam
ple
item
s 90
min
utes
for
w
riti
ng s
ecti
on
Add
itio
nal t
ime
allo
wed
if
stud
ent i
s m
akin
g pr
ogre
ss
1 O
E w
riti
ng
Mon
itor
stu
dent
ac
hiev
emen
t tow
ard
stat
e st
anda
rds,
pro
vide
hig
h sc
hool
dip
lom
a
N
N
Y
AC
T
Col
lege
ad
mis
sion
s Fu
ll s
ampl
e fo
rm
80 m
inut
es
--35
min
utes
re
adin
g
--45
min
utes
ed
itin
g
40 M
C r
eadi
ng
75 M
C e
diti
ng
Sel
ecti
on o
f st
uden
ts f
or
high
er e
duca
tion
Y
Y
N
AP
Lan
guag
e an
d C
ompo
siti
on
Col
lege
ad
mis
sion
s
Full
sam
ple
form
18
0 m
inut
es
--60
min
utes
re
adin
g
-- 1
20 m
inut
es
wri
ting
52 M
C r
eadi
ng
1 O
E r
eadi
ng
2 O
E w
riti
ng
Pro
vide
opp
ortu
niti
es f
or
HS
stu
dent
s to
rec
eive
co
lleg
e cr
edit
and
ad
vanc
ed c
ours
e pl
acem
ent
Y
N
Y
SA
T I
C
olle
ge
adm
issi
ons
Full
sam
ple
form
75
min
utes
40 M
C r
eadi
ng
38 M
C e
diti
ng
Sel
ecti
on o
f st
uden
ts f
or
high
er e
duca
tion
Y
Y
N
Tab
le c
onti
nues
69
70
Tes
t T
est F
unct
ion
Mat
eria
ls
Exa
min
ed
Tim
e L
imit
N
umbe
r an
d T
ype
of I
tem
s P
urpo
se
Rea
ding
S
ecti
on?
Edi
ting
S
ecti
on?
Wri
ting
S
ecti
on?
SA
T I
I L
iter
atur
e C
olle
ge
adm
issi
ons
Full
sam
ple
form
60
min
utes
60
MC
rea
ding
S
elec
tion
of
stud
ents
for
hi
gher
edu
cati
on
Y
N
N
SA
T I
I W
riti
ng
Col
lege
ad
mis
sion
s Fu
ll s
ampl
e fo
rm
60 m
inut
es
-- 4
0 m
inut
es
edit
ing
-- 2
0 m
inut
es
wri
ting
60 M
C e
diti
ng
1 O
E w
riti
ng
Sel
ecti
on o
f st
uden
ts f
or
high
er e
duca
tion
N
Y
Y
CO
MP
ASS
Rea
ding
C
olle
ge
plac
emen
t Fu
ll s
ampl
e fo
rm
No
tim
e li
mit
N
umbe
r of
MC
qu
esti
ons
give
n de
pend
s up
on
abil
ity
Ass
ess
whe
ther
adm
itte
d st
uden
ts p
osse
ss e
ntry
le
vel r
eadi
ng s
kill
s
Y
N
N
CO
MP
AS
S W
riti
ng
Skil
ls
Col
lege
pl
acem
ent
Full
sam
ple
form
N
o ti
me
lim
it
Num
ber
of M
C
ques
tion
s gi
ven
depe
nds
upon
ab
ilit
y
Ass
ess
whe
ther
adm
itte
d st
uden
ts p
osse
ss e
ntry
le
vel e
diti
ng s
kill
s
N
Y
N
Uni
vers
ity
of G
eorg
ia
Eng
lish
/Lan
guag
e A
rts
Pla
cem
ent T
est (
UO
G
EL
A P
lace
men
t Tes
t)
Col
lege
pl
acem
ent
Full
sam
ple
form
O
ne 5
5-m
inut
e M
C s
essi
on f
or
both
rea
ding
and
ed
itin
g
One
60-
min
ute
wri
ting
ses
sion
60 M
C f
or b
oth
read
ing
and
edit
ing
1 O
E w
riti
ng
Ass
ess
whe
ther
adm
itte
d st
uden
ts p
osse
ss e
ntry
le
vel E
ngli
sh s
kill
s
Y
Y
Y
Not
es.
MC
= m
ultip
le-c
hoic
e O
E =
ope
n-en
ded
70
71
Alignment Among Georgia Math Assessments
In this section, we describe the results of our alignment exercise for the math
assessments. The results are organized so that alignment among tests with the same
function is presented first, followed by a discussion of alignment among tests with
different functions.
Alignment is described by highlighting similarities and differences with respect to
technical features, content, and cognitive demands. That is, we first present how the
assessments vary along characteristics such as time limit, format, contextualized items,
graphs, diagrams, and formulas. We then document differences with respect to content
areas, and conclude with a discussion of discrepancies in terms of cognitive requirements.
Table 4.3 presents the alignment results for the math assessments. The numbers
in Table 4.3 represent the percent of items falling into each category. As an example of
how to interpret the table, consider the SAT I results; 58% of its items are multiple-
choice, 25% are quantitative comparisons, and 17% are grid-in items. With respect to
contextualization, 25% of the SAT I questions are framed as a real-life word problem.
Graphs are included within the item-stem on 7% of the questions, but graphs are not
included within the response options (0%), and students are not asked to produce any
graphs (0%). Similarly, diagrams are included within the item-stem on 18% of the
questions, but diagrams are absent from the response options (0%), and students are not
required to produce a diagram (0%). With respect to content, the SAT I does not include
trigonometry (0%), and assesses elementary algebra (37%) most frequently. In terms of
cognitive demands, procedural knowledge (53%) is the focus of the test, but conceptual
understanding (32%) and problem solving (15%) are assessed as well. Results for the
other tests are interpreted in an analogous manner.
72
Tab
le 4
.3
Alig
nmen
t A
mon
g th
e T
echn
ical
, Con
tent
, and
Cog
niti
ve D
eman
ds C
ateg
orie
s fo
r th
e M
ath
Ass
essm
ents
For
mat
Con
text
Gra
phs
D
iagr
ams
F
orm
ulas
Con
tent
Cog
niti
ve
Dem
ands
T
est
MC
Q
C
GR
O
E
C
S
RO
P
S
RO
P
M
G
P
A
EA
IA
C
G
PG
T
R
SP
M
ISC
CU
P
K
PS
Sta
te A
chie
vem
ent T
est
GH
SG
T
100
0 0
0
62
9
0 0
21
0
0
11
6
25
18
2 9
29
0 18
0
29
69
2
Col
lege
Adm
issi
ons
Tes
ts
AC
T
100
0 0
0
22
5
2 0
13
0
0
15
0
17
22
5 15
25
8
3 5
40
53
7
SA
T I
58
25
17
0
25
7 0
0
18
0 0
1
8
13
37
2 6
19
0 13
11
32
53
15
SAT
II
Mat
h L
evel
IC
10
0 0
0 0
18
8 0
0
26
0 0
12
0
2
30
10
12
28
4 8
6
34
58
8
SAT
II
Mat
h L
evel
II
C
100
0 0
0
12
12
2
0
2 0
0
10
0
2 14
22
12
14
18
6
12
26
54
20
Col
lege
Pla
cem
ent T
ests
CO
MP
AS
S
Alg
ebra
10
0 0
0 0
5
16
0
0
0 0
0
16
5
11
37
21
26
0 0
0 5
5
90
5
CO
MP
AS
S
Num
eric
al
Skil
ls/P
re-
alge
bra
100
0 0
0
41
0
0 0
0
0 0
0
0
88
0 0
0 0
0 12
0
12
82
6
GE
MP
T
100
0 0
0
6
0 3
0
6 0
0
13
3
3 44
31
9
13
0 0
0
0 10
0 0
UO
G M
ath
Pla
cem
ent
Tes
t 10
0 0
0 0
8
0
8 0
19
0
0
27
0
0 27
27
15
12
19
0
0
23
77
0
. Form
at
Con
text
ualiz
atio
n D
iagr
ams
MC
= m
ultip
le-c
hoic
e ite
ms
C
=
cont
extu
aliz
ed it
ems
S
=
grap
h/di
agra
m w
ithin
item
-ste
m
QC
= q
uant
itativ
e co
mpa
riso
n ite
ms
R
O
=
grap
h/di
agra
m w
ithin
res
pons
e op
tions
G
R =
fill
-in-
the-
grid
item
s
P
=
grap
h/di
agra
m n
eeds
to b
e pr
oduc
ed
Tab
le c
onti
nues
72
73
Not
es.
Form
at
Con
text
ualiz
atio
n D
iagr
ams
MC
= m
ultip
le-c
hoic
e ite
ms
C
=
cont
extu
aliz
ed it
ems
S
=
grap
h/di
agra
m w
ithin
item
-ste
m
QC
= q
uant
itativ
e co
mpa
riso
n ite
ms
R
O
=
grap
h/di
agra
m w
ithin
res
pons
e op
tions
G
R =
fill
-in-
the-
grid
item
s
P
=
grap
h/di
agra
m n
eeds
to b
e pr
oduc
ed
OE
= o
pen-
ende
d ite
ms
OE
= o
pen-
ende
d ite
ms
Form
ulas
C
onte
nt A
reas
C
ogni
tive
Dem
ands
M
=
for
mul
a ne
eds
to b
e m
emor
ized
PA
=
prea
lgeb
ra
C
U
=
conc
eptu
al u
nder
stan
ding
G
=
for
mul
a is
pro
vide
d
EA
=
elem
enta
ry a
lgeb
ra
PK
=
pr
oced
ural
kno
wle
dge
IA
=
inte
rmed
iate
alg
ebra
P
S =
pr
oble
m-s
olvi
ng
CG
=
coor
dina
te g
eom
etry
P
G =
pl
ane
geom
etry
T
R =
tr
igon
omet
ry
SP
=
stat
istic
s an
d pr
obab
ility
MIS
C
=
mis
cell
aneo
us to
pics
73
Tab
le c
onti
nues
74
Alignment Among Tests With the Same Function
State Achievement Tests
Only one state achievement test is included in this analysis, the GHSGT Math. It
is a 3-hour, 60-item, multiple-choice test assessing math knowledge up to geometry.
Most of its items assess procedural knowledge (69%), but a moderate proportion assesses
conceptual understanding as well (29%).
College Admissions Tests
We examined four college admissions tests: the ACT, SAT I, SAT II Math Level
IC, and SAT II Math Level IIC. All tests, except the SAT I, have a one-hour time limit.
The SAT I has a 75-minute time limit. All four exams are also predominantly multiple-
choice, although the SAT I includes quantitative comparison (25%) as well as grid-in
(17%) items. Contextualized questions are most prevalent on the SAT I (25%) and least
prevalent on the SAT II Math Level IIC (12%). Students are rarely asked to work with
graphs, and questions that contain graphs within the item-stem constitute no more than
12% of items on the college admissions measures. Questions that include diagrams
within the item-stem are more prevalent, comprising 26%, 18%, and 13% of items on the
SAT II Math Level IC, SAT I, and ACT, respectively. However, questions with
diagrams are infrequent on the SAT II Math Level IIC (2%). Formulas are also
uncommon, but there are differences with respect to the extent to which formulas are
necessary. Whereas the ACT, SAT II Math Level IC, and SAT II Math Level IIC include
items in which a memorized formula is needed (15%, 12%, and 10%, respectively), these
items are largely absent from the SAT I (1%).
Although college admissions exams generally sample from the same content
areas, they do not do so to the same extent. Elementary algebra comprises most of the
SAT I items (37%). The SAT II Math Level IC also emphasizes elementary algebra
(30%), but focuses on planar geometry as well (28%). The ACT shows a similar content
emphasis as that of the SAT II Math Level IC; 22% of its items assess elementary algebra
and 25% assess planar geometry. The SAT II Math Level IIC, on the other hand, draws
from more advanced content areas, such as intermediate algebra (22%) and trigonometry
(18%).
75
In terms of cognitive demands, all four tests assess procedural knowledge to a
similar degree. Procedural knowledge items constitute between 54% and 58% of the
items found on college admissions measures. However, there is more variation among
the exams with respect to emphasis on problem solving. The SAT I and SAT II Math
Level IIC place relatively greater emphasis on problem solving (20% and 15%,
respectively) than do the ACT and SAT II Math Level IC (7% and 8%, respectively).
College Placement Tests
There are four college placement tests in this analysis: the COMPASS Algebra,
COMPASS Numerical Skills/Prealgebra, GEMPT, and UOG Math Placement Test. All
four tests consist only of multiple-choice items, but the COMPASS Algebra and
COMPASS Numerical Skills/Prealgebra are computer-adaptive measures whereas the
GEMPT and UOG Math Placement Test are paper-and-pencil measures. The
COMPASS Numerical Skills/Prealgebra contains many items framed in a real-life
context (41%), but contextualized items are largely excluded from the COMPASS
Algebra, GEMPT, and UOG Math Placement Test (5%, 6%, and 8%, respectively).
Questions in which graphs are included within the item-stem are absent on all of the
college placement measures except the COMPASS Algebra (16%). Items that contain
diagrams are excluded from the COMPASS Algebra and COMPASS Numerical
Skills/Prealgebra, but comprise a small proportion of the GEMPT (6%), and a moderate
proportion of the UOG Math Placement Test (19%). With respect to items requiring a
memorized formula, the COMPASS Numerical Skills/Prealgebra excludes such items,
but the other college placement measures include this type of question to some extent
(13%-27%).
With the exception of the COMPASS Numerical Skills/Prealgebra, which is
narrowly focused on a single topic area, namely prealgebra (88%), the other college
placement measures show a broad range of content sampling. The COMPASS Algebra
and the GEMPT assess elementary algebra most frequently, (37% and 44%, respectively)
but also assess intermediate algebra to a moderate extent (21% and 31%, respectively).
Content differences between these two tests are most evident with respect to sampling of
coordinate geometry, which comprises 26% of items on the COMPASS Algebra, but 9%
76
of items on the GEMPT. As with the COMPASS Algebra and the GEMPT, the UOG
Math Placement Test focuses most on elementary algebra and intermediate algebra (27%
of items on each content area). However, the UOG Math Placement test also assesses
trigonometry (19%), a topic that is devoid from all other college placement measures.
Few differences are observed with respect to cognitive requirements. Procedural
knowledge problems constitute the majority of the college placement measures, ranging
from 77% of the UOG Math Placement Test questions to 100% of the GEMPT questions.
Problem-solving items are uncommon (0%-6%), as are conceptual understanding items,
although conceptual understanding is represented on the UOG Math Placement test to a
moderate extent (23%).
Alignment Among Tests with Different Functions
With the exception of the SAT I, none of the math assessments requires students
to generate their own answers. Excluding the COMPASS Numerical Skills/Prealgebra
(41%), questions framed within a realistic context typically represent a small proportion
of college placement tests (5%-8% if the COMPASS Numerical Skills/Prealgebra is
excluded), a small to moderate proportion of college admissions measures (12%-25%),
and is most prevalent on the state achievement test (62%). Questions that contain graphs
within the item-stem are relatively uncommon, comprising no more than 16% on any of
the studied exams. Questions that contain diagrams within the item-stem are more
prevalent, but typically represent only a small or moderate fraction of a test. Questions
that contain diagrams within the item-stem represent 2%-26% of college admissions
items, 0%-19% of college placement items, and 21% of the GHSGT. Items requiring a
memorized formula are generally rare, and comprise less than 16% on all but one test.
The exception is the UOG Math Placement Test, where memorized formulas constitute
27% of its items.
With respect to the content category, college admissions exams assess logic
(coded as miscellaneous) and trigonometry more frequently than do the state achievement
or college placement measures. Trigonometry items, for example, are included on 4%-
18% of college admissions tests (if the SAT I is excluded), 0% of the GHSGT, and 0% of
college placement tests, the UOG Math Placement Test notwithstanding. The
77
anomalously high proportion of trigonometry items found on the UOG Math Placement
Test (19%) reflects the fact that this exam is used to place students into calculus, whereas
the other college placement measures are used to place students into either remedial
courses (COMPASS Numerical Skills/Prealgebra) or math courses no higher than
trigonometry (COMPASS Algebra). The state achievement test, the GHSGT, is broadly
distributed across a range of content areas, particularly prealgebra and planar geometry
(25% and 29%, respectively).
In terms of cognitive requirements, all tests emphasize procedural knowledge
most frequently, although there is variation with respect to extent. Procedural knowledge
items are most common on college placement tests (77%-100%), followed by the state
achievement test, the GHSGT (69%), and least common on college admissions tests
(53%-58%). Problem-solving items are most prevalent on college admissions (7%-20%)
and are rarely included on either the GHSGT (2%) or college placement tests (0%-6%).
Conceptual understanding items are also most likely on college admission exams,
comprising between 26%-40% of the items.
Discussion
Below, we discuss the implications of the discrepancies among the math
assessments. We begin by highlighting instances in which differences are justified, then
address whether there were any misalignments that may send students confusing signals.
We also explore the possibility that state achievement tests can inform postsecondary
decisions.
Which Discrepancies Reflect Differences in Test Use?
As noted in Chapter 1, content discrepancies may reflect differences in intended
test use. To illustrate, consider the SAT II Math Level IIC and GHSGT. Although both
the SAT II Math Level IIC and GHSGT include topics from a wide variety of courses,
the SAT II Math Level IIC includes many trigonometry and problem-solving items (18%
and 20%, respectively), whereas the GHSGT rarely includes such material (0% and 2%,
respectively). In this particular case, the SAT II Math Level IIC and GHSGT have
disparate functions, and content differences reflect variations in purpose. Because the
78
SAT II Math Level IIC is used to select among higher-achieving students for entrance
into universities and colleges, it includes many problem-solving and trigonometry items
in order to distinguish among higher-proficiency examinees. The GHSGT, on the other
hand, is used to monitor student achievement statewide, and therefore requires items of
more moderate difficulty that can be attempted by students with a wider range of
proficiency levels.
The above example represents justifiable discrepancies across tests with different
purposes. However, there are also instances in which discrepancies within tests of
similar purposes are warranted as well. As mentioned earlier, that trigonometry is
prevalent on the UOG Math Placement Test but absent from the COMPASS Algebra is
justifiable given that the former test is used to place students into calculus, whereas the
latter test is used to place students into math courses no higher than trigonometry.
Likewise, differences between the SAT I and ACT with respect to emphasis on problem-
solving or non-routine logic problems is indicative of the fact that the SAT I is intended
to be a reasoning measure, but the ACT is intended to assess content found in high-school
math courses.
Is There Evidence of Misalignment?
As defined in Chapter 1, misalignments refer to those discrepancies that are not
attributable to test function, and therefore send students confusing signals regarding the
kinds of skills that are needed to perform well on a given test. In our analysis of the
math tests, we could not find any examples of misalignments, as discrepancies among
college admissions, college placement, and state achievement measures appear to have
stemmed from variations in test use. Additionally, differences among tests of similar
purposes are either small or moderate, or reflect nuances in purpose (e.g., see the above
UOG Math Placement Test and COMPASS Algebra example).
Can State Achievement Tests Inform Postsecondary Admissions and Course Placement Decisions?
Although there are many discrepancies among exams of different functions, it
may still be possible that a test can serve multiple purposes satisfactorily. Currently,
79
some measures are used for more than one purpose. Many postsecondary institutions, for
example, allow students to submit scores from college admissions exams such as the SAT
I or ACT as a means of exemption from a remedial college placement test. Potentially,
state achievement tests can be used for similar purposes, but no postsecondary institution
to date has made use of GHSGT scores for placement decisions. However, policymakers
have advocated using scores on some graduation tests for purposes beyond monitoring
student achievement (Olson, 2001b; Schmidt, 2000) because such a policy change would
not only reduce testing burden, but it would also motivate students to focus on state
standards rather than on external tests like the SAT I or ACT (Healy, 2001; Olson, 2001a;
Standards for Success, 2001). Below, we discuss the potential of the GHSGT for guiding
college placement and admissions decisions. We also discuss the possibility that testing
burden can be eased by using scores from the SAT II Math Level IIC for other purposes,
namely placement into an appropriate math course.
It may be possible to use GHSGT scores to exempt students from remedial
courses, as its content is more rigorous than that of the COMPASS Numerical
Skills/Prealgebra. Because the GHSGT assesses elementary algebra and geometry more
frequently than does the COMPASS Numerical Skills/Prealgebra, a sufficiently high
score on the GHSGT should logically excuse students from having to take the
COMPASS Numerical Skills/Prealgebra.
However, the GHSGT cannot be used to place students into an appropriate math
course because it rarely assesses material beyond geometry. Compared to the
COMPASS Algebra (which assesses intermediate algebra) or the UOG Placement Test
(which assess trigonometry), the GHSGT would not provide sufficient information
regarding whether an examinee has the necessary background to enroll in a high-level
math course such as trigonometry or calculus.
For the same reason, the GHSGT is not a viable alternative to any of the college
admissions measures. It contains fewer intermediate algebra and trigonometry items
than either the SAT II Math Level IC or SAT II Math Level IIC, and contains a lower
proportion of problem-solving items than either the SAT I or ACT. Because the GHSGT
does not sample as extensively from these content areas, its discriminating power is likely
to be more limited than that of the college admissions exams.
80
It may be possible to ease students’ testing burden in other ways beyond
expanding the use of the GHSGT scores. For instance, the SAT II Math Level IIC can be
used to place students into an appropriate math course up to calculus. The SAT II Math
Level IIC contains approximately the same proportion of intermediate algebra items as
the COMPASS Algebra, as well as the same proportion of trigonometry items as the
UOG Math Placement Test. The SAT II Math Level IIC also has the added advantage of
assessing problem-solving to a greater extent than do these two other tests. Conceivably,
academic counselors can use the scores from the SAT II Math Level IIC to advise
students which math course is most appropriate. To determine the feasibility of the SAT
II Math Level IIC as a measure that informs placement decisions, more research is
needed to explore the interrelationships between the SAT II Math Level IIC, the UOG
Math Placement Test, and grades in higher-level math courses (i.e., calculus).
Alignment Among Georgia ELA Assessments
Below we present the ELA results. As with math, we discuss discrepancies both
within and across test functions. The results are also organized by skill, namely reading,
editing, and writing. In some instances, there are only two tests that assess the same skill
and share the same function, so it is important to recognize that patterns or comparisons
between these tests may not be indicative of more general trends within this category of
tests.
Alignment is characterized by describing differences with respect to technical
features, content, and cognitive demands. Specifically, we discuss differences in time
limit and format, then document discrepancies with respect to topic, voice, and genre of
the reading passages, before concluding with variations in cognitive processes.
The alignment results for tests that measure reading skills are presented in Tables
4.4-4.5. Tables 4.6-4.7 provide the results for exams that assess editing skills, and Tables
4.8-4.9 provide the findings for exams that assess writing skills. For each table, the
numbers represent the percent of items falling in each category. To provide a concrete
example of how to interpret the findings, consider the content category results for the AP
Language and Composition, presented in Table 4.4. With respect to topic, 50% of the
reading passages included on the AP Language and Composition are personal accounts,
81
whereas 25% of the topics are about humanities, and the remaining 25% are about natural
science. It does not include topics from fiction or social science (0% each). In terms of
the author’s voice, 75% of the passages are written in a narrative style, whereas the other
25% are written in an informative manner. With respect to genre, only essays (100%) are
used; passages on the AP Language and Composition are not presented as letters, poems,
or stories (0% each). Results for the other tests are interpreted in a similar manner.
82
Tab
le 4
.4
Alig
nmen
t W
ithi
n th
e C
onte
nt C
ateg
ory
for
the
Rea
ding
Pas
sage
s
Top
ic
V
oice
Gen
re
Tes
t
Fic
tion
H
uman
itie
s N
atur
al
Sci
ence
S
ocia
l S
cien
ce
Per
sona
l A
ccou
nts
N
arra
tive
Des
crip
tive
Per
suas
ive
Info
rmat
ive
L
ette
r E
ssay
P
oem
St
ory
Sta
te A
chie
vem
ent T
est
GH
SG
T
4
5 9
9 14
23
73
0 2
25
2
55*
16*
30
Col
lege
Adm
issi
ons
Tes
ts
AC
T
25
25
25
25
0
25
25
25
25
0 75
0
25
AP
Lan
guag
e an
d C
ompo
siti
on
0
25
25
0 50
0 25
25
0
0
100
0 0
SA
T I
20
40
20
20
0
20
40
20
20
0
80
0 20
SA
T I
I L
iter
atur
e
63
0 0
13
25
63
0
0 13
13
25
50
13
Col
lege
Pla
cem
ent T
ests
CO
MP
AS
S R
eadi
ng
13
25
25
25
13
38
0 0
63
0
88
0 13
UO
G E
LA
Pla
cem
ent T
est
0
50
0
0 50
50
0 0
50
0
100
0 0
Not
e.
* O
ne p
assa
ge c
onta
ined
bot
h an
ess
ay a
nd a
poe
m.
82
83
Reading Measures
Alignment Among Tests of the Same Function
State Achievement Tests
The only state achievement test included in our sample is the GHSGT, which
assesses reading proficiency solely with multiple-choice items. Because the GHSGT
does not contain separate sections for editing and reading items, we cannot determine
testing time earmarked specifically for assessing reading skills, although testing time
devoted to assessing both types of skills is 180 minutes (see Table 4.2).
Most of the reading passages on the GHSGT are fiction (45%), written in a
narrative style (73%), and presented as an essay (55%) (see Table 4.4). The GHSGT
reading items assess inference skills (70%) most frequently, although a moderate
proportion of questions also assess recall skills (30%) (see Table 4.5).
Table 4.5 Alignment Within the Cognitive Demands Category for Tests
Measuring Reading Skills
College Admissions Tests
Four college admissions exams assess reading proficiency: the ACT, AP
Language and Composition, SAT I, and SAT II Literature. With the exception of the AP
Language and Composition, no college admissions test assesses reading skills with open-
ended items. Testing time devoted to measuring reading skills is 60 minutes for both the
Test Recall Inference Evaluate Style
State Achievement Test
GHSGT 30 70 0
College Admissions Tests
ACT 58 42 0
AP Language and Composition 23 77 0
SAT I 18 83 0
SAT II Literature 13 80 7
College Placement Tests
COMPASS Reading 75 25 0
UOG ELA Placement Test 46 54 0
84
SAT II Literature and AP Language and Composition. Because the SAT I does not
contain separate sections for editing and reading items, we cannot determine testing time
earmarked specifically for assessing reading proficiency, although testing time devoted to
assessing both types of skills is 75 minutes (see Table 4.2).
Reading passage topics also vary from one measure to the next (see Table 4.4).
The SAT II Literature emphasizes fiction (63%) whereas the AP Language and
Composition emphasizes personal accounts (50%). The SAT I favors humanities (40%),
but the ACT is evenly distributed among fiction, humanities, natural science, and social
science (25% each). Narrative pieces are included on all college admissions measures,
and range from 40% of the SAT I passages to 100% of the SAT II Literature passages.
Essay is generally the most common genre, appearing on 75% of the ACT, 80% of the
SAT I, and 100% of the AP Language and Composition passages. However, the SAT II
Literature is more likely to include poems (50%) than essays (25%). With the exception
of the ACT, college admission exams place greatest emphasis on interpretation and
analysis of the reading passages. Inference items range from 42% of the ACT questions
to 83% of the SAT I questions (see Table 4.5).
College Placement Tests
Two college placement tests, the COMPASS Reading and the UOG ELA
Placement Test, contain reading items. Both assess reading proficiency with the
multiple-choice format. As with the GHSGT and SAT I, the UOG ELA Placement Test
does not specify testing time devoted specifically to measuring reading proficiency.
Because the COMPASS Reading is a computer-adaptive measure, testing time varies by
examinee (see Table 4.2).
Reading topics on the COMPASS Reading are distributed among humanities,
natural science, and social science (25% each), but reading topics on the UOG ELA
Placement Test are distributed between humanities and personal accounts (50% each)
(see Table 4.4). With respect to author’s voice, the COMPASS Reading favors
informative works (63%), whereas the UOG ELA Placement Test is evenly split between
narrative and informative pieces (50% each). Essay is the most commonly used genre on
either test (88%-100%). For the cognitive demands category, the COMPASS Reading
85
emphasizes recall to a great extent (75%), but the UOG ELA Placement Test focuses on
both recall and inference (46% and 54%, respectively) (see Table 4.5).
Alignment Among Tests of Different Functions
Across all measures, reading skills are assessed primarily with multiple-choice
items. Among the assessments that contain separate sections for reading, testing time
ranges from 35 minutes for the ACT to 60 minutes for the SAT II Literature and AP
Language and Composition. All assessments contain reading passages on two or more
topics, and every reading assessment includes a topic from humanities except the SAT II
Literature. Every measure also includes both narrative and informative passages. Essay
is the most prevalent genre, comprising 55% of the GHSGT, 88% -100% of college
placement, and 75%-100% of college admissions tests, the SAT II Literature exam
notwithstanding. Instead, the SAT II Literature favors poems. Both the GHSGT (70%)
and most of the college admissions measures focus heavily on inference questions (77%-
83%, excluding the ACT), whereas college placement measures place relatively greater
emphasis on recall (46%-75%).
Editing Measures
Alignment Among Tests of the Same Function
State Achievement Test
The GHSGT is the only measure that falls within this category. Again, we cannot
determine testing time devoted to assessing editing skills because editing and reading
items are combined within a single section (see Table 4.2). As with reading, editing
proficiency is assessed only with multiple-choice items. Most of the passages included
on the GHSGT editing section are fictional works (60%) in which the author uses a
narrative voice (80%). Story is the most common genre (60%), although a significant
number of passages are also presented as essays (40%) (see Table 4.6). With respect to
cognitive demands, inference skills are assessed most frequently (44%), although a
moderate proportion of items also assess recall (30%) and evaluate style (26%) skills.
86
Tab
le 4
.6
Alig
nmen
t W
ithi
n th
e C
onte
nt C
ateg
ory
for
the
Edi
ting
Pas
sage
s
Top
ic
V
oice
Gen
re
Tes
t
Fic
tion
H
uman
itie
s N
atur
al
Sci
ence
S
ocia
l S
cien
ce
Per
sona
l A
ccou
nts
N
arra
tive
Des
crip
tive
Per
suas
ive
Info
rmat
ive
L
ette
r E
ssay
P
oem
St
ory
Sta
te A
chie
vem
ent T
est
GH
SG
T
60
20
20
0
0
80
0 0
20
0
40
0 60
Col
lege
Adm
issi
ons
Tes
ts
AC
T
0
60
20
0 20
40
0 0
60
0
100
0 0
SA
T I
N/A
N/A
N/A
SA
T I
I W
riti
ng
0
100
0 0
0
50
0 0
50
0
100
0 0
Col
lege
Pla
cem
ent T
ests
CO
MP
AS
S W
riti
ng S
kill
s
0 0
50
50
0
0 0
0 10
0
0 10
0 0
0
UO
G E
LA
Pla
cem
ent T
est
N
/A
N
/A
N
/A
86
87
Table 4.7 Alignment Within the Cognitive Demands Category for Tests
Measuring Editing Skills Test Recall Inference Evaluate Style
State Achievement Test
GHSGT 30 44 26
College Admissions Tests
ACT 48 4 48
SAT I 0 100 0
SAT II Writing 50 3 47
College Placement Tests
COMPASS Writing Skills 80 3 17
UOG ELA Placement Test 100 0 0
College Admissions Tests
Items measuring editing skills are included on three college admissions tests, the
ACT, SAT I, and SAT II Writing. The exams are predominantly multiple-choice, with
testing time ranging from 40 minutes for the ACT to 45 minutes for the SAT II Writing
(see Table 4.2). As mentioned earlier, the SAT I does not specify the specific amount of
testing time devoted to measuring editing skills.
The SAT I does not include a reading passage, but instead uses a few sentences as
item prompts. In contrast, the ACT and SAT II Writing include reading passages. These
reading passages are typically essays about humanities, and written in either a narrative
or informative voice (see Table 4.6). The ACT and SAT II Writing items are equally
distributed among recall (48% and 50%, respectively) and evaluate style items (48% and
47%, respectively), but the SAT I assesses only inference skills (100%) (see Table 4.7).
College Placement Tests
Two college placement tests, the COMPASS Writing Skills and UOG ELA
Placement Test, assess editing skills via multiple-choice items. Neither test specifies the
time limit devoted solely to measuring editing skills (see Table 4.2). The UOG ELA
Placement Test uses sentences to assess editing skills, but the COMPASS Writing Skills
includes passages. Topics on the COMPASS Writing Skills are evenly split between
88
natural science and social science (50%), and passages are typically essays (100%) in
which the author uses an informative voice (100%) (see Table 4.6). In terms of cognitive
demands, both tests emphasize recall items (80%-100%).
Alignment Among Tests of Different Functions
Across all measures, editing skills are assessed solely with multiple-choice items.
All tests, except the SAT I and UOG ELA Placement Test, include reading passages.
(The SAT I and UOG ELA Placement Test use sentences as prompts). Of those
measures that include a reading passage, all include an essay, written in a narrative or
informative voice. More variation is observed with respect to reading passage topics.
The GHSGT favors fiction (60%) whereas humanities topics are most likely to appear on
the ACT or SAT II Writing (60% and 100%, respectively). In contrast, natural science
and social science are the most frequent topics on the COMPASS Writing Skills (50%
each topic).
Both the college placement and college admissions exams tend not to span the full
spectrum of the cognitive demands category. The ACT and SAT II Writing emphasize
recall and evaluate style items, but are generally devoid of inference items, whereas the
reverse is true for the SAT I. College placement measures focus heavily on recall items
(80%-100%) and rarely include inference and evaluate style items. In contrast, the
GHSGT assesses all three levels of the cognitive demands category; the test is almost
evenly distributed among recall, inference, and evaluate style items (30%, 44%, and 26%,
respectively).
Writing Measures
Alignment Among Tests of the Same Function
State Achievement Tests
The GHSWT requires students to provide one writing sample within a 90-minute
testing session (see Table 4.2). Topics are typically drawn from humanities or personal
accounts (see Table 4.8). With respect to scoring criteria, GHSWT requires students to
demonstrate mechanics, word choice, style, organization, and insight (see Table 4.9).
89
Table 4.8 Alignment Among the Writing Prompt Topics
Table 4.9 Alignment Among the Scoring Criteria for Tests Measuring Writing Skills
College Admissions Tests
Of the college admissions measures, only the SAT II Writing and AP Language
and Composition require a writing sample. The SAT II Writing provides students with a
one- or two-sentence writing prompt on a topic (usually humanities), and allows 20
minutes for students to respond (see Tables 4.2 and 4.8). In contrast, prompts on the AP
Language and Composition are typically reading passages, and students are required to
provide a total of three writing samples in over two hours (see Table 4.2).2 Topics can
2 The AP Language and Composition requires a total of three writing samples, two of which are produced during the 120-minute writing session, and one during the 60-minute reading session. However, because examinees also respond to a set of multiple-choice items during the reading session, it is unknown the amount of time students devote specifically to the writing sample.
Topic
Test Fiction Humanities Natural Science
Social Science
Personal Accounts
State Achievement Test
GHSWT X X
College Admissions Tests
AP Language and Composition X X
SAT II Writing X
College Placement Test
UOG ELA Placement Test X X X X
Scoring Criteria Elements Test
Mechanics Word Choice Organization Style Insight
State Achievement Test
GHSWT X X X X X
College Admissions Tests
AP Language and Composition X X X X X
SAT II Writing X X X X
College Placement Test
UOG ELA Placement Test X X X X X
90
vary, but are usually about humanities or personal accounts (see Table 4.8). The AP
Language and Composition emphasizes all elements of the scoring criteria, but the SAT
II Writing downplays the importance of insight (see Table 4.9).
College Placement Tests
Only one college placement exam, the UOG ELA Placement Test, requires a
written composition. Students are to write a single composition within a 60-minute
session (see Table 4.2). Topics vary widely from humanities and natural science to social
science and personal accounts (see Table 4.8). The scoring rubric emphasizes mechanics,
word choice, organization, style, and insight.
Alignment Among Tests of Different Functions
Writing measures can vary from 20 minutes for a single writing sample (SAT II
Writing) to over 2 hours for three writing samples (AP Language and Composition).
Humanities and personal accounts are the most common topic, as every test includes a
writing prompt from one, if not both, of these areas. All but one writing test emphasizes
mechanics, word choice, organization, style, and insight. The exception is the SAT II
Writing, which omits insight from its scoring criteria.
Discussion
Our discussion of the discrepancies among ELA assessments parallels that of the
math discussion. We first identify examples of discrepancies that are justifiable, then
discuss the implications of the misalignments. We also discuss the feasibility of using
state achievement tests to inform admissions decisions.
Which Discrepancies Reflect Differences in Test Use?
As in math, some discrepancies among the ELA assessments reflect differences in
purpose. Consider, for instance, discrepancies between the scoring standards of the
GHSWT and the AP Language and Composition. For the former test, maximum scores
are awarded to writing samples that have minor diction errors, mechanics lapses, and
underdeveloped paragraphs. Under the AP Language and Composition guidelines, such
91
compositions might receive adequate scores, but would not be viewed as exemplary
papers. Because the AP Language and Composition is used to award academic credit to
students who demonstrate college-level proficiency, whereas the GHSWT is used to
monitor the achievement of all students within the state, including those not planning to
attend a postsecondary institution, discrepancies between their scoring criteria are
warranted.
Even when two measures have similar test functions, discrepancies may still be
warranted. For example, the large discrepancy between the SAT I (100%) and the ACT
(4%) and the SAT I and SAT II Writing (3%) with respect to inference items is
attributable to subtleties in purpose. The SAT I is intended to be a measure of reasoning
proficiency, so great emphasis on inference questions is justifiable. The ACT and SAT II
Writing, on the other hand, are curriculum-based measures, so relatively greater focus on
skills learned within English classes (i.e., recall and evaluate style skills) is to be
expected.
Is There Evidence of Misalignment?
Although the majority of the ELA discrepancies is small or moderate, or stems
from variations in test function, one instance of misalignment pertains to the scoring
criteria of the SAT II Writing. Insight is included within the scoring criteria of the
GHSWT, AP Language and Composition, and UOG ELA Placement test, but is omitted
from the scoring rubrics of the SAT II Writing. Given that insight is included in the
standards of most English courses, it appears that the SAT II Writing standards are
incongruent with those that are typically expressed. Potentially, this misalignment can
send students mixed messages about the importance of insight with respect to writing
skills. If the developers of the SAT II Writing were to add insight to the scoring criteria,
or provided a clear rationale of why insight has been omitted from the scoring rubrics,
students would receive a more consistent signal about the importance of insight with
respect to writing proficiency.
92
Can State Achievement Tests Inform Postsecondary Admissions Decisions?
As mentioned earlier, policymakers are exploring the possibility that scores on
graduation tests can be used to inform college admissions decisions. In reading, that the
GHSGT assesses inference skills to approximately the same extent as college admission
measures may suggest that the GHSGT might be a viable alternative to college
admissions tests. However, inference items can vary with respect to cognitive
sophistication elicited. A previous study by Education Trust (1999) showed that ELA
inference items could vary greatly with respect to nuance of interpretations. Given the
differences in the intended test uses, it is very likely that inference items on the GHSGT
may not be as complex as that elicited by college admissions exams. More research
needs to be conducted to determine whether the GHSGT can discriminate among higher-
achieving examinees as well as college admissions exams.
With respect to assessing writing skills, the GHSWT holds more promise as an
alternative to college admissions tests. Neither the ACT nor the SAT I requires a writing
sample, and the SAT II Writing allows 20 minutes for a writing sample. Given the short
time limit, the SAT II Writing composition represents a very limited indicator of writing
proficiency. The GHSWT, on the other hand, allows students 90 minutes for a
composition. Arguably, the GHSWT would allow admissions officers to better judge
applicants’ writing skills than the ACT, SAT I, or SAT II Writing. However, as
discussed earlier, the current scoring rubrics for the GHSWT may not be rigorous enough
to be of use for some institutions, especially the selective ones. Therefore, changes to the
scoring guidelines may need to be implemented if the GHSWT writing samples were to
be used to inform admissions decisions at these higher-selectivity schools. Again, any
policy changes regarding the use of the GHSWT to inform admissions decisions will
require more research, particularly the relationship between the GHSWT scores and first-
year college grade point average.