an exploration of ged standard score stability · 2013-08-02 · an exploration of ged® standard...

An Exploration of GED Standard

Score Stability: 2001 Through 2005

GED Testing Service Research Studies, 2006-3

General Educational Development Testing Service of the American Council on Education

© 2006

American Council on Education ACE, the American Council on Education, and GED are registered marks of the American Council on Education.

GED® Testing Service

One Dupont Circle NW, Suite 250 Washington, DC 20036-1163 (202) 939-9490 Fax: (202) 659-8875 www.gedtest.org

All rights reserved. No part of this book may be reproduced or transmitted in any form or by any means electronic or mechanical, including photocopying, recording, or by any information storage and retrieval system, without permission in writing from the publisher.

Standard Score Stability 1

Running head: GED® STANDARD SCORE STABILITY

An Exploration of GED® Standard Score Stability: 2001 Through 2005

G. Anthony Benners

Fordham University

Carol E. George-Ezzelle

GED® Testing Service of the American Council on Education


An Exploration of GED® Standard Score Stability: 2001 through 2005

The present investigation was aimed at exploring the stability of the standard score

distributions on the GED Tests taken by U.S. high school seniors in equating studies

conducted by GED Testing Service during the span of 5 years from 2001 (the norming year)

to 2005. Three questions were addressed by this exploration: First, are the characteristics of

the norming sample representative? Second, have self-reported characteristics of high school

seniors changed since 2001, the year of standardization/norming of the new test series?

Third, are there any trends in test performance during these years that suggest high school

seniors are mastering the content of the GED Tests or are otherwise becoming ignorant of the

content?

High School Senior Characteristics

The chief demographic variable used to address the question of population

characteristics was self-reported ethnicity. Figure 1 displays the population estimates for

ethnic breakdown of U.S. high school students who graduated during the 2001-2002 school

year (National Center for Education Statistics [NCES], 2005). Whites were the largest group

at 69% of the total number of graduates, followed by African Americans at 13%. Hispanics

made up 12% of the total graduates whereas other ethnicities including Asians comprised

only 6%. As for the ethnic breakdown of the GED Tests equating samples, Figures 2 through

6 display the percentage distributions of ethnic categories across the 5 years for each GED

Test: Language Arts, Writing; Social Studies; Science; Language Arts, Reading; and

Mathematics, respectively. With few exceptions, the ethnic composition of the equating

samples is in line with the NCES estimates. In 2004, the percentage of Whites was higher

than national estimates for the Social Studies and Math Tests, and the percentage of Hispanic


and African American were lower than national estimates. Nevertheless, ethnic diversity was

achieved in all the equating samples, and they appear to be representative of the target

population.

Another variable considered in the present study was the self-reported grades for

classroom subject areas that corresponded to test content, e.g., English composition grades

for those who took the Writing Test were examined to determine whether grade percentage

distributions had changed over the 5 years considered. Figures 7 through 11 display the self-

reported grades across for English composition, social studies, science, English literature,

and math courses, respectively. Comparison of the grade distributions of 2001 and 2002 to

2003 through 2005 indicate a greater percentage of reported letter grade A, a greater

percentage leaving the question blank, and a lower percentage of reported letter grade B in

the later years, particularly in 2004.

GED® Test Performance

Performance on each test was measured by average standard scores and pass rates

using a cutoff score of 410. Figures 12, 14, 16, 18, and 20 display a line graph of the average

standard scores over time on the Writing, Social Studies, Science, Reading, and Math Tests,

respectively. Average standard scores for the U.S. candidates are also displayed (indicated by

a dashed line) starting with 2002, the year the new series became operational. Error bars at

each point represent 95% confidence intervals. Sample sizes and forms taken by the high

school seniors are displayed just above the x-axis. An alternate forms study was conducted in

2004; all other years involved equating study designs. Individuals who did not respond to

more than one third of the questions or did not correctly answer any questions on a test were


excluded from the analyses. Pass rates for the five tests are displayed in Figures 13, 15, 17,

18, and 21.

Average performance on all tests but the Language Arts, Writing Test decreased from

2001 to 2002; for three of the tests—Social Studies, Language Arts, Reading, and

Mathematics—the average performance never achieved the level observed in 2001; for the

Science Test, the year 2004 was the only year in which the average performance surpassed

(by 4 standard score points) the performance observed in the norming year 2001.

Language Arts, Writing Test

The average standard score for the Language Arts, Writing Test was fairly consistent

across years except for an increase in 2004. The pass rate ranged from 77% to 79% except

for in 2003 when it fell to 72%. Grade distributions were examined to explore the increase in

average score in 2004 and the decrease in pass rate for 2003. The grade distribution was

consistent for the years 2003, 2004, and 2005; differences in average scores and pass rates

are not easily explained by differences in grade distributions. A comparison of essay score

distributions in 2002 and 2004 for Forms D, E, and F (Table 22) revealed a greater

percentage of essay scores at the 3 and 4 levels in 2004 than in 2002 (the same forms were

administered those years); an increase of 1 point on an essay translates to approximately 40

standard score points.

Social Studies Test

The average standard scores for the Social Studies Test in 2002-2005 was

consistently and statistically significantly lower than in 2001. Pass rates for the Social

Studies Test dropped from 82% in 2001 and from 68% to 71% in the years 2002 to 2005.

Differences in self-reported grades were more substantial in 2001-2002 vs. 2003-2005;


differences in average scores and pass rates are not easily explained by differences in grade

distribution.

Science Test

For the Science Test, average standard scores dropped in 2002 and then further

dropped (32 points) in 2003. The pass rate was at 82% for 2001 and then ranged from 61% to

79%. Grade distributions were similar for 2001 and 2002, and the years 2003 and 2005; in

2004 vs. other years, a greater percentage of high school seniors reported having earned

mostly letter A grades in their science courses.

Language Arts, Reading Test

The average standard score for the Language Arts, Reading Test decreased from 2001

to 2002 and were stable for the years of 2002 through 2005. This decrease is mirrored by the

decrease in the pass rate from 2001 to 2002 from 83% to 73%. The pass rate remained steady

at 78% 2003 to 2005.

Mathematics Test

For the Mathematics Test, average standard scores decreased in 2002 and 2003, but

the scores rose to above 500 in 2004. Pass rates display the same fluctuation over time. The

pass rate decreased from 83% in 2001 to a range of 72% to 74% in years 2002 and 2003, and

they then increased to 81% before dropping to 76% in 2005.

Distribution of Observed GED® Standard Scores

Estimates of standard scores associated with the 90%, 75%, 50%, 25%, and 10%

quartiles from 2001 to 2005 were calculated. The results were plotted along with the normal

distribution (N.D.) expectation, which was the distribution imposed during the scaling.

Overall, the percentage of scores observed in one or both of the tails of the standard score


distributions varied not only from the normal distribution imposed during scaling but also

varied from the distribution observed during the year of norming.

Language Arts, Writing Test

The Language Arts, Writing Test showed signs of the standard score distribution

bunching at the lower end of the score scale. For example, instead of the 90th percentile at

630 under a normal distribution, the observed standard score at the 90th percentile ranged

from 650 to 690. The 75th percentile under N.D. would be at 570; for 2 years it was observed

at 590 and 620. The 10th percentile under N.D. would be at 370; for 3 years it was observed

at 400.

Social Studies Test

The Social Studies Test showed signs of the distribution shifting towards the lower

end of the score scale. For example, the 75th percentile under N.D. would be at 570; for the 5

years it was observed ranging from 540 to 560. The 25th percentile under N.D. would be at

430; for 2002-2005 it was observed ranging from 380-390.

Science Test

The Science Test showed signs of the distribution bunching at the low end of the

score scale. For example, the 25th percentile under N.D. would be at 430; for 2002-2005 it

was observed at 360-420. The 10th percentile under N.D. would be at 370; for 2002-2005 it

was observed ranging from 310-350.

Language Arts, Reading Test

The Language Arts, Reading Test showed signs of the distribution flattening. For

example, the 90th percentile under N.D. would be at 630; for 2001-2005 it was observed at

660-720.


Mathematics Test

The Mathematics Test showed signs of the distribution bunching at the low end of the

score scale for some years.

Summary

The following summarizations may be made based on the results of this study. Based

on the similarity of the ethnicity distributions within the equating samples and national

estimates, it appears that the equating samples were representative of the U.S. population of

high school seniors. Average standard scores on the GED Tests over the years considered in

this study do not suggest that high school seniors are mastering the content of the tests;

however, there does appear to be a pattern in standard scores fluctuations. All tests reflected

a drop in average performance from 2001 (the norming year) to 2002. Differences in self-

reported grade distributions may explain the increase in the average Science Test score in

2004 but not the increase in 2003. One reason for the increase in average performance for

nearly all tests in 2004 could be the ease of administration for the alternate forms study. In

summary, both the differences in the percentages of standard scores observed in one or both

of the tails of each test’s 2002-2005 standard score distributions from that observed during

the year of norming and the variability of the performance over time on several tests warrant

a renorming study.


Ethnicity for H.S. Seniors 2001-2002

69%

13%

12%6%

White, non-HispanicBlack, non-HispanicHispanic

Other

Figure 1. Population estimates (National Center for Education Statistics, 2005).

PERCENT ETHNICITY OVER TIME

67%

15%10%

6%

1%

68%

17%

9%5%

1%

72%

12%10%

6%

1%

74%

11%7% 7%

1%

67%

9%13%

6% 5%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

Whi

te

Afri

can

His

pani

c

Oth

er

mis

sing

Whi

te

Afri

can

His

pani

c

Oth

er

mis

sing

Whi

te

Afri

can

His

pani

c

Oth

er

mis

sing

Whi

te

Afri

can

His

pani

c

Oth

er

mis

sing

Whi

te

Afri

can

His

pani

c

Oth

er

mis

sing

PER

CEN

T

WRITING

20022001 20042003 2005

Figure 2. Ethnicity distribution for high school seniors taking the Language Arts, Writing Test.



69%

13%10%

7%

1%

68%

14% 12%

6%1%

68%

13% 11%

6%

1%

74%

11%9%

6%

1%

68%

11%13%

6%1%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%W

hite

Afri

can

His

pani

c

Oth

er

mis

sing

Whi

te

Afri

can

His

pani

c

Oth

er

mis

sing

Whi

te

Afri

can

His

pani

c

Oth

er

mis

sing

Whi

te

Afri

can

His

pani

c

Oth

er

mis

sing

Whi

te

Afri

can

His

pani

c

Oth

er

mis

sing

PER

CEN

T

SOCIAL STUDIES

20022001 20042003 2005

Figure 3. Ethnicity distribution for high school seniors taking the Social Studies Test.



67%

12% 13%

7%

1%

67%

16%11%

5%

0%

69%

13% 11%

6%

1%

71%

12%8% 7%

3%

69%

11%13%

6%1%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%W

hite

Afri

can

His

pani

c

Oth

er

mis

sing

Whi

te

Afri

can

His

pani

c

Oth

er

mis

sing

Whi

te

Afri

can

His

pani

c

Oth

er

mis

sing

Whi

te

Afri

can

His

pani

c

Oth

er

mis

sing

Whi

te

Afri

can

His

pani

c

Oth

er

mis

sing

PERC

ENT

SCIENCE

20022001 20042003 2005

Figure 4. Ethnicity distribution for high school seniors taking the Science Test.



66%

16%

10%7%

2%

70%

13%11%

5%2%

68%

13% 12%

6%

1%

72%

12%9%

5%2%

68%

12%13%

6%2%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%W

hite

Afri

can

His

pani

c

Oth

er

mis

sing

Whi

te

Afri

can

His

pani

c

Oth

er

mis

sing

Whi

te

Afri

can

His

pani

c

Oth

er

mis

sing

Whi

te

Afri

can

His

pani

c

Oth

er

mis

sing

Whi

te

Afri

can

His

pani

c

Oth

er

mis

sing

PERC

ENT

READING

20022001 20042003 2005

Figure 5. Ethnicity distribution for high school seniors taking the Language Arts, Reading Test.



0% 0% 0% 0% 0%

13%11%

5%1%

71%

12%9%

6%

1%

76%

10%7% 7%

1%

68%

9%13%

6% 4%

70%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%W

hite

Afri

can

His

pani

c

Oth

er

mis

sing

Whi

te

Afri

can

His

pani

c

Oth

er

mis

sing

Whi

te

Afri

can

His

pani

c

Oth

er

mis

sing

Whi

te

Afri

can

His

pani

c

Oth

er

mis

sing

Whi

te

Afri

can

His

pani

c

Oth

er

mis

sing

PERC

ENT

MATH

20022001 20042003 2005

Figure 6. Ethnicity distribution for high school seniors taking the Mathematics.


PERCENT GRADES OVER TIME

31%

47%

19%

3%

0% 0%

29%

48%

20%

2%0% 0%

33%

39%

17%

2%0%

9%

34%37%

14%

3%

0%

11%

40%

31%

14%

3%2%

10%

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

A B C D F M A B C D F M A B C D F M A B C D F M A B C D F M

PER

CEN

T

ENGLISH COMPOSITION

20022001 20042003 2005 Figure 7. Percentage of high school seniors indicating overall grade in English Composition.



31%

44%

21%

3%

0% 0%

36%

44%

18%

2%0% 0%

34%

39%

19%

2%0%

5%

34%

38%

18%

2%0%

7%

36%

41%

17%

2%0%

4%

0%

10%

20%

30%

40%

50%


PER

CEN

T

SOCIAL STUDIES

20022001 20042003 2005

Figure 8. Percentage of high school seniors indicating overall grade in Social Studies.



25%

43%

27%

4%

1% 0%

24%

45%

28%

3%

0% 0%

26%

42%

23%

4%

0%

5%

30%

37%

23%

3%0%

6%

26%

45%

23%

3%

0%

4%

0%

10%

20%

30%

40%

50%


PER

CEN

T

SCIENCE

20022001 20042003 2005

Figure 9. Percentage of high school seniors indicating overall grade in Science.



27%

43%

27%

3%

0% 0%

28%

45%

23%

3%

0% 0%

27%

41%

23%

3%

0%

5%

32%

37%

21%

4%

0%

6%

31%

42%

21%

3%

0%

3%

0%

10%

20%

30%

40%

50%


PER

CEN

T

ENGLISH LITERATURE

20022001 20042003 2005

Figure 10. Percentage of high school seniors indicating overall grade in English Literature.



0% 0% 0% 0% 0% 0%

28%

39%

26%

6%

0% 0%

24%

37%

28%

7%

0%

4%

32%

35%

23%

5%

0%

4%

28%

38%

25%

5%

0%

3%

0%

10%

20%

30%

40%

50%


PER

CEN

T

MATH

20022001 20042003 2005

Figure 11. Percentage of high school seniors indicating overall grade in Mathematics.


Average Standardized Score by Year (Writing)

501

479474 478

489 493

480

512

493

400410420430440450460470480490500510520530540550560570580590600

2000 2001 2002 2003 2004 2005 2006

YEAR

Mea

n St

anda

rdiz

ed S

core

U.S. Candidates H.S. Seniors (full sample)

1076 1133 1375 1077 2514Frequencies ~>

A,B,C C.D,E,F C,G,H,I D,E,F C,J,KFORM S ~>

-32, t (2450)= 6.61 p<.001

Figure 12. Average Language Arts, Writing Test standard scores.

PASS/FAIL RATE (WRT)

22.2 23.3 27.6 20.6 22.6

77.8 76.7 72.4 79.4 77.4

0.0

20.0

40.0

60.0

80.0

100.0

2001 2002 2003 2004 2005YEAR

PER

CEN

T

FAIL PASS

Figure 13. Pass rates for the Language Arts, Writing Test.


Average Standardized Score by Year (Social Studies)

498

471 470465

479

542

508

521

506

400410420430440450460470480490500510520530540550560570580590600

2000 2001 2002 2003 2004 2005 2006

YEAR

Mea

n St

anda

rdiz

ed S

core

U.S. H.S. Seniors U.S. Candidates

1371 1488 1969 1023 2562Frequencies ~>

A,B,C A,D,E,F A,G,H,I D,E,F A,J,KFORMS ~>

Figure 14. Average Social Studies Test standard scores.

PASS/FAIL RATE (SOC)

18%30% 32% 32% 29%

82%70% 68% 68% 71%

0%

20%

40%

60%

80%

100%

2001 2002 2003 2004 2005YEAR

PER

CEN

T

FAIL PASS

Figure 15. Pass rates for the Social Studies Test.


Average Standardized Score by Year (Science)

501

486

454

505

478

544

523515

509

400410420430440450460470480490500510520530540550560570580590600

2000 2001 2002 2003 2004 2005 2006

YEAR

Mea

n St

anda

rdiz

ed S

core


536 997 1936 950 2512Frequencies ~>A,B,C C.D,E,F C,G,H,I D,E,F C,J,KFORMS ~>

Figure 16. Average Science Test standard scores.

PASS/FAIL RATE (SCI)

18% 27%39%

21% 29%

82% 73%61%

79% 71%

0%

20%

40%

60%

80%

100%

2001 2002 2003 2004 2005YEAR

PER

CEN

T

FAIL PASS

Figure 17. Pass rates for the Science Test.


Average Standardized Score by Year (Reading)

518

495

507516

510

566

536544

526

400410420430440450460470480490500510520530540550560570580590600

2000 2001 2002 2003 2004 2005 2006

YEAR

Mea

n St

anda

rdiz

ed S

core


1266 2026 1958 1286 2582Frequencies ~>


Figure 18. Average Language Arts, Reading Test standard scores.

PASS/FAIL RATE (RDG)

17% 27% 22% 22% 22%

83% 73% 78% 78% 78%

0%

20%

40%

60%

80%

100%

2001 2002 2003 2004 2005YEAR

PER

CEN

T

FAIL PASS

Figure 19. Pass rates for the Language Arts, Reading Test.


Average Standardized Score by Year (Math)

512

486481

502495

503

463467 469

400410420430440450460470480490500510520530540550560570580590600

2000 2001 2002 2003 2004 2005 2006

YEAR

Mea

n St

anda

rdiz

ed S

core


744 1902 2548 1085 2649Frequencies ~>


Figure 20. Average Mathematics Test standard scores.

PASS/FAIL RATE (MATH)

17% 26% 28% 19% 24%

83% 74% 72% 81% 76%

0%

20%

40%

60%

80%

100%

2001 2002 2003 2004 2005YEAR

PER

CEN

T

FAIL PASS

Figure 21. Pass rates for the Mathematics Test.


ESSAY SCORE DISTIBUTION

11.2%

4.7%

40.6%

8.8%

21.7%

5.5%7.4%

11.0%

3.0%

32.7%

8.0%

28.1%

6.5%

10.7%

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

1 1.5 2 2.5 3 3.5 4

ESSAY SCORE

PER

CEN

TYR: 2002, AVG: 493 YR: 2004, AVG: 512

Figure 22. Comparison of essay score distributions in 2002 and 2004.


References

National Center for Education Statistics (2005). Digest of Education Statistics.

Washington, DC.

GED® Testing Service

One Dupont Circle NW, Suite 250 Washington, DC 20036-1163

(202) 939-9490 Fax: (202) 659-8875 www.gedtest.org

an exploration of ged standard score stability · 2013-08-02 · an exploration of ged® standard...

Documents