Modified Achievement Testsfor Students with Disabilities:Basic Psychometrics and Group Analyses

Ryan J. KettlerVanderbilt University

CCSSO’s National Conference on Student AssessmentJune 2008

Reliability and Concurrent Validity with a Proficiency Tests

CAAVES Item Mod Data 2008 2

Concurrent Validity with Participation Criteria

CAAVES Item Mod Data 2008 3

CAAVES Item Mod Data 2008 4

Reading: Descriptive Statisticsfor Unmodified and Modified Tests

Distributions by Group and Condition in Reading

CAAVES Item Mod Data 2008 5

Students without Disabilities, Ineligible

Students with Disabilities, Ineligible

Students with Disabilities, Eligible

Effect Sizes of Modifications on Reading Performance

CAAVES Item Mod Data 2008 6

Group Improvement by Condition in Reading

CAAVES Item Mod Data 2008 7

Mean Item Difficulty by Group and Condition in Reading

CAAVES Item Mod Data 2008 8

* Student abilities were equated using a Rasch model.

CAAVES Item Mod Data 2008 9

Math: Descriptive Statistics forUnmodified and Modified Tests

Distributions by Group and Condition in Math

CAAVES Item Mod Data 2008 10

Students without Disabilities, Ineligible

Students with Disabilities, Ineligible

Students with Disabilities, Eligible

Effect Sizes of Modifications on Math Performance

CAAVES Item Mod Data 2008 11

Group Improvement by Condition in Math

CAAVES Item Mod Data 2008 12

Mean Item Difficulty by Group and Condition in Math

CAAVES Item Mod Data 2008 13

* Student abilities were equated using a Rasch model.

Upcoming Analyses: Difficulty Change on Individual Items

CAAVES Item Mod Data 2008 14

Students without Disabilities, Ineligible

Students with Disabilities, Eligible

Conclusions

Assessments can be made more accessible for eligible students (with disabilities who have persistent academic difficulties).

Modifications can change the difficulty of tests and items without invalidating the resulting scores.

Much more work needs to be done!CAAVES Item Mod Data 2008 15