Psychometric Issues in the Use of Testing Accommodations

Chapter 4

David Goh

Testing Accommodations and Psychometric Soundness

• Federal and state laws require that nonbiased procedures be used in assessment

• But does using testing accommodations make the technical aspects of the tests questionable?– Reliability: accuracy– Validity: effectiveness, consistency and

stability of scores

Standardization

• Standardization implies uniformity of procedures in administering a test and scoring the results

• Poses a problem for diverse learners who exhibit different physical, sensory, linguistic, cultural and or psychological qualities from the general population

• In theory all accommodations compromise standardization, so issues of validity and reliability need to be examined

Reliability

• Refers to consistency and stability• High reliability produces consistent results • Low reliability produces inconsistent results• Ex- ELLs using a bilingual interpreter may make different

interpretations• Accommodations and modifications may increase

measurement errors and decrease reliability• Methods of examining reliability include comparing

scores of the same test takers on different administration, with different sets of times, or with different scores or examiners

Validity

• Refers to effectiveness- is the test testing what it intended

• If used right- accommodations can increase validity

• If used inappositely, accommodations may nullify the results

• Types: content, response process, internal structure, other variables, and consequences of testing

Effect of Accommodations

• Limited evidence on the effect of test accommodations on reliability and validity

• Studies have shown that testing accommodations on SAT and GRE did not significantly effect reliability and validity– However, findings are not generalizable– But, some accommodations have been found

to increase student performance- ex- extended time

Flagging Test Scores

• On large scale test a very controversial issue– Reveals examinee has a disability; prompts invalid

inferences about score and prejudice; identifies and violates person with disabilities privacy that is protected under the law

• If comparability can be demonstrated between standard and non-standard administration, there is no need to flag– However, if test results are not comparable, then

testing accommodations need to be noted and explained