applying principles of universal design to assessment item modification peter a. beddow iii...
TRANSCRIPT
CAAVES Item Modification 2008 1
Applying Principles of Universal Design to Assessment Item Modification
Peter A. Beddow III Vanderbilt University
Nashville, TN
June 2008
CAAVES Item Modification Study 2008 2
Background
Definition of Universal Design: The design of products and
environments to be usable by all people, to the greatest extent possible, without the need for adaptation or specialized design.
(Center for Universal Design, NC State University, 2008; http://www.design.ncsu.edu/cud/about_ud/udprincipleshtmlformat.html#top)
CAAVES Item Modification Study 2008 3
Universal Design (cont.)
Equitable use Flexibility in use Simple and intuitive Perceptible information Tolerance for error Low physical effort Size and space for approach
and use
(Center for Universal Design, 2008)
CAAVES Item Modification Study 2008 4
Universal Design (cont.)
Principles of Universal Design are intended to maintain equity among user groups while providing the same or equivalent means of use for all users;
Ideally, services, products, or systems employing universal design principles: Eliminate unecessary complexity; Arrange information consistent with its importance; Use prompting and feedback during and after task completion; Present information redundantly (visual, verbal, tactile); Maximize legibility and construct; Simplify directions and instructions; Reduce the potential for error.
Item Accessibility and Modification Guide
An Instrument for Designing and Modifying Assessment Items
CAAVES Item Modification Study 2008 6
Purpose of the Instrument
The Item Accessibility and Modification Guide (Beddow, Kettler, & Elliott, 2007) was designed to facilitate the analysis of test item features to enhance meaningful responses from, and provide equal access for, all students;
The instrument can be used: To aid in examination and modification of existing
items; To influence the design of new items; or As a training tool for item writers.
CAAVES Item Modification Study 2008 7
Influence
The Item Accessibility and Modification Guide was directly influenced by National Center for Educational Outcomes publications, specifically:
NCEO’s Policy Directions 18 (Johnstone, Thurlow, Moore, & Altman, 2006) which provides expert review recommendations for test item.
Section 508 guidelines.
To date there has been no effort to provide a rating instrument to assist in the consideration of universally designed assessment items.
CAAVES Item Modification Study 2008 8
Organization
The Item Accessibility and Modification Guide consists of two sections:
Considerations for Universally Designed Assessment Items (7 categories, 38 items);
Computer-based Tests: Considerations for Universally Designed Items (5 categories, 26 items).
CAAVES Item Modification Study 2008 9
Procedure
The rater is asked to record the item being reviewed, his/her name, and a brief description of the construct the item is intended to measure.
For each accessibility item, the rater is asked to check Yes or No (or N/A if the question is not applicable to the target item).
For each category, the rater is asked to circle the number that best represents his/her summative evaluation of the item’s accessibility for that category based on checklist responses:
0 = The item is Not Accessible 1 = The item is Minimally Accessible 2 = The item is Moderately Accessible 3 = The item is Maximally Accessible
The Total Item Accessibility Rating is found by totalling overall evaluation ratings for the item across categories.
It should be noted that the overall rating is intended as a simple way to categorize items by high, medium, or low accessibility, as opposed to providing a definitive indicator of accessibility that is based on research data.
CAAVES Item Modification Study 2008 10
Considerations for Universally Designed Assessment Items
7 categories:
Content Relevance (2 items); Text Content (10 items); Text Economy/Conciseness (3 items); Text Appearance (7 items); Visual Content (if applicable; 7 items); Fairness for Subgroups (8 items); and Format Flexibility (5 items).
CAAVES Item Modification Study 2008 11
Considerations for Universally Designed Assessment Items
CAAVES Item Modification Study 2008 12
Considerations for Computer-based Test Items
5 categories:
Layout and Design (6 items); Navigation (5 items); Screen Reader Considerations (5 items); Test Specific Options (4 items); and Computer Capabilities (6 items).
CAAVES Item Modification Study 2008 13
Considerations for Computer-based Test Items
CAAVES Item Modification Study 2008 14
The Item Accessibility and Modification Guide
The instrument is available electronically at:
http://www.vanderbilt.edu/lsi/caavespage/Item_Accessibility_and_Modification_Guide.pdf
Manuscript:
http://www.vanderbilt.edu/lsi/caavespage/Item_Accessibility_and_Modification_Guide_Manuscript.pdf
