Warlick, Quenemoen, Rigney 2004
Options for including students with learning disabilities in assessments for
school accountabilityKen Warlick, University of Kentucky
Rachel Quenemoen, University of Minnesota
Sue Rigney, U.S. Department of Education
Warlick, Quenemoen, Rigney 2004
Objective
• Increase awareness of implications of federal assessment and accountability requirements on students with learning disabilities
• Increase awareness regarding accommodations
• Increase awareness regarding alternate assessments
• Increase awareness on emerging trends in assessment
Warlick, Quenemoen, Rigney 2004
Purpose of No Child Left Behind
“…to ensure that all children have a fair, equal, and significant opportunity to obtain a high-quality education and reach, at a minimum, proficiency on challenging State academic achievement standards and state academic assessments”
Warlick, Quenemoen, Rigney 2004
Purpose of Assessment Requirements of IDEA
• Improve results for student with disabilities through improved teaching and learning
• Raise expectations for students with disabilities
• Increase access to the general curriculum
• Provide parents information about their child’s achievement in relationship to the performance of other children in their school
Warlick, Quenemoen, Rigney 2004
Issues in both NCLB and IDEA
• Students with disabilities previously exempted from assessment and accountability system
• Students with disabilities previously received instruction in separate curriculum
• Change from low to high expectations for students with disabilities
• State leadership in fostering school and district accountability
Warlick, Quenemoen, Rigney 2004
Assessment Options
• General assessment
• General assessment with accommodations (or modifications)
• Alternate assessment on grade level achievement standards
• Alternate assessment on alternate achievement standards
• Computer-based assessments
Warlick, Quenemoen, Rigney 2004
Accommodations: The National Picture
• Accommodations use is on the rise.• About 50% of students with learning disabilities
receive an accommodation during testing.• Evidence from experimental studies indicates that
some accommodations boost performance.• Most common accommodations are
– Small group administration– Read-aloud– Extended time
Warlick, Quenemoen, Rigney 2004
Definitional IssuesSome accommodations are considered to change the construct assessed, and others are viewed as “ok”
Okay Not Okay
Accommodation
Adaptation
Standard administration
Modification
Modification
Non-approved accommodation
Non-allowed accommodation
Non-standard administration
Know what terms mean in your state and district!
Warlick, Quenemoen, Rigney 2004
What Makes An Assessment Accommodation OK?
It is aligned with instructional accommodations, but is not an excuse not to teach
Student needs it to demonstrate knowledge and skills – or to participate in assessment
The consequences of using the accommodation have been carefully considered
OK is not defined by what the test publisher says (unless has research evidence)
Warlick, Quenemoen, Rigney 2004
Can states close achievement gap without paying attention to LD?
• Keep high incidence populations in school and in special education (LD 3 X as likely to drop out of school as non-disabled)
• LD comprises 50% of special education population• LD is a life-long disability. Student’s needs
change over time. They need learning and coping strategies but special education does not cure them
Warlick, Quenemoen, Rigney 2004
Research on Accommodations for LD Students – What Does It Tell Us?
• Accommodations most frequently studied are oral presentation (13) and extended time (10)
• Various research designs are used
• Many limitations in conducted research are identified
Warlick, Quenemoen, Rigney 2004
Issues:
• Tendency to allow too many accommodations, possibly reducing expectations for student learning.
• Poor decision making about accommodations, reflecting lack of knowledge about instructional accommodations.
• Use of accommodations as an excuse to exclude students scores on reports or accountability.
Warlick, Quenemoen, Rigney 2004
Promising Practices:
• States keeping track of how many students use accommodations, which ones, and their impact on achievement.
• Out-of-the-box thinking about the use of accommodated test scores.
• Clear decision-making criteria (e.g., alignment to instructional accommodations) and training on how to make decisions.
• Teaching high school students (at the latest) about the tests they take and about the accommodations that they need.
Warlick, Quenemoen, Rigney 2004
Implications for IEP teams
• Must be familiar with the state or district content/learning standards
• Must be familiar with the curriculum
• Must be familiar with typical instructional activities in general education
Warlick, Quenemoen, Rigney 2004
Alternate Assessments
Aligned with the State’s content standards.
Yield results separately in reading/language arts and math.
Designed and implemented to support use of the results to determine AYP.
Warlick, Quenemoen, Rigney 2004
Alternate Assessments should have…
Clearly defined structure Guidelines for which students may
participate Clearly defined scoring criteria and
procedures Report format that clearly
communicates student performance in terms of the academic achievement standards defined by the State
Warlick, Quenemoen, Rigney 2004
Alternate AssessmentsMust meet the same requirements for high technical quality that apply to regular assessments under NCLB:
Validity
Reliability
Accessibility
Objectivity
Consistent with nationally-recognized professional and technical standards.
Warlick, Quenemoen, Rigney 2004
States may use more than one alternate assessment
Alternate assessment scored against grade-level standards
Alternate assessment scored against alternate achievement standards
Warlick, Quenemoen, Rigney 2004
Out-of-Level Assessments
Could only be considered an alternate assessment based on alternate achievement standards if
Alternate achievement standards defined through a documented and validated standards-setting process
Proficient results included in the 1% cap
Warlick, Quenemoen, Rigney 2004
Researching Out-of-Level Testing
10 Recommendations: See Myth-Busters Handout
Warlick, Quenemoen, Rigney 2004
Universally Designed Assesmsents: A Quick Definition
Universally designed assessments are built from the beginning and continually refined to be accessible and valid for the greatest number of students!
Warlick, Quenemoen, Rigney 2004
But, what does that really mean?
• Do we want to change the standard of performance? NO
• Can we forget about accommodations if we do this? NO
• Is this all figured out – for now and forever? NO
• Is this something that will benefit only students with disabilities? NO
Warlick, Quenemoen, Rigney 2004
Universally designed assessments reflect good measurement qualities:
Minimize skills required beyond those being measured
Reflects the intended content standard (reviewers have information about what is being measured)
Measures What it Intends to Measure
Warlick, Quenemoen, Rigney 2004
An Example of Universal Design:
Mathematics Tests
The reading requirements of a math test often prevent students with marginal reading ability from demonstrating competency in math.
Warlick, Quenemoen, Rigney 2004
Ordering Pizza (Original Item)The cafeteria manager surveyed the students in a middle school to find out if they would buy Brand X pizza on Friday if the manager sold it. She made a circle graph to display the results of her survey.
NO
YES
Based on the results of the survey, answer the following questions:
1.1. What fraction of students would buy Brand X pizza on Friday?What fraction of students would buy Brand X pizza on Friday?
2.2. What percent of students would buy Brand X pizza on Friday?What percent of students would buy Brand X pizza on Friday?
3.3. There are 1200 students in this school. How many students will There are 1200 students in this school. How many students will buy Brand X pizza on Friday if the manager’s survey is accurate?buy Brand X pizza on Friday if the manager’s survey is accurate?
Warlick, Quenemoen, Rigney 2004
Ordering Pizza (Revised Item)Maria surveyed the students in her school to find out if they liked pizza on Friday. She made a circle graph to display the results of her survey.
NO
YES
1.1. What What fractionfraction of students said “yes”? of students said “yes”?
2.2. What What percentpercent of students said “yes”? of students said “yes”?
3.3. There are 1200 students in Maria’s school. There are 1200 students in Maria’s school. How manyHow many students said “yes”?students said “yes”?
Warlick, Quenemoen, Rigney 2004
Suppose a test item requires a student to read an Aesop’s fable
What could this item measure?– Decode text?– Comprehend extended passages?– Extent of vocabulary?– Understand the moral or point of the fable?– Discuss the common elements of any fable?– Compare and contrast fables with news reports?– Articulate the relationship between the fable and
the overall culture? – Anything else? (National Center on Accessing the Curriculum, 2003)
Warlick, Quenemoen, Rigney 2004
The “door” we need to go through: What is meant by the construct of ‘reading’ What about ‘literacy?’
The answer(s) to these questions will determine the universal design
or accommodations that can be used when certain content
standards are taught and assessed!
Warlick, Quenemoen, Rigney 2004
Modes of print interaction: with examples of accommodations
• Visual • Tactile (feeling print)• Auditory (listening to
printed messages)• Multi-modal (using
any combination of the above modalities)
• Printed text; ASL• Text in Braille and
Nemeth Codes• Listen to taped text• Computer-based
“assistive” reading/ viewing programs
Warlick, Quenemoen, Rigney 2004
Individualized accommodation decisions should be linked to the standard, the construct assessed, the nature of instruction or the assessment, and the student’s characteristics
Purposeful reading – reading to select
and apply relevant information for a given task
Does this allow different modes of print interaction? And, what are the implications of these different modes for accommodations?
Warlick, Quenemoen, Rigney 2004
Issues:• Universal design does not eliminate the
need for accommodations.
• Universal design requires the systematic application of good test development – more attention needs to be given at the same time that we need to develop more assessments.
• Little research exists on universally designed assessments.
Warlick, Quenemoen, Rigney 2004
Promising Practices:
• Item developers who know disability issues and their interaction with test formats and procedures.
• “Bias” or “sensitivity” review committee with at least one member who knows disability issues.
• Field testing that includes students with disabilities who use accommodations during the field test.
Warlick, Quenemoen, Rigney 2004
What is computer-based assessment
• Does NOT mean adaptive testing
• DOES mean making use of technology for accommodations
• DOES mean grappling with very, very important issues
Warlick, Quenemoen, Rigney 2004
Opportunities
• Efficient administration• Preferred by students• Improved writing performance• Built-in accommodations• Immediate results• Efficient item development• Increased authenticity
Warlick, Quenemoen, Rigney 2004
Challenges
• Use of technology cannot take the place of content mastery
• Issues of equity and skill in computer use
• Added challenges for some students
• Technological challenges
• Security of online data
• Lack of expertise in designing accessible Web pages
• Prohibitive development cost
Warlick, Quenemoen, Rigney 2004
Example: Kentucky Online Assessment
• Web-based, individualized assessment :– Students with IEP or 504 Plan that specifies need for
"reader" as an instructional and assessment accommodation;
– Students who require and routinely use text-reader or screen-reader technologies to access printed material in classroom instruction and assessment;
– Students who have accessed and used the CATS Online Practice Area.
• Based upon success of pilot studies, 16 districts, 31 Schools, & 204 students participated in “live” CATS Online in the spring of 2003