ell-language-based accommodations for content area assessments the university of central florida...
TRANSCRIPT
ELL-Language-based Accommodations for Content Area Assessments
The University of Central Florida
Cocoa Campus
Jamal Abedi
University of California, Davis
July 7, 2011
1
ELL Language-Based Accommodations
English dictionary
English glossary
Bilingual dictionary/glossary
Customized Dictionary
Native language testing
Read-aloud test items or directions
Linguistically modified test
Computer testing with pop-up glossaries
2
English Dictionary
Providing an English Dictionary is another commonly used accommodation for ELL students (Abedi, Courtney, & Leon, 2003; Abedi, Lord, Boscardin & Miyoshi, 2000).
The use of a dictionary and extra time affect the performance of all students (Abedi, Lord, Hofstetter & Baker, 2000; Hafner, 2001; Maihoff, 2002; Thurlow, 2001; Thurlow & Liu, 2001).
By gaining access to definition of content-related terms, recipients of a dictionary may be advantaged over those who did not have access to the dictionaries. This may compromise the validity of assessment (Abedi, Courtney, Mirocha, Leon & Goldberg, 2005).
The dictionary as a form of accommodation suffers from yet another major limitation, the feasibility issue (Abedi, Courtney, Mirocha, Leon & Goldberg, 2001).
Consequently, the results of accommodated and non-accommodated assessment may not be aggregated.
3
English Glossary
English glossary with extra time raised performance of both ELL and non-ELL students (Abedi, Hofstetter & Lord, 2000).
ELL students’ performance increased by 13% when they were tested under the glossary with extra time accommodation.
While this looks promising, it does not present the entire picture.
Non-ELL students also benefited from this accommodation, with an increase of 16% (Abedi, Hofstetter & Lord1998, 2000).
Thus, the results of the accommodated outcome cannot be aggregated with the non-accommodated outcome.
4
Customized Dictionary
Customized dictionary was introduced as a more valid alternative to English/bilingual dictionaries (Abedi, Courtney, Mirocha, Leon & Goldberg, 2001).
It is a cut-and-paste of the actual dictionaries.
In only includes terms that are: (1) in the test and (2) non-content related.
Results of studies suggest that it is highly effective and valid accommodation for ELL students.
5
Linguistically Modified Test
There are, however, some accommodations that help ELL students with their English language needs without compromising the validity of assessment.
Studies suggested that the linguistically modified version of the tests items is an effective and valid accommodation for ELL students (Abedi, Hofstetter, Lord & Baker, 2000; Maihoff, 2002; Rivera & Stansfield, 2001).
This accommodation also helped students with learning disabilities.
Thus, an accommodation may have the potential to be effective and help provide valid assessment outcomes for ELL students.
6
Computer Testing
Research findings suggest computer testing as an effective and valid accommodation for ELL students (Abedi, Courtney, Mirocha, Leon, & Goldberg, 2001).
ELL students show higher levels of motivation on the assessments administered by computer.
Different types of accommodations that have been shown to be useful for ELL students may be incorporated into the computer testing system (Abedi, et al, 2011).
7
Native Language Testing Translating tests into students’ native language is an
accommodation used by many states across the country (Abedi, Lord, Hofstetter & Baker, 2000; Rivera, Stansfield, Scialdone & Sharkey, 2000).
Issues concerning translation and content coverage across the forms must be seriously considered.
Students’ background variables, particularly their level of proficiency in L1 and L2, must be studied before considering this accommodation.
Using native language assessment may not produce desirable results if the language of instruction and assessment are not aligned (Abedi, Hofstetter & Lord, 2004).
8
Examining Complex Linguistic Features in Content-Based Test Items
Unnecessary complex linguistic features slow down the reader, make misinterpretation more likely, and add to the reader’s cognitive load; thus interfering with concurrent tasks. These features include:
Concrete vs. abstract or impersonal presentationsItem lengthUnfamiliar VocabularyNominal heavinessRelative clauseConditional clausePassive voice Long noun phrasesSubordinate clauses
9
Impact of language factors on content assessments for ELLs
Continuum of Linguistic Complexity and Item Characteristics
1 Items with no linguistic complexity:
Familiar or frequently used words; word length generally shorter
Short sentences and limited prepositional phrases
Concrete item(s) and a narrative structure
No complex conditional or adverbial clauses
No passive voice or abstract or impersonal presentations
10
Impact of language factors on content assessments for ELLs
Continuum of Linguistic Complexity and Item Characteristics
2 Items with a minimal level of linguistic complexity:
Familiar or frequently used words; short word length
Moderate sentence length with a few prepositional phrases
Concrete item(s)
No subordinate, conditional, or adverbial clauses
No passive voice or abstract or impersonal presentations
11
Impact of language factors on content assessments for ELLs
Continuum of Linguistic Complexity and Item Characteristics
3 Items with a moderate level of linguistic complexity:
Unfamiliar or seldom used words
Long sentence (s)
Abstract concept (s)
Complex sentence/conditional tense/adverbial clause(s)
A few passive voice or abstract or impersonal presentations
12
Impact of language factors on content assessments for ELLs
Continuum of Linguistic Complexity and Item Characteristics
4 Items with a high level of linguistic complexity:
Relatively unfamiliar or seldom used words
Long or complex sentence(s)
Abstract concept(s)
Difficult subordinate, conditional, or adverbial clause(s)
Passive voice/ abstract or impersonal presentations
13
Impact of language factors on content assessments for ELLs
Continuum of Linguistic Complexity and Item Characteristics
5 Items with a maximum level of linguistic complexity:
Highly unfamiliar or seldom used words
Very Long or complex sentence(s)
Abstract concept(s)
Very difficult subordinate, conditional, or adverbial clause(s)
Many passive voice/ abstract or impersonal presentations
14
Sample Original and Revised Item
• Below is an example of a test items that deemed to be Below is an example of a test items that deemed to be linguistically complex and a linguistically modified version of linguistically complex and a linguistically modified version of the items. the items.
• Original: Original: If Y represents the number of newspapers that Lee If Y represents the number of newspapers that Lee delivers each day, which of the following represents the total delivers each day, which of the following represents the total number of newspapers that Lee delivers in 5 days?number of newspapers that Lee delivers in 5 days? – A) 5 + Y A) 5 + Y – B) 5 x Y B) 5 x Y – C) Y + 5 C) Y + 5 – D) (Y + Y) x 5D) (Y + Y) x 5
• Modified: Modified: Lee delivers Y newspapers each day. How many Lee delivers Y newspapers each day. How many newspapers does he deliver in 5 days?newspapers does he deliver in 5 days?
(Adopted from Abedi, Lord & Plummer, 1997, p. 21)(Adopted from Abedi, Lord & Plummer, 1997, p. 21) 15
Linguistic Modifications made on the item
• Conditional clause changed to separate sentence
• Two relative clauses removed and recast
• Long nominals shortened
• Question phrase changed from “which of the following represents“ to “how many”
• Item length changed from 26 to 13 words
• Average sentence length changed from 26 to 6.5 words
• Number of clauses changed from 4 to 2
• Average number of clauses per sentence changed from 4 to 1
16
Conclusions and Recommendation
Assessments and instructions for ELLs :Assessments and instructions for ELLs :
Must be based on a sound psychometric principlesMust be based on a sound psychometric principlesMust be controlled for all sources of nuisance or Must be controlled for all sources of nuisance or confounding variablesconfounding variablesMust be free of unnecessary linguistic complexitiesMust be free of unnecessary linguistic complexitiesMust include sufficient number of ELLs and SWDs in its Must include sufficient number of ELLs and SWDs in its development process (field testing, standard setting, development process (field testing, standard setting, etc.)etc.)Must be free of biases, such as cultural biasesMust be free of biases, such as cultural biasesMust be sensitive to students’ linguistics and cultural Must be sensitive to students’ linguistics and cultural needsneeds
17
Conclusions and Recommendations
Accommodations:
Must be relevant in addressing assessment issues for ELL students
Must be effective in making assessments more accessible to ELL students
Should not alter the construct being measured
Must provide results that can be aggregated with the assessment outcomes under standard conditions
Must be feasible in national and state assessments
18
Conclusions and Recommendations
Examples of research-supported accommodations:
Providing a customized dictionary is a viable alternative to providing traditional dictionaries.
The linguistic modification of test items that reduce unnecessary linguistic burdens on students is among the accommodations that help ELL students without affecting the validity of assessments.
Computer testing with added extra time and glossary was shown to be a very effective, yet valid accommodation (Abedi, Courtney, Leon and Goldberg, 2003)
19