the aa-mas
TRANSCRIPT
The AA-MAS: Lessons Learned by the
Pennsylvania GSEG
Amanda Kloo
University of Pittsburgh
PDE Bureau of Special Education PaTTAN PDE Bureau of Assessment and Accountability University of Pittsburgh CAST
The Target Population… • ASIST Guidelines…
Opportunity to learn grade level content Significantly Depressed Academic Achievement
and Progress
1. Ineligible for the PASA 2. Have a grade-level standards-aligned IEP
detailing needs for significant instructional accommodations and modifications
3. Have persistent academic difficulties despite receiving research-based interventions
4. Established patterns of significantly low achievement of multiple valid measures
NCSA 2011
2
The Target Population…
• PROJECT RAISSE… Longitudinal Analysis of SWD’s Achievement on
PSSA (n=16,894) • Students with IEPs who score in the below-basic
category are likely to remain at this performance level across time
• The greatest “movement” between performance categories is from basic to proficient and from proficient back to basic
• Economic disadvantage and minority status are associated with decreased probabilities of improved performance across time
• Children with mental retardation have the lowest likelihood of achieving proficiency; children with speech language impairment had the highest likelihood
NCSA 2011
3
The Target Population… • PROJECT RAISSE… Norm-referenced reading and math
achievement (n=141) • As children with IEPs move into the upper
grades, the achievement gap widens Teacher Interviews and Classroom
Observations • As grade level increases, differentiated
instruction decreases (as does “special education time”)
• Instruction aligned to grade-level standards was priority
NCSA 2011
4
2010 PSSA-M Math GRADES 4-11 • 1.7%-2.6% of the total tested population
• 9.6%-17.2% of the tested population with IEPs
• 40% Female; 60% Male
• 60% White; 40% Minority
• 3% ELL
• 46% Economically Disadvantaged
• Primary Disability- Analysis in process 71% SLD 10% OHI
NCSA 2011
5
Achievement History of 2010 PSSA-M Group
2009 PSSA Below Basic Basic Proficient Advanced
4 50% (915)
40% (737)
9% (174)
1% (20)
5 64% (1461)
24% (547)
10% (237)
2% (38)
6 64% (1579)
30% (752)
4% (109)
1% (26)
7 72% (1897)
24% (626)
4% (97)
<1% (20)
8 71% (1994)
24% (685)
4% (126)
1% (11)
NCSA 2011
6
AA-MAS Impact on Achievement
Not Proficient to
Proficient
Proficient to Not
Proficient
Stayed Not Proficient
Stayed Proficient
4 49% 1% 41% 1%
5 41% 2% 41% 2%
6 43% 2% 51% 2%
7 37% 1% 58% 1%
8 37% 1% 58% 1%
NCSA 2011
7
Themes • The AA-MAS did help some low-achieving
students earn proficiency on the state test
• However, it “helped” about as many low-achievers as it did not
• Some high-achieving students sacrificed proficiency on the AA-MAS
• Test performance of SWD is variable…
(Maybe too variable to demonstrate meaningful results/warrant a modified test?)
NCSA 2011
8
Big Picture • 41% of students formerly Not Proficient
on the state test scored Proficient based on AA-MAS achievement
• All told, 10% of the state’s IEP students assigned to the AA-MAS scored in the Below Basic Range
• 42% of their peers with IEPs not assigned to the AA-MAS scored in the Below Basic Range on the general assessment
NCSA 2011
9
Given What We Have Learned About the Target Population…
A Guide to UDL Test Item Modification for
Pennsylvania Application of Universal Design for Learning
Principles to AA-MAS: Recommendations from Pennsylvania’s GSEG
NCSA 2011
10
Purpose of “The Guide”…
• To affirm Pennsylvania’s current test development efforts for the PSSA-M
• To illustrate how the principles of UDL might inform future AA-MAS test development
• To extend the sound modification work already taking place in many states tackling the AA-MAS like Pennsylvania
• To highlight the positive implications of applying the principles of UDL to all assessment and instruction efforts in the state
NCSA 2011
11
Considerations…
• Paper-pencil assessment delivery • “Selected examples” for UDL-aligned item
modifications • Construct-irrelevant modifications • DOES NOT represent actual test items or
test item modifications on the operational PSSA-M Examples include only
• publicly released item samples • items that are representative of both skill
and content
NCSA 2011
12
Components of “The Guide”
1. Guiding Matrix for Applying Universal Design for Learning (UDL) Principles to AA-MAS
2. UDL modified exemplary passages 3. Item detail charts 4. Pennsylvania State Assessment Anchors 5. Glossary of terms
NCSA 2011
13
Structure of UDL Modifications
ID
LC
SR
NCSA 2011
14
LC
• LC-1 Syntax –
• LC-2 Simplifying vocabulary
• LC-3 Reduced sentence length
• LC-4 Language translation
• LC-5 Clarify anaphoric references
NCSA 2011
15
ID-1 Question placement
ID-2 Emphasize key information
ID-3 Chunking or combining tasks (questions / response options)
ID-4 Guide information processing
ID-5 Contextualizing skills –skill icons
ID-6 Hint
ID-7 Line numbering
ID-8 Passage primer
ID
NCSA 2011
16
SR-1 Progress map
SR-2 Self check
SR-3 Optional Workspace
SR-4 Skill Icon Preview
SR-5 Reduce Reflexive Eye Movement
SR
NCSA 2011
17
UDL Modification Detail Charts GUIDING MATRIX FOR READING
NCSA 2011
18
Item Details Chart
ID
NCSA 2011
19
LC-1 LC-2 LC-3
NCSA 2011
20
UDL Modification Detail Charts: Mathematics NCSA 2011
21
NCSA 2011
22
Next Steps for the AA-MAS… • “The UDL Guide” was presented to the BSE
and BAA as a research product from the GSEG Currently, test developers consider it to be
informative but adhere to vendor articulated guidelines for AA-MAS item modification/ enhancement
• Spring 2011: Pennsylvania continued administration of the AA-MAS in Math and operationalized the AA-MAS in Reading and Science
• The state continues its efforts to examine assessment trends and analyze achievement data to improve assessment and instruction opportunities for students with disabilities
NCSA 2011
23
What does “ALL” of this means for students with disabilities in PA?...
NCSA 2011
24
Thank you!
Amanda Kloo
PI- Pennsylvania GSEG Project The PSSA-Modified
Research Assistant Professor
University of Pittsburgh
Department of Instruction and Learning Special Education Program
NCSA 2011
25
For more information…
• Pennsylvania Department of Education Bureau of Special Education
• Assessment • http://www.portal.state.pa.us/portal/server.pt/com
munity/special_education/7465/assessment/607491
NCSA 2011
26