![Page 1: Exploring Alternate AYP Designs for Assessment and Accountability Systems 1 Dr. J.P. Beaudoin, CEO, Research in Action, Inc. Dr. Patricia Abeyta, Bureau](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649e6c5503460f94b6a754/html5/thumbnails/1.jpg)
Exploring Alternate AYP Designs for Assessment
and Accountability Systems
1
Dr. J.P. Beaudoin, CEO, Research in Action, Inc.Dr. Patricia Abeyta, Bureau of Indian Education
NIEA 2008 Annual ConferenceSeattle, Washington
![Page 2: Exploring Alternate AYP Designs for Assessment and Accountability Systems 1 Dr. J.P. Beaudoin, CEO, Research in Action, Inc. Dr. Patricia Abeyta, Bureau](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649e6c5503460f94b6a754/html5/thumbnails/2.jpg)
Session Goals
2
• Frame the technical and resource considerations for decision-makers
• Provide clarification on the
regulatory provisions associated with NCLB
• Share assessment and accountability design features
![Page 3: Exploring Alternate AYP Designs for Assessment and Accountability Systems 1 Dr. J.P. Beaudoin, CEO, Research in Action, Inc. Dr. Patricia Abeyta, Bureau](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649e6c5503460f94b6a754/html5/thumbnails/3.jpg)
Guiding Questions
3
urpose
How does the purpose for the change/action serve Native children?
irection What direction will the change/action
provide the educational community?
otivation To what degree will the change/action
motivate the Native peoples?
![Page 4: Exploring Alternate AYP Designs for Assessment and Accountability Systems 1 Dr. J.P. Beaudoin, CEO, Research in Action, Inc. Dr. Patricia Abeyta, Bureau](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649e6c5503460f94b6a754/html5/thumbnails/4.jpg)
30,000ft. SYSTEM VIEW
4
![Page 5: Exploring Alternate AYP Designs for Assessment and Accountability Systems 1 Dr. J.P. Beaudoin, CEO, Research in Action, Inc. Dr. Patricia Abeyta, Bureau](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649e6c5503460f94b6a754/html5/thumbnails/5.jpg)
School System Components
5
Academic Content Standards“What do students need to know and be able to demonstrate?”
• Instruction “How is content organized, provided, modeled, and expanded upon?”
• Assessment “To what degree is the content expressed accurately by the leaner?”
• Accountability “Which learning indicators are being reached, showing progress, or remaining unchanged?”
![Page 6: Exploring Alternate AYP Designs for Assessment and Accountability Systems 1 Dr. J.P. Beaudoin, CEO, Research in Action, Inc. Dr. Patricia Abeyta, Bureau](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649e6c5503460f94b6a754/html5/thumbnails/6.jpg)
Integration
6
Things to Consider…
• Changes to one component will have some influence on the remaining components
• Improvements in one area often result in benefits throughout the system
• Isolated changes create bulkization, inefficiencies, and internal stressors
![Page 7: Exploring Alternate AYP Designs for Assessment and Accountability Systems 1 Dr. J.P. Beaudoin, CEO, Research in Action, Inc. Dr. Patricia Abeyta, Bureau](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649e6c5503460f94b6a754/html5/thumbnails/7.jpg)
7
Leverage
![Page 8: Exploring Alternate AYP Designs for Assessment and Accountability Systems 1 Dr. J.P. Beaudoin, CEO, Research in Action, Inc. Dr. Patricia Abeyta, Bureau](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649e6c5503460f94b6a754/html5/thumbnails/8.jpg)
Compliance
If you change the academic content standards used for AYP…
If you change the assessments used for AYP…
If you change how AYP is defined…
8
Then you must provide evidence that addresses Section 1 of the USDE’s Peer Review Guidance
Then you must provide evidence that addresses Sections 2-7 of the USDE’s Peer Review Guidance
Then you must create an AYP Workbook that address 10 accountability principles.
![Page 9: Exploring Alternate AYP Designs for Assessment and Accountability Systems 1 Dr. J.P. Beaudoin, CEO, Research in Action, Inc. Dr. Patricia Abeyta, Bureau](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649e6c5503460f94b6a754/html5/thumbnails/9.jpg)
Overview: Standards and Assessments Peer Review
9
Conditioned on what is changed, evidence must be presented to the BIE and USDE Peer Reviewers…
Section 1-Content Standards
Section 2-Achievement Standards
Section 3-Coherent Assessment System
Section 4-Technical Quality
Section 5-Alignment
Section 6-Inclusion
Section 7-Reporting
![Page 10: Exploring Alternate AYP Designs for Assessment and Accountability Systems 1 Dr. J.P. Beaudoin, CEO, Research in Action, Inc. Dr. Patricia Abeyta, Bureau](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649e6c5503460f94b6a754/html5/thumbnails/10.jpg)
Overview: AYP Workbook
10
Principle 1-Includes all schools and districts
Principle 2-Includes all students
Principle 3-Includes an allowable method for AYP determinations
Principle 4-Includes annual accountability decisions
Principle 5-Includes subgroup accountability
Principle 6-Based on academic assessments
Principle 7-Has additional indicators
Principle 8-Separate decision for reading and mathematics
Principle 9-System validity and reliability
Principle 10-Includes participation rates
Conditioned on what is changed, policy and empirical evidence must be submitted for review by the BIE and USDE…
![Page 11: Exploring Alternate AYP Designs for Assessment and Accountability Systems 1 Dr. J.P. Beaudoin, CEO, Research in Action, Inc. Dr. Patricia Abeyta, Bureau](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649e6c5503460f94b6a754/html5/thumbnails/11.jpg)
11
Overview: Review Process
![Page 12: Exploring Alternate AYP Designs for Assessment and Accountability Systems 1 Dr. J.P. Beaudoin, CEO, Research in Action, Inc. Dr. Patricia Abeyta, Bureau](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649e6c5503460f94b6a754/html5/thumbnails/12.jpg)
ASSESSMENT
12
![Page 13: Exploring Alternate AYP Designs for Assessment and Accountability Systems 1 Dr. J.P. Beaudoin, CEO, Research in Action, Inc. Dr. Patricia Abeyta, Bureau](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649e6c5503460f94b6a754/html5/thumbnails/13.jpg)
Generic Assessment Life Cycle
13
![Page 14: Exploring Alternate AYP Designs for Assessment and Accountability Systems 1 Dr. J.P. Beaudoin, CEO, Research in Action, Inc. Dr. Patricia Abeyta, Bureau](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649e6c5503460f94b6a754/html5/thumbnails/14.jpg)
Foundational Components
Academic content standards Articulate what student are expected to
know and be able to do Grade level expectations in reading/ELA
and mathematics Grade span clusters in science Contain skill and process knowledge
14
![Page 15: Exploring Alternate AYP Designs for Assessment and Accountability Systems 1 Dr. J.P. Beaudoin, CEO, Research in Action, Inc. Dr. Patricia Abeyta, Bureau](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649e6c5503460f94b6a754/html5/thumbnails/15.jpg)
Foundational Components
Academic achievement standards Performance level descriptors No less than three achievement levels
(with applicable labels) “Cut-Score” established using a
technically valid standard-setting approach (e.g, Modified Bookmark, Body of Work, Teacher Judgment)
Established by subject matter experts (teachers) and other panelists with applicable skills
15
![Page 16: Exploring Alternate AYP Designs for Assessment and Accountability Systems 1 Dr. J.P. Beaudoin, CEO, Research in Action, Inc. Dr. Patricia Abeyta, Bureau](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649e6c5503460f94b6a754/html5/thumbnails/16.jpg)
Foundational Components
Assessments aligned to intended purposes
Assessments in grades 3-8 and high school: Reading/ELA Mathematics Science (3 grade clusters)
Standardized administration and scoring procedures
16
![Page 17: Exploring Alternate AYP Designs for Assessment and Accountability Systems 1 Dr. J.P. Beaudoin, CEO, Research in Action, Inc. Dr. Patricia Abeyta, Bureau](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649e6c5503460f94b6a754/html5/thumbnails/17.jpg)
Foundational Process
Typically 18-24 month development process
Documented evidence Reviewed by external (to USDE)
experts Evaluated evidence against Peer
Review Guidance Resubmissions typical
17
![Page 18: Exploring Alternate AYP Designs for Assessment and Accountability Systems 1 Dr. J.P. Beaudoin, CEO, Research in Action, Inc. Dr. Patricia Abeyta, Bureau](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649e6c5503460f94b6a754/html5/thumbnails/18.jpg)
USDE PEER REVIEW GUIDANCE
Standards and Assessments
18
![Page 19: Exploring Alternate AYP Designs for Assessment and Accountability Systems 1 Dr. J.P. Beaudoin, CEO, Research in Action, Inc. Dr. Patricia Abeyta, Bureau](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649e6c5503460f94b6a754/html5/thumbnails/19.jpg)
Section 1: Academic Content Standards
Students know and are able to…
Content standards are:
Adopted in mathematics; reading/ELA Adopted in science (grade spans) Coherent and rigorous Developed by stakeholders
19
![Page 20: Exploring Alternate AYP Designs for Assessment and Accountability Systems 1 Dr. J.P. Beaudoin, CEO, Research in Action, Inc. Dr. Patricia Abeyta, Bureau](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649e6c5503460f94b6a754/html5/thumbnails/20.jpg)
Section 2: Academic Achievement Standards
The level students have mastered the content is…
Achievement standards are: Adopted in mathematics, reading/ELA, and
science Assigned achievement levels and content-
based descriptors Established using an appropriate standard-
setting procedure Aligned to the content standards
20
![Page 21: Exploring Alternate AYP Designs for Assessment and Accountability Systems 1 Dr. J.P. Beaudoin, CEO, Research in Action, Inc. Dr. Patricia Abeyta, Bureau](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649e6c5503460f94b6a754/html5/thumbnails/21.jpg)
Section 3: Coherent Assessment System
The different assessment components within the larger assessment system contribute…
A coherent system: Is based on state and/or local assessments Provides integrated achievement data Aggregates only comparable measures Utilizes multiple measures focused on
higher order thinking skills Implements alternate assessments
21
![Page 22: Exploring Alternate AYP Designs for Assessment and Accountability Systems 1 Dr. J.P. Beaudoin, CEO, Research in Action, Inc. Dr. Patricia Abeyta, Bureau](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649e6c5503460f94b6a754/html5/thumbnails/22.jpg)
Section 4: Technical QualityThe technical quality of the assessments is…
High quality assessments are: Valid measures based on their purpose Constructed to measure the content standards Accurate and consistent Fair and accessible Consistent across different forms and events Standardized in their administration and
scoring
22
![Page 23: Exploring Alternate AYP Designs for Assessment and Accountability Systems 1 Dr. J.P. Beaudoin, CEO, Research in Action, Inc. Dr. Patricia Abeyta, Bureau](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649e6c5503460f94b6a754/html5/thumbnails/23.jpg)
Section 5: Alignment
The assessments reflect the content standards characteristics by…
Strong alignment is provided by using:• Using procedures to improve alignment
• Techniques to match the content standards and patterns of emphasis
• Capture the range and depth of knowledge in the standards
• Pattern of emphasis
• Approaches to measure both content and process knowledge
• Content-based performance descriptors
23
![Page 24: Exploring Alternate AYP Designs for Assessment and Accountability Systems 1 Dr. J.P. Beaudoin, CEO, Research in Action, Inc. Dr. Patricia Abeyta, Bureau](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649e6c5503460f94b6a754/html5/thumbnails/24.jpg)
Section 6: InclusionThe assessment system includes all students
by…
Students participate in the assessments as demonstrate by:
Impact data showing high rates of student participation
Affording students with disabilities participation under accommodated conditions
Accommodating students learning to speak English
Including migrant and other highly mobile students
24
![Page 25: Exploring Alternate AYP Designs for Assessment and Accountability Systems 1 Dr. J.P. Beaudoin, CEO, Research in Action, Inc. Dr. Patricia Abeyta, Bureau](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649e6c5503460f94b6a754/html5/thumbnails/25.jpg)
Section 7: ReportingThe assessment reports provide accurate and
timely information about student achievement by…
Student achievement data is: Reported using valid and credible procedures Based on all students from valid assessments Reported at the individual student, subgroup,
school, district, and state-levels Disseminated in a timely manner Easily understood by parents and the public
25
![Page 26: Exploring Alternate AYP Designs for Assessment and Accountability Systems 1 Dr. J.P. Beaudoin, CEO, Research in Action, Inc. Dr. Patricia Abeyta, Bureau](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649e6c5503460f94b6a754/html5/thumbnails/26.jpg)
USDE REVIEW GUIDANCEAYP Workbook
26
![Page 27: Exploring Alternate AYP Designs for Assessment and Accountability Systems 1 Dr. J.P. Beaudoin, CEO, Research in Action, Inc. Dr. Patricia Abeyta, Bureau](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649e6c5503460f94b6a754/html5/thumbnails/27.jpg)
Principal 1: Includes All Schools and Districts
Includes all schools and districts Holds all schools to the same
criteria Incorporates the academic
achievement standards Provides information in a timely
manner Includes report cards Includes rewards and sanctions
27
![Page 28: Exploring Alternate AYP Designs for Assessment and Accountability Systems 1 Dr. J.P. Beaudoin, CEO, Research in Action, Inc. Dr. Patricia Abeyta, Bureau](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649e6c5503460f94b6a754/html5/thumbnails/28.jpg)
Principal 2: Includes All Students
Includes all students Consistently defines full
academic year (FAY) Includes mobile students
28
![Page 29: Exploring Alternate AYP Designs for Assessment and Accountability Systems 1 Dr. J.P. Beaudoin, CEO, Research in Action, Inc. Dr. Patricia Abeyta, Bureau](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649e6c5503460f94b6a754/html5/thumbnails/29.jpg)
Principal 3: Method of AYP Determinations
Applies universal proficiency by 2013-2014 as the long-term goal
Uses a method to make subgroup, school, and district AYP determinations
Establishes a starting point with statewide annual measurable objectives (AMOs)
Establishes intermediate goals29
![Page 30: Exploring Alternate AYP Designs for Assessment and Accountability Systems 1 Dr. J.P. Beaudoin, CEO, Research in Action, Inc. Dr. Patricia Abeyta, Bureau](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649e6c5503460f94b6a754/html5/thumbnails/30.jpg)
Principal 4: Includes Annual Decisions
Determines annually the progress of schools and districts
30
![Page 31: Exploring Alternate AYP Designs for Assessment and Accountability Systems 1 Dr. J.P. Beaudoin, CEO, Research in Action, Inc. Dr. Patricia Abeyta, Bureau](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649e6c5503460f94b6a754/html5/thumbnails/31.jpg)
Principal 5: Includes Subgroup Accountability
Includes all the required student subgroups
Holds schools and LEAs accountable for the progress of student subgroups
Includes students with disabilities (SWD) and limited English proficient (LEP) students
Includes an established minimum n-count
Includes safeguards to protect student privacy
31
![Page 32: Exploring Alternate AYP Designs for Assessment and Accountability Systems 1 Dr. J.P. Beaudoin, CEO, Research in Action, Inc. Dr. Patricia Abeyta, Bureau](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649e6c5503460f94b6a754/html5/thumbnails/32.jpg)
Principal 6: Based on Academic Assessments
Based primarily on academic assessments
32
![Page 33: Exploring Alternate AYP Designs for Assessment and Accountability Systems 1 Dr. J.P. Beaudoin, CEO, Research in Action, Inc. Dr. Patricia Abeyta, Bureau](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649e6c5503460f94b6a754/html5/thumbnails/33.jpg)
Principal 7: Includes Additional Indicators
Includes graduate rate for high schools
Includes an additional academic indicators for elementary and middle schools
Establishes valid and reliable additional indicators
33
![Page 34: Exploring Alternate AYP Designs for Assessment and Accountability Systems 1 Dr. J.P. Beaudoin, CEO, Research in Action, Inc. Dr. Patricia Abeyta, Bureau](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649e6c5503460f94b6a754/html5/thumbnails/34.jpg)
Principal 8: Includes Separated Decisions for Reading/ELA and Math
Holds students, schools, and districts separately accountable for reading/ELA and mathematics
34
![Page 35: Exploring Alternate AYP Designs for Assessment and Accountability Systems 1 Dr. J.P. Beaudoin, CEO, Research in Action, Inc. Dr. Patricia Abeyta, Bureau](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649e6c5503460f94b6a754/html5/thumbnails/35.jpg)
Principal 9: System Validity and Reliability
Produces reliable decisions Produces valid decisions Addresses changes in
assessments and student populations
35
![Page 36: Exploring Alternate AYP Designs for Assessment and Accountability Systems 1 Dr. J.P. Beaudoin, CEO, Research in Action, Inc. Dr. Patricia Abeyta, Bureau](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649e6c5503460f94b6a754/html5/thumbnails/36.jpg)
Principal 10: Includes Participation Rates
Includes a way to calculate the rate of participation on the statewide assessments
Applies the 95% participation criteria to student subgroups and small schools
36
![Page 37: Exploring Alternate AYP Designs for Assessment and Accountability Systems 1 Dr. J.P. Beaudoin, CEO, Research in Action, Inc. Dr. Patricia Abeyta, Bureau](https://reader035.vdocuments.us/reader035/viewer/2022062409/56649e6c5503460f94b6a754/html5/thumbnails/37.jpg)
37
Research in Action, Inc.
World-Class Educational Quality™
Voice: 1.225.571.2408 E-Mail: [email protected]: 1.225.644.8472 Website: www.ria2001.org