policy definitions, achievement level descriptors, science achievement standards (cut scores) for...
TRANSCRIPT
Policy Definitions, Achievement Level Descriptors, Science Achievement Standards (Cut Scores)
for General Education and for Alternate Assessment
1
Overview Kathleen Vanderwall, OAIS
External Evaluation Mary Seburn, Educational Policy Improvement Center
Implementing 2009 Content Standards for Instructional Improvement
Monument School District, Rachel Aazzerah North Clackamas School District, Meagan Sternberg
2
Policy Definitions are general statements that describe rigor across grade levels and content areas and provide an overarching definition for each achievement level. They are used for all subjects.
Achievement Level Descriptors (ALDs) explain a student’s knowledge and skills at a specific Achievement Level by grade group and subject and are a key element in the establishment of cut scores.
ODE focused the Achievement Standards Verification panelists’ attention by using outside references with data on national and international benchmarks.
4
Specify the knowledge, skills and abilities a student needs to know in order to be classified as:
• Does Not Yet Meet• Nearly Meets• Meets• Exceeds
5
Scores on Oregon’s scale. Describe what students should be able to do in each achievement level.
Participants actively discuss their expectations of the “target student” in each achievement level using a booklet of 70 Oregon test items ordered from easiest to most difficult.
6
May-September 2010 Moving Science Forward Professional Development
2009, 2010 October Statewide In-Service- new standards and assessments presentations September 2009-present Science Monthly Updates,
AA Updates, Superintendents Pipeline February 2011 Draft 2009 Test Specifications and
Blueprint Documents posted to web January-May 2011 Announcements regarding
reporting, assessments, scoring guides and Science Standard Verification
August-September 2011 Survey October 2011 – May 2012 Communication Strategy
7
General Policy Definitions (adopted March 2010)
Science Policy Definitions(provide consistency across grades-approved by Science Panel January 2011)
Science Achievement Level Descriptors for grade 5, 8, HS
(adopted May 2011)
Final Achievement Level Descriptors and Achievement Standards linked to outside references will be available to inform partners.
8
Dr. Mary Seburn Director Research, Design and Analytics DivisionEducational Policy Improvement Center
9
• Modified Bookmarking method • Comparative data from NAEP, PISA, OUS, other states• Nearly Meets, Meets, and Exceeds • Grades 5, 8, and high school Science• 24 stakeholders • Three days
• Recommended increased achievement standards for all grade levels• Except for “Nearly Meets”; current standards confirmed
10
12
The Educational Policy Improvement Center (EPIC) •Documented evidence of technical adequacy
• Procedural validity, as observed• Standards for Educational and Psychological Testing• NCLB Peer Review• Best practice
•Materials review•Observations•Interviews •Evaluations
Twenty-one evaluation elements total, all met:
13
Panelist Selection Panelist and Facilitator
Training Methodology Process Documentation
“The training materials were helpful.”
“I am confident I understand my role in the standards verification process.”
“The training clearly identified the goals for the standards verification procedure.”
14
“I understood how to place my bookmarks.”
“Overall, I am satisfied with my group's final bookmarks.”
“I am confident that the bookmark procedure used produced valid Achievement Standards.”
“The bookmark procedure was well described.”
15
16
• “Steps in the process were very good, group and individual work time allowed for flexibility and allowed us to come to consensus.”
• “I would love to do this again. I can’t think of anything they could do better. Very successful.”
• “I really believe in the process, it is difficult, but it’s why democracy works.”
Rachel Aazzerah7-12 Science/Math TeacherProfessional Learning Community Leader/Data Driven Decision Making Coordinator
Monument School District
17
Interaction and Change Life Science (6.2L.2)
Explain how individual organisms and populations in an ecosystem interact and how changes in populations are related to organisms.
18
Interaction and Change Earth and Space Science (7.2E.3)
Evaluate natural processes and human activities that affect global environmental change and suggest and evaluate possible solutions to problems.
19
Structure and Function Life Science (H.1L.3)
Explain and apply laws of heredity and their relationship to the structure and function of DNA.
20
North Clackamas School District is in the middle of reviewing and realigning science courses with a focus on making sure students are exposed to Engineering Design content, practices and work samples at each grade level.
This is challenging in some science courses. However, we feel it is critical work with the
new focus on Engineering and Technology at the national level.
22
Happy Valley and Rock Creek Middle Schools have an elective called “Engineer It!”.
At HVMS, almost all students take this class.
23
Raising the Science Achievement Standards increases the rigor and importance of teaching and learning science in Oregon schools.
24
State Board adoption of recommended achievement standards and achievement level descriptors
Peer Review Submission- December 2011
25
Dianna Carrizales-Engelmann
Brad Lenhardt
Office of Student Learning and Partnerships
Daniel Farley
University of Oregon
27
Population District Uses Diploma Options
Post-Secondary Implications
Students with the most significant cognitive disabilities
AYP reporting (up to 1%) and district achievement reporting
•Extended Diploma
•Modified Diploma
Supported living facilities to post-secondary education and vocational careers
29
General Assessment Alternate Assessment
Population All students Students with IEPs
Participation ‘10
~296,090 ~5,776
Decision Default decision IEP team decision
Development Based on grade level content
Based on grade level content reduced in depth, breadth, and complexity
Administration
Primarily Online Paper and pencil, individually administered
Assessment areas
Reading, Writing*, Mathematics, Science
Reading, Writing*, Mathematics, Science
Stakes Used to demonstrate Essential skills towards a regular diploma
Can be used to demonstrate Essential Skills in Reading toward the Modified Diploma or can lead to the Extended Diploma 30
General Assessment
Alternate Assessment
Participation (2010)
123,036 1736
Achievement Standards
RIT scores Scaled Scores
ALDs Developed by staff approved by expert panelists
Developed by staff approved by expert panelists
Standard Setting process
Verification of staff- recommended cutscores
Panel-recommended cutscores based on knowledge of population and assessment
31
University of Oregon Behavioral Research and Teaching Dr. Gerald Tindal
Bookmarking process 13 educators representing 9 counties SPED and content expertise 3 rounds Review of impact data Recommendations: Both increases and decreases
related to refinements to the new content standards
34
Achievement Level
CurrentScience Cut Scores
ProposedScience Cut Scores
Change to Cut Score(+/- Resulting from
Recommended Minus Current Cut Score)
Grade 5 Grade 8 HS Grade 5 Grade 8 HS Grade 5 Grade 8 HSNearly Meets 100 89 91 100 95 98 0 +6 +7
Meets 108 96 107 107 101 103 -1 +5 -4Exceed
s 116 114 114 114 113 109 -2 -1 -5
35
Cut Scores Impact Data
GradeNearly Meets Meets Exceeds
Does Not Yet Meet
Nearly Meets Meets Exceeds
Meets /Exceeds 2011
Meets /Exceeds 2010
5 100 107 114 28% 11% 37% 23% 60% 58.10%8 95 101 113 20% 12% 53% 15% 68% 80.20%
HS 98 103 109 36% 21% 31% 12% 43% 33.60%
36
Does Not Yet Meet Nearly Meets Meets Exceeds
Science Policy Definitions(Apply to all grade groups)
Performance indicates that the student has a limited understanding of specific academic concepts linked to grade-level content that has been reduced in depth, breadth, and complexity.Demonstration of Science Knowledge and Skill in familiar situations with significant limitations.
Performance indicates an inconsistent understanding of specific academic concepts linked to grade-level content that has been reduced in depth, breadth and complexity.Demonstration of Science Knowledge and Skill in familiar situations with some success.
Performance indicates an understanding of specific academic concepts linked to grade-level content that has been reduced in depth, breadth, and complexity.Demonstration of Science Knowledge and Skill in familiar situations with success.
Performance indicates a consistent understanding of the academic concepts linked to grade-level content that has been reduced in depth, breadth, and complexity.Demonstration of Science Knowledge and Skill in familiar situations with consistent success and in unfamiliar situations with some success.
37
Rigor of process Appropriateness of panelists Quality of personnel/oversight Thoroughness of training Validity of assessment Panelist Confidence Assessment data
38
Does Not yet Meet: Students do not demonstrate mastery of grade-level knowledge and skills required for proficiency.
Nearly Meets: Students demonstrate partial mastery of grade-level knowledge and skills required for proficiency.
Meets: Students demonstrate mastery of grade-level knowledge and skills required for proficiency.
Exceeds: Students demonstrate mastery of grade-level knowledge and skills exceeding the requirement for proficiency.
40
Does Not Yet Meet: Students demonstrate limited mastery of science, scientific inquiry and engineering design knowledge and skills, through the application of basic concepts with occasional success.
Nearly Meets: Students demonstrate partial mastery of science, scientific inquiry and engineering design knowledge and skills, through the application of basic concepts with regular success.
Meets: Students demonstrate mastery of science, scientific inquiry and engineering design knowledge and skills, through the interpretation and application of grade-level concepts with consistent success.
Exceeds: Students demonstrate mastery of science, scientific inquiry and engineering design knowledge and skills, through the interpretation and application of advanced concepts with consistent success.
41