proof of concept study: grade 5 mathematics training tammy howard, phd director, accountability...
TRANSCRIPT
Proof of Concept Study:
Grade 5 Mathematics Training
Tammy Howard, PhDDirector, Accountability Services
NC Department of Public Instruction
August 4, 2015
PurposesValid and reliable measures of content
standards
Growth data for educator effectiveness reports
Assessments throughout the year that inform and guide instruction and that predict performance on future assessments
A stand-alone summative assessment at the end of the year that may be connected to the interim assessments
2
Research Questions
Determine the feasibility of implementing the concept across grades 3–8 data analysis to affirm reliability
input/feedback on the integration of the assessments in instruction
logistical/operational (delivery mode, scoring)
professional development requirements
possibility of different item types
3
Research Questions
Is it valid and reliable to combine results on the interim assessments for proficiency and growth reporting; thereby, eliminating an end-of-year summative assessment?
Do interim results provide teachers and students with useful information to inform and improve the delivery of instruction?
Will interim assessments predict performance on the end-of-year test?
4
Research Questions
Are the interim assessments required of all students or are some of the interim assessments optional?
Does the structure of the content standards require adjusting design of the through-grade model?
Is it feasible to incorporate constructed-response items or writing prompts on the interim assessments?
5
Research Questions
Is the performance affected by the delivery of through-grade assessment versus a traditional summative assessment model?
What information will be available for student-level and teacher-level reports and how is such information best delivered and presented?
Is the professional development adequate?
6
Research Questions
Is it feasible to deliver the assessments both online and paper/pencil?
Does the through-grade model provide parents with useful information and do they view the model as an effective way to assess students?
7
Concept Design―Grade 5
Students take 3 interim assessments during the school year The interim assessments are not included in
accountability calculations
Students take a shortened end-of-year test End-of-grade test without embedded field test
items
The modified end-of-year test is included in accountability calculations
8
9
Interim 1 Interim 2 Interim 3 End-of-Year
(Covers Interims 1– 4)
Design
Design could be altered based on outcome of study May have overlapping content May predict performance throughout the year
Information for Teachers
10
Student-level reports
Number of items correct by content standard
Number of overall items correct
Item reviews
TimelineYear Administration Grade Levels Purpose
2015–16 Proof of Concept(sample population)
Grade 5: MathGrade 6: Reading
Determine feasibility of concept
Summer 2016: Decision-point of how to proceed
2016–17 Field Test(sample population) Or
Release RFP for assessment
Grades 3–8: Math Grades 4–8: Reading
Test items for inclusion in operational test forms
2017–18 Pilot Operational Year(statewide)
Grades 3–8: Math Grades 4–8: Reading
Full Implementation
11
12
Sample―Grade 5Grade 5 Math — 45 schools, 4,021
students
Stratified random sample pulled
Generalizable to the entire state
Demographic variables: region, ethnicity, gender, economically disadvantaged
School-level achievement variable: mean scale score
13
Interim Test Specifications—Grade 5
Step 2 in the test development
Separate meetings held late June and early July
Invitees selected by NCDPI Curriculum and Instruction Mathematics and ELA
Active grade 5 Math and grade 6 ELA classroom teachers, instructional coaches, and LEA curriculum and instruction leaders
First half of meeting―Training by NCDPI Curriculum and Instruction
Second half of meeting—NCDPI Test Development collected feedback and recommendations
Test Development team discusses teacher recommendations with NCDPI Curriculum and Instruction to finalize test specifications
Feedback will be collected from sampled schools
Interim Design—Grade 5 Mathematics Grade 5
Calculator active and inactive sections
Multiple-choice and gridded-response items
ELA/Reading Grade 6
3 selections
Multiple-choice and constructed response (CR) items
– 1 CR item on interims two and three
Both
Paper/pencil
Up to 90 minutes max
20–25 items pulled from the EOG item pool (if available)
One form for each interim
14
Interim Assessment Policy—Grade 5 Testing Windows (3)
Interim 1 October 1–30, 2015
Interim 2 December 8, 2015 – January 22, 2016
Interim 3 March 3–31, 2016
Districts/charters can determine the testing days within the designated windows
Interims are not required to be administered to all students on same day, but should be administered within the same week
Make-up administrations are optional
Should be administered by the classroom teacher
Proctors are not required
Administration does not require the removal of displays
15
16
Interim Assessment Policy―Grade 5
Eligibility
Mathematics Grade 5
All students enrolled in grade 5 at sampled schools who participate in the EOG mathematics assessment
English Language Arts/Reading Grade 6
All students enrolled in grade 6 at sampled schools who participate in the EOG English Language Arts/ reading assessment
Both
Transfer students−−Take the interim(s) and the modified end-of-year (EOY) test
No opt out
17
Interim Assessment Policy—Grade 5
Students Not Eligible to Participate
Students with disabilities whose IEP documents participation in the NCEXTEND1 alternate assessment
LEP students who scored below Level 4.0 Expanding on the W-APT and are in their first year in U.S. schools are not eligible to participate in the grade 6 ELA study, but they are eligible to participate in the grade 5 math study
Students who are granted a medical exception from the Division of Accountability Services for the EOG assessments
18
Interim Assessment Policy—Grade 5
Accommodations
IEP, 504, and/or LEP teams do not have to reconvene and document the accommodations for the Proof of Concept Studies.
Students may use the accommodations that are specified on their current IEPs, Section 504 Plans, or LEP documentation for the EOG for the modified end-of-year test.
Instructional accommodations may be used for the interims except for the Read Aloud and Signing/Cueing accommodations for grade 6 ELA/Reading. Reading aloud or signing/cueing the selections, questions, or answer choices invalidates results because the interims measure reading skills.
19
Interim Assessment Policy―Grade 5
Special Print Versions
Braille, Large Print (LP), One Item Per Page (OIPP), and Large Print One Item Per Page Editions (LP/OIPP) can be ordered from TNN
Accommodation Notification Request Forms for special print versions do not need to be sent in for interim assessments
Submit orders at least 30 working days before the actual administration date
20
Interim Assessment Policy—Grade 5
Materials Ordering
Materials will be ordered through TNN (after initial order, materials will be auto shipped for the remainder of the study)
Assessment Guide
– guide for interims (separate guides: 1 for ELA/Reading; 1 for Math)
– guide for end-of year (separate guides: 1 for ELA/Reading; 1 for Math)
Answer sheets
Test books (separate test books for the subjects)
21
Interim Assessment Policy― Grade 5
Required Supplemental Materials
English Language Arts/Reading
Blank paper
Mathematics
Blank paper
Graph paper (auto shipped for interims and end-of-year)
Calculators
– Any four-function calculator with memory key
22
Interim Assessment Policy—Grade 5
Security
Assessment guides are not secure test materials
Store at the school until all interims have been administered, then securely destroy
Following the administration, interim assessment booklets are to be kept at the schools for 4 weeks, then securely destroyed
Booklets must remain in the school
Booklets should be accounted for at all times
Local decision where booklets are stored at the school (secure facility not accessible to students)
Teachers may use the booklets with students in reviews
Parents may view the booklets only within the school setting. The teacher may share with parents the student’s scores on the items through customary communication (i.e., individual parent/teacher conferences at the school)
Interim assessments (booklets, items, content) cannot be shared with other schools
23
NC Education Page
• Use the following link to the course
https://center.ncsu.edu/ncaccount