developing monitoring and pre-scoring plans for alternate/alternative assessments virginia...
TRANSCRIPT
Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative
Assessments
Virginia Department of Education
Division of Student Assessment
and School Improvement
August 2011
Purpose of this Presentation To assist school divisions in developing
plans to monitor and pre-score Collections of Evidence (COE) for the Virginia Alternate Assessment Program (VAAP) and the Virginia Grade Level Alternative (VGLA) and Course Work Compilations (CWC) for the Virginia Substitute Evaluation Program (VSEP).
Developing Monitoring and Pre-Scoring Plans for Alternate/Alternative
Assessments What are monitoring and pre-scoring? Why should divisions develop monitoring and pre-
scoring plans? Who should be involved in developing the
monitoring and pre-scoring plans? What components should the monitoring plan
include? What components should the pre-scoring plan
include?
What are monitoring
and pre-scoring?
Monitoring
A periodic and systematic review of COEs and CWCs while they are in the development process with opportunities for feedback and intervention.
What Monitoring is Not
Checking in with the teacher
Flipping through COEs or CWCs
Reviewing COEs and CWCs without providing feedback
Pre-Scoring
A detailed review of COEs and CWCs to address technical errors (i.e., grading, SEI tags) prior to scoring with opportunities for feedback and corrections.
What Pre-Scoring is Not
Assigning a rubric score to the evidence
Altering student evidence or student responses
Judging the instructional quality of the evidence
Why should divisions
develop monitoring and pre-scoring
plans?
Monitoring Keeps division staff abreast of the status
of each COE and/or CWC
Ensures that teachers are providing instruction based on the Standards of Learning (SOL)
Ensures that instruction is on target with the division’s pacing guide
Monitoring Ensures that technical issues (i.e.,
grading, SEI tags) are appropriately addressed
Provides the opportunity to address issues with ample time for correction and/or intervention
Pre-Scoring Provides a final opportunity to correct
technical errors and omissions that may result in the loss of credit to students prior scoring
Results in the creation of better COEs and CWCs
Who should be involved
in developing the monitoring and pre-scoring plans?
Who should be involved in developing the plans?
Depends upon:
Number of COEs and CWCs
Available central office resources
Available building level resources
Who should be involved in developing the plans?Central Office Staff
Division Director of Testing (DDOT) or designee
Special Education Director or designee
Director of Instruction or designee
Title III Coordinator or designee
Building Level Staff
Principal or designee
School Test Coordinator
Special Education Chairperson
What components should the
monitoring plan include?
Critical Planning InformationDevelop a Master Participation List for each
alternate/alternative assessment program that includes... Name of student Grade level of student School Content area being assessed by
alternate/alternative assessment Teacher(s) responsible for instruction in the
content area assessed
Monitoring Plan Decisions Who will monitor?
How will monitors be trained?
When will monitoring occur?
Who will receive feedback?
How will feedback be given?
What options are available if corrections are needed?
Remember!
Your monitors will provide training and/or technical guidance to teachers submitting COEs and are not allowed to score these COEs.
Who will monitor?Staff with content knowledge and alternate/alternative
assessment knowledge Hired consultants Special education or testing specialists Instructional specialists LEP specialists School test coordinators Title I mathematics and reading specialists Experienced teachers Building administrators
How will monitors be trained? Provide access to implementation
manuals and other resources (i.e. testing memos, VGLA worksheets, SEI tags, Curriculum Frameworks)
Conduct local training for monitors to include review and discussion of scoring rules, SOL or ASOL information, issues generated from the previous year’s scoring, etc.
When will monitoring occur? Monthly
End of each grading period
At the same time as benchmark testing
Every other month
Other
Who will receive feedback? Teachers
Principals
Alternate/Alternative Assessment Leaders
Central Office Staff
Trainers/Technical Assistance Providers
How will feedback be given? Post It notes on COEs and CWCs
Locally created feedback forms given to teachers, principals, and/or others
Meetings with teachers, principals, and/or others about monitoring results
Other
What options are available if corrections
are needed? Individualized consultation and support
Peer or small group training sessions
Division-level professional development
Access to state and local SOL and/or ASOL assessment resources
Other
A Sample Monitoring Plan
for 500 COEs School-based Review Teams will be created in
each school.
DDOT and Special Education Director will train School-based Review Teams using implementation manuals, VDOE power points, division pacing charts, and other resources.
Teachers submit COEs or CWCs to the School-based Review Team at the end of each nine-weeks.
A Sample Monitoring Plan
School-based Review Teams will report their finding on each COE or CWC using a locally created form. A copy of the form will be distributed to the teachers, principals and central office staff.
Central office will dispatch instructional specialist and assessment specialist to schools in need of additional support and training.
A Sample Monitoring Plan
1Select And TrainSchool-based Review Teams
3Instructional and assessment specialists
dispatched to schools based on 9 Week Reports
2School-based Review Teams check COEs
every 9 weeks according to Pacing Chart and
report to principals andcentral office staff
A Sample School-Based Review Team
9-Week Monitoring Form – Grade 4 Reading
Student 100 - 90 % of SOL Evidenced
89 - 70 % of SOL Evidenced
69- 50 % of SOL Evidenced
Less than 50% of SOL Evidenced
Ted Smith X
Kay Blue X
Joe Davis X
Ann Jones X
A Sample Teacher Review Form
VGLA 9 Week Review SheetSchool-Based Teams
Teacher: ____________________
Student: ___________________________ Grade Level:______
Content Area:_________________________________________
REVIEW for: ____ 1st Nine Weeks ____ 3rd Nine Weeks
____ 2nd Nine Weeks ____ 4th Nine Weeks
Reviewed By:____________________________ Date:________
Collection Status: Address the following questions: (1) Is there evidence for all of the standards for the nine weeks according to the Pacing Chart? (2) Does the work submitted align with the standards satisfactorily? (2) Does the evidence demonstrate student mastery? (3) Has the student work been graded accurately? (4) Other?
Recommendation(s): Be as specific as possible
Follow-up review needed for implementation of recommendations:
___Yes ____No
Follow –up Review Date: ________________
Pre-Scoring Decisions Who will pre-score?
How will pre-scorers be trained?
When will pre-scoring occur in relation to submission date?
Who will receive feedback?
How will feedback be given?
What options are available if corrections are needed?
Persons selected for pre-scoring VAAP COEs can not serve as scorers.
Pre-scorers should be knowledgeable about all scoring rules and the SOL and/or ASOL content.
Pre-scorers should be detail oriented.
Who will pre-score?
Remember!
Your pre-scorers will provide training and/or technical guidance to teachers submitting VAAP COEs and are not allowed to score these COEs.
Training should include a list of specific issues to look for in each COE or CWC
grading
correctly completed SEI tags
detailed anecdotal records (VAAP only)
captioned photographs (VAAP only)
How will pre-scorers be trained?
Other helpful questions to ask in
Pre-scoring Is the evidence organized according to the scoring worksheet?
Are all required state and local forms included and completed?
Select a time to pre-score:
that will give ample time for COEs or CWCs to be complete or near completion.
that will give opportunity to return COEs or CWCs to teachers for corrections and returned by local due date.
When will pre-scoring occur in relation to
local due date?
Teacher(s) submitting COE or CWC.
Building level principals
Others
Who will receive feedback?
Post-it Notes on COEs and CWCs
Local created forms given to teachers, principals, and/or others
Meetings with teachers, principals, and/or others
Other
How will feedback be given?
Return COEs to teachers for corrections.
Assign staff to assist teacher in corrections such as grading evidence, organizing evidence, completing SEI tags.
Options may be limited or non-existent due to time constraints
Other
What options are available if corrections
are needed?
Sample Pre-scoring Plan Pre-scoring will be conducted by teachers,
building level staff and central office staff in three phases Teacher Level Building Level Central Office Level
Scoring rules will be reviewed with all persons involved in pre-scoring. Checklists will be provided.
Sample Pre-Scoring Plan
PHASE 1 – Teacher Review
Conducted April 12-16
Submitting Teacher reviews COEs or CWCs using Teacher Checklists in Implementation Manuals
PHASE 2 – School-based
Review
Conducted April 19 – 26
COEs or CWCs using notes and local forms
Building Administrator reviews COEs or CWCs using Administrator Checklist in Implementation Manuals
Sample Pre-Scoring PlanPhase 3 – Central Office Pre-scoring Team Review
Conducted April 27 – May 7
Central Office team reviews COEs and CWC against scoring rules. Findings noted on Post-It notes placed on evidence.
Central Office teams return COEs or CWCs to schools to correct errors and omissions.
COEs or CWCs are corrected by teachers. Teachers sign affidavits
COEs or CWCs submitted to principals or designee. Principal or designee signs affidavits.
COEs or CWC submitted for scoring.
Monitoring & Pre-Scoring Results
Reduces or Eliminates COEs and CWCs with:
Missing Evidence or SEI tags
Unacceptable Evidence (textbooks, homework)
Inaccurate or Ungraded Evidence
Monitoring & Pre-Scoring Results
Eliminates Surprises:
COEs or CWCs not done for students educated outside of the division
Incomplete COEs or CWCs in the division
COEs or CWCs not driven by IEP, 504 or LEP Plans
Questions
Virginia Department of EducationOffice of Test Administration, Scoring, and
Reporting
804 225-2102
Office of Special Education Instructional Services
804-225-2932