crct- m pre-administration workshop february 2013 webinar etiquette please use the audio setup...
TRANSCRIPT
CRCT-M Pre-Administration
WorkshopFebruary 2013
Webinar Etiquette•Please use the Audio Setup Wizard in the Tools Menu to configure and test your audio settings before the presentation begins.•To eliminate interference from background noise in your area, please leave the Talk Button on mute.•Due to the number of participants, we request that questions be submitted via Chat.
Agenda
• CRCT-M Overview• Pre-Administration• Testing• Post Test Activities• Reporting• Contact Information• Questions
2
Key Points• Test is only available for students receiving special education services
who meet specific CRCT-M eligibility guidelines and whose IEPs reflect this assessment.
• Districts were to order CRCT-M test materials during December –January via TestTime.
• Remember an eligible student may take the CRCT-M in the subject(s) determined through the eligibility process, but will also participate in the CRCT in science and social studies and for those subjects (Reading, ELA, and Math) for which they were not eligible for CRCT-M.
3
Key Points
4
• Students who achieved Basic Proficiency for 2 consecutive years in the same content area are ineligible for the CRCT-M in that content area for 2013.
NEW
Key Points• Test materials differ from CRCT
– Vendor• CRCT-M : Georgia Center for Assessment (GCA) in Athens,
GA• CRCT: CTB/McGraw Hill in Indianapolis, Indiana
– Test booklets• Different test booklet for each subject
– Answer document• Single answer document
– Test Examiner’s Manual• Similar to CRCT, but there are differences in text and script
– System/School Coordinator Manuals• Arranged like CRCT, but has specifics about GCA material
distribution
5
Key Points• System Test Coordinators (STC) can return scorable and non-
scorable documents separately and at different times. – Ship scorables to the Georgia Center for Assessment (GCA)
as soon as possible after the conclusion of testing. – Once schools have returned all scorable answer
documents, the STC may ship to GCA (and we encourage you to do so).
• You are not required to wait 5 days to ship for scoring.• The timely return of scorables is critical to the attainment of the
necessary representational sample for reporting.
6
NEW
Manuals
The School and System Test Coordinator’s Manual and the Test
Examiner’s Manualshttp://www.gadoe.org/Curriculum-Instruction-and-Assessment/Assessment/Pages/CRCTM-Resources.aspx
7
CRCT-M Overview
Will cover the following:–Main Administration–Retest Administration–Exception Testing–Test Construction
8
Main Administration
• Main Administration: April 1- May 3– Eligible students with IEP’s
Grades 3-8– Reading– Mathematics– English/Language Arts
9
Retest Administration
• Retest Administration: May 13 – July 26– Includes students who perform at Level 1 in
Reading (grades 3, 5, & 8) or Math (grades 5 & 8) during the Main administration.
• Exception Testing– Qualified students who miss the system’s
scheduled Retest window may receive an exception to test within the state Retest Administration window.
– All requests for exception testing must be approved through the GaDOE.
10
Test Construction
• Content Areas– Grades 3-8: Reading, ELA, & Math– A student may take between 1 – 3 CRCT-M
tests• Will take CRCT in any content area that is
not assessed by CRCT-M– All students will take the CRCT in Science and
Social Studies
• All items are multiple-choice– There are 10 fewer items than on the CRCT.
11
Test Construction
Number of Items by Subject
Operational Field Test Total
Reading 30 10 40
ELA 40 10 50
Math 50 10 60
12
Test Construction
• Field Test Items– Help to create a pool of items for future
administrations.– Are included with Operational items in
all content areas at all grade levels.– Are not reported as part of the student’s
score.
13
Pre-Administration
Will cover the following:– System Test Coordinator
Responsibilities– School Test Coordinator
Responsibilities– Receiving Your Materials Shipment– Distribution to Schools– Security– Pre-ID Labels– Coding for GNETS
14
System Test Coordinator Responsibilities
• Reviews the Test Coordinator’s Manual (TCM) and Test Examiner’s Manuals (TEM).
• Organizes the training sessions for School Test Coordinators.
• Receives CRCT-M materials and verifies that all boxes have been received.
• Distributes CRCT-M materials to School Test Coordinators.
• Please order additional materials as soon as possible.
15
School Test Coordinator Responsibilities
• Reviews the Test Coordinator’s Manual (TCM) and Test Examiner’s Manual (TEM).
• Organizes training sessions for Test Examiners.
• Receives CRCT-M materials from the System Test Coordinator and verifies that all boxes have been received.
• Notifies the System Test Coordinator if additional materials are needed.
16
Receiving Your CRCT-M Materials Shipment
• System Shipment includes:– System and School Packing List– Shipment Verification Form– System and School Security Checklists– Return Shipping Labels– System Overage materials (see next slide)
17
Overage Materials
• System Test Coordinators will use the System overage materials to provide additional materials to those schools that exhaust the School materials.– Group Information Sheets (GIS)– Test Coordinator’s & Test Examiner’s Manuals– Test Booklets and Answer Sheets– Paper bands– Empty cartons
18
Overage Materials
• For grades 3 – 8, you will receive an overage of 30% (with a minimum of 2 booklets) at the system level.
• System Test Coordinators may order additional materials from GCA.
• Extra test booklets will also be sent for Read-alouds.
19
Receiving Your Materials Shipment - Systems
• Arrival of test materials– Shipping dates are estimated based on the dates
provided through TestTime in December/January.– Test materials will be shipped to systems at least two
weeks prior to testing.– Shipping schedule will take into account Spring Breaks.
• Complete the System Inventory Checklist– Step-by-step instructions to be completed by the Test
Coordinators immediately upon receipt of materials.– System Inventory Checklist provided in the TCM on page
15.
20
Distribution to Schools• School Test Coordinators
– Complete the School Inventory Checklist provided in the TCM on page 17.
• Boxes are labeled for each School and include:– School Packaging List & School Security Checklist– Group Information Sheets (GIS)– Test Coordinator’s Manuals & Test Examiner’s Manuals– Test booklets & answer sheets
• Schools receive an overage of materials– If additional materials are needed, School Test Coordinators
contact the System Test Coordinator.
• Retain boxes for return shipping
21
Security
• All systems must adhere to the 5-day test window. The days must be consecutive and do not include weekends.
• Materials must be kept in secure storage before, during, and after testing
• Test books remain secure until distributed to Test Examiners on the first day of testing
22
Security• School Test Coordinators use the School Security
Checklist to check materials out to Test Examiners. All materials are returned immediately to the School Test Coordinator for secure storage at the end of each day’s testing session.
• Each classroom must have adequate staff to monitor testing.
• Test Irregularities: Contact GaDOE for instructions. You may be asked to complete the Test Irregularities form online on the MyGaDOE Portal.
23
Pre-ID Labels
• Test ID – Transmission instructions for Pre-ID Cycle 3
required that systems use 09 to identify students who will take CRCT-M this spring.
– Verify with your FTE Coordinator that your district complied with these instructions.
24
Pre-ID Labels• Pre-ID labels will come with CRCT-M School
Test Coordinator Kit.• Two barcode labels for each student come with
CRCT-M.– Only 1 label will be used for CRCT-M
• Visible information includes:– Student name & GTID– School and System names & codes– Grade– Student DOB and gender– Ethnicity, Race, and PSRC– Barcode number
25
Pre-ID Labels
• Do not use if inaccurate• Barcodes override bubbling
26
Pre-ID Labels• Student information is provided to GCA by
GaDOE. If the Pre-ID label is incorrect:– Examiners must notify the School Test Coordinator– School Test Coordinator must update the schools
student record
• Do not use the label if information is incorrect. – Barcode information will override bubbled information.
• When unavailable, information must be bubbled on the answer document.
• When bubbling demographic information, fill in the text boxes first, then bubble.
27
Pre-ID Labels• Barcode applied incorrectly
– If answers have been gridded:• Transcribe student answers & demographics onto a replacement
answer sheet.• Write VOID over the original answer sheet and return it with your
non-scorables.
– If answers have not been gridded:• Write VOID over the original answer sheet and return it with your
non-scorables.
• Student transfers– Not applied to an answer document:
• Destroy the barcode.
– Applied to an answer document:• Write VOID over the original answer sheet and return it with your
non-scorables
28
Coding GNETS Programs• In order to receive results for GNETS Programs, an
identifier code needs to be bubbled on each student’s answer document in the box marked GNETS.
• Codes and instructions are included in the TEM and TCM.
Alpine Program 01Burwell Program 02Cedarwood Program 03Coastal Academy Program 04Coastal GA Comprehensive Academy 05Mainstay 06Dekalb-Rockdale Program 07Elam Alexander Academy 08Flint Area Learning Center 09Harrell Learning Center 10H.A.V.E.N. Academy 11Heartland Academy 12
Horizon Academy 13
Northstar Educational & Therapeutic Services 14
North Metro Program 15
Northwest Georgia Educational Program 16
Oak Tree 17
GNETS of Oconee 18
Pathways Educational Program 19
River Quest Program 20
Rutland Academy 21
Sand Hills Program 22
South Metro Program 23
Woodhall Program 24
29
CRCT-M Answer Sheet
30
CRCT-M Retest Answer Sheet
31
NEW
TESTING
Will cover the following:– Preparation– Administration– Accommodations– Testing in Small Groups– Irregularities
32
Testing Preparation• Remove or cover instructional materials that may
prompt student responses.• Arrange seating to promote individual work.• Post a sign on the outside of the classroom door:
“Testing: Do Not Disturb.”• One proctor is recommended for up to 30 students and
required for more than 30 students• Examiner Materials:
– Test Examiner’s Manual– Test Books and Answer Sheets– Scratch paper for math– Two No. 2 pencils with erasers per student– No calculators, unless specified in IEP
33
Test Administration• Students take one content area per day
– Prescribed order: Reading, ELA, Math
• Both sections from a single content area must be administered the same day.
• Students can write in the test books. Highlighters are not allowed.
• Students take a stretch break.• Teacher monitoring – “Circulate & Check” to
prevent cheating, use of calculators, going ahead, or looking back.
34
Accommodations• Be familiar with accommodations listed in the IEPs.• Accommodations must
– be required by the student in order to used during the assessment
– be provided during routine instruction and assessment in the classroom
• Accommodations may not– alter, explain, simplify, paraphrase, or eliminate any test
item, reading passage, or choice option– Provide verbal or other clues or suggestions that hint or give
away the correct response to the student
• For Large Print and Braille, transcribe student answers on to a regular answer document.– Follow best-practice from Student Assessment Handbook
35
Coding Accommodations
• State Required Codes (SRC)• Accommodation Type (Setting, Presentation,
Response, Schedule)• Conditional Accommodation• Braille/Large Print• Read-Aloud Guidelines – in TEM
– Sample test items for Read-Aloudhttp://www.gadoe.org/Curriculum-Instruction-and-Assessment/
Assessment/Pages/CRCT-Resources.aspx
36
Coding Accommodations
If an Accommodation Type is bubbled; ACCOMMODATIONS must be bubbled also.
Bubble SRC even if it is included on label; however, if the current SRC is not on the label or does not match, discard the label and bubble all demographic information ..
37
Coding Accommodations• On each content-area test page
– EL Deferred will only appear in Reading and ELA
38
Testing in Small Groups• The CRCT-M is a test designed for small numbers
of students. • Therefore, districts are assigned only 1 form.
We have taken care of sampling distribution across forms.
• Districts will not receive any special packaging of small group materials.
• Because districts receive only 1 form, all test materials can be used for those students who need a special administration of the exam in a small group environment.
39
Irregularities (IR)• Events and circumstances, or departures
from standardized testing procedures are irregularities. – They may have an impact on student
performance that is not possible to define.– They are reported and student scores are
flagged simply to say, “There is something different about the conditions under which this score was obtained. Use caution in interpreting the score.”
40
Irregularities (IR)• All test administration irregularities must be
reported by Examiners to the School Test Coordinator.
• The School Test Coordinator sends irregularities to the System Test Coordinator for verification and to obtain the correct code.
• The System Test Coordinator contacts GaDOE immediately for direction.
41
Irregularities (IR)
• If instructed to do so by GaDOE, the System Test Coordinator will investigate and complete the Testing Irregularities Documentation Form provided online on the MyGaDOE Portal.
• Improperly coded irregularities will default to the holding of test scores.
42
Examples of Irregularities (IR)
• Irregularities in Test Administration:• Materials being distributed inappropriately• Directions not being followed• Improper monitoring of test administration• Incorrect/inappropriate accommodations
provided for students
43
Invalidations (INV)
• Some irregularities are so serious in the way they affect student performance or compromise the integrity of the testing that they result in an invalidation.– Content being disclosed, coached, or
distributed– Cheating– Test Administrator altering responses during
or after testing.
44
Participation Invalidation (PIV)
• In addition to irregularities (IR) and invalidations (IV), systems will have to code for Participation Invalidations (PIV) for students who receive an accommodation that is not on the list of state-approved accommodations or not on the student’s IEP.
• A potential PIV occurs when a student receives an unapproved conditional accommodation such as the reading of the test passages or use of a calculator on the Math test.
• Federal regulations stipulate that when inappropriate accommodations are used, students may not be considered participants in accountability calculations.
45
Participation Invalidation (PIV)
46
• A potential PIV will also occur when students who are ineligible to take the CRCT-M are given the test.– CRCT-M mistakenly given instead of the CRCT.– CRCT-M given to students who earned scores
at or above 330 (Basic Proficiency: PL3) for two consecutive years.
– CRCT-M administered to students who Met or Exceeded Standard on the CRCT in 2012.
NEW
Coding Irregularities and Invalidations
• Incorrect coding may result in no reports being sent.
47
Reducing Irregularities
• Ensure proper training of all test administrators• Maintain clear and consistent training and
support materials.• Make certain all accommodations are clear to
test administrators particularly if the Test Examiner is not the students’ regular teacher.
• Make certain that Test Examiners are constantly monitoring the testing environment.
• Respond to student actions appropriately and immediately.
• Implement a program for reporting irregularities during testing.
48
Characteristics of a Quality Investigation
• Examiner should notify Principal and School Test Coordinator of incident.
• School Test Coordinator should notify System Test Coordinator.
• Request detailed written statements from all parties involved if necessary.
– Follow up on details (connect the dots).
• Follow district procedures regarding misconduct.
49
Characteristics of a Quality Investigation
• Provide a written summary of incident and investigation findings (page 171 SAH).
• Provide a written summary of incident and investigation findings.
• Consult with GaDOE as needed.• System Test Coordinator is responsible for reporting
to GaDOE.
• Report to GaDOE immediately, before investigation.
• Final documentation should be entered into portal.
50
Post-Test Activities
Covers the following:• Test Examiner Responsibilities• School Test Coordinator Responsibilities• System Test Coordinator Responsibilities• Packing Return Shipment• Return Shipment Pick-up• Returning Make-up Materials• Return Shipment Errors
51
POST-TEST ACTIVITIES: Test Examiner
ResponsibilitiesTest Examiner shall:• Collect all test materials daily.• Review answer documents.
– Where applicable, verify that the identifying information is correctly gridded.
– If instructed by the System Test Coordinator, complete Local Use Coding (Section 13)
– Complete Sections 10-11 (on front page of answer document) and 18-19 (on each content area test page)
– Transcribing – monitored by certified educator
52
POST-TEST ACTIVITIES: Test Examiner Responsibilities
Scorables• Completed Group
Information Sheet (GIS)
• Used Student Answer Sheets
Non-Scorables•Used Test Booklets •Large Print and Braille Test booklets•Unused and Voided Answer sheets•Unused Test Booklets•Mathematics Scratch Paper•Manuals (TEM & TCM)•Unused GIS•Scratch Paper
53
POST-TEST ACTIVITIES: School Test Coordinator
54
Transposing Student Answer Responses:• When required and prior to returning answer sheets to
the System Testing Coordinator, verify student answer responses have been transposed to an answer sheet.
• Transposing should be witnessed by a certified educator. If only two people are present one must be an administrator (SAH Page 43).
• Complete the VALIDATION FORM FOR TRANSPOSING ANSWER DOCUMENTS found on Page 172 of the Student Assessment Handbook. File one copy with school testing records and return a copy to the System Testing Coordinator.
POST-TEST ACTIVITIES: School Test Coordinator
• Verify that Test Examiners have returned all materials. – Materials for make-up testing administered outside of
the system’s scheduled window must be ordered by the System Test Coordinator.
• Complete the School Security Checklist and place in the return box.
• System Test Coordinators will need to make copies of any pages that indicate missing or non-returned materials and place these copies in the security checklist envelope for each school. System Coordinator’s will retain all original school security checklists.
• Verify that “VOID” is written across every voided Answer Sheet. Return voided documents with your non-scorable test materials.
55
POST-TEST ACTIVITIES:School Test Coordinator
• Destroy all Mathematics scratch paper.
• Destroy all soiled documents (biohazard). – Record as soiled on the School Security
Checklist.
• Arrange all used documents with the student name face-up.
56
• Verify that each Group Information Sheet (GIS) is filled out correctly.
POST-TEST ACTIVITIES:School Test Coordinator
57
POST-TEST ACTIVITIES:School Test Coordinator
58
• Secure each grade’s materials with a paper band using one GIS.
• Pack scorable and non-scorable materials separately.
Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org
POST-TEST ACTIVITIES: School Test Coordinator
Scorables
– GCA will send additional boxes (flattened)with original shipment.
– Pack scorables in the order indicated in the graphic.
– A Group Information Sheet should be placed on the top of each grade level packet.
– If more than one box is needed, do not separate grade level packets.
59
Dr. John D. Barge, State School Superintendent“Making Education Work for All Georgians”www.gadoe.org
POST-TEST ACTIVITIES: School Test Coordinator
Nonscorables
– Use original shipping boxes.– Pack all nonscorable items in the box, including:
• All unused manuals• Unused and voided Student Answer Sheets• Unused Student Test Booklets• Used Student Test Booklets• Unused GIS• Math scratch paper • Return to the System Test Coordinator per their instructions
•All Braille materials should be returned in the original packaging as shipped to the school.
60
POST-TEST ACTIVITIES: System Test Coordinator
Responsibilities• So that districts are prepared to address “non-
participants” during accountability processing, make sure to:– Collect by school and by subject … the names of each
student who should have tested, but was a non-participant
– Districts will be asked to categorize their “non-participation” as one of the following four choices…Medical Emergency, Illness, Other, Unknown
– Direct questions on the definition of “non-participant” and the above categories to your district’s GaDOE Accountability Specialist
61
POST-TEST ACTIVITIES: System Test Coordinator
• Prepare materials for pick-up.Reminders:– Do not consolidate materials from more than one school
in the same box.– Use one label per box.
• Use UPS Return Shipping Labels for all test materials.• Note: You will not need any additional shipping labels to return
your materials.
– Identify the sequence of boxes for each school on each shipping label (example: Box 1 of 10).
– The Security Checklist should be placed in Box 1 of the scorable shipment (Shipment 1).
– Securely seal all boxes.
62
POST-TEST ACTIVITIES: System Test Coordinator
• Follow the shipping instructions that appear in your test materials and the TCM.
63
POST-TEST ACTIVITIES: Returning Make-up
Materials• Make-up Testing
– In most systems, make-up testing is accomplished within the system’s scheduled main administration window. These materials should be returned with the main administration materials.
– For make-up testing outside of the system’s scheduled testing window, contact GCA for the appropriate testing materials.
64
POST-TEST ACTIVITIES: Return Shipment Errors
• Document preparation errors and packaging errors slow down the scoring process and can delay your score reports.– Document preparation errors
• Missing GIS• Schools packed together
– Document data errors• Xeroxed GIS• Student marks on timing tracks• Barcode incorrectly placed
– Pre-ID placed over the wrong section65
POST-TEST ACTIVITIES: Return Shipment Errors
• Packaging & Transportation Errors– Use of boxes other than the custom CRCT-M
shipping cartons– Boxes incorrectly numbered– All materials not available for pick-up as
scheduled– Unlabeled boxes– Labels applied incorrectly– Self-scheduling using a different carrier
66
CRCT-M Reports• Electronic Test Results via My GaDOE Portal
– Available after completion of post-administration analyses with 50% of population
– PDF files – Building Roster by Grade (with summary page)– PDF files – Performance Level 1 Roster– System Data file and Record Layout– District testing early in the window should return materials
as soon as possible
• Paper reports (follow receipt of electronic results)– Building Roster by Grade (with summary page)– Performance Level 1 Roster– Individual Student Report– Student Score Labels
67
Summary Reports
• Paper Results (Delivered to Systems in Early July)– Summary Report of All Student Populations
School, System, and State Performance Summary Report School, System, and State
68
Retest Reports
• Electronic Test Results via My GaDOE Portal– PDF files – Building Roster by Grade (with summary
page)– PDF files – Performance Level 1 Roster– System Data file and Record Layout
• Paper reports (follow receipt of electronic results)– Building Roster by Grade (with summary page)– Performance Level 1 Roster– Individual Student Report– Student Score Labels
69
Ordering Materials
• When System overage materials have been exhausted, System Test Coordinators can order additional materials by going to:
https://gcap.tsars.uga.edu/testtime
• Orders will be shipped using 2nd Day shipping
unless expedited shipping is needed. • Requests for Advanced Reading Aids (Kurzweil) are
due February 18.– Fax form to Mary Nesbit-McBride at 770.344.4503
(page 144-145 Student Assessment Handbook)
NEW
70
Ordering Materials
71
CRCT-M CONTACT INFORMATION:
Shipping Information• Shipments to Districts:
– Your initial shipment of CRCT-M materials will be delivered by UPS.
• For more information Call GCA– (888) 392-8977
– Deliveries will be made at least 2 weeks prior to the first day of testing, taking spring break into account based on the dates provided to GCA through TestTime.
72
Shipping Information• Systems must schedule pick-up as
soon as testing is completed.• Return scorables within 5 days from the end of CRCT-M
testing.• Return non-scorables within 10 days from the end of
CRCT-M testing.
• Must call UPS to schedule pickup• Phone: 1-800-377-4877• Shipments are all prepaid and the shipping code appears
on the UPS label.• The How to Schedule a Pickup Using UPS Returns Services
document provides detailed shipping directions.
NEW
73
IMPORTANT• Once schools have returned all scorable answer documents,
the STC may ship to GCA (and we encourage you to do so).• You are not required to wait 5 days to ship for scoring.• The timely return of scorables is critical to the
attainment of the necessary representational sample for reporting.
• If a student’s answers are supposed to be transposed to an answer sheet, please check and be sure that this has been done before test materials are shipped to vendor.
74
CRCT-M CONTACT INFORMATION: GCA Contacts
• Georgia Center for Assessment (GCA)– Phone: (888) 392-8977
• Steve Cramer, Director– Phone: (706) 542-5589– E-mail: [email protected]
• Stephanie Lai, Program Manager– Phone: (706) 542-6606– E-mail: [email protected]
75
CRCT-M CONTACT INFORMATION:
Georgia Department of Education
Mary Nesbit-McBride, Ph.D.CRCT-M Program Manager- Phone: (404) 232-1207- E-mail: [email protected]
Tony EitelAssessment Administration Director- Phone: (404) 656-0478- E-mail: [email protected]
76
Questions?
77