2013-2014 tcap administration training for school assessment coordinators

2013-2014 TCAPAdministration Training

for School Assessment Coordinators

Thompson School DistrictColorado Department of Education

Who should be trained?

Everyone involved in TCAP administration must be trained each year. This includes DACs, SACs, Proctors and any other school or district staff involved in any aspect of TCAP administrationDACs/SACs must collect and maintain a record of signed training completion forms as well as signed confidentiality agreements for all who come in contact with test materialsSACs responsible for ensuring that all test proctors are properly trained

Overview

1. Colorado Achievement Assessments2. Personnel / Core Assessment Team3. Important Dates4. TCAP Assessment Administration6. Training7. Test Accommodations8. Test Security9. Data Quality10. Communication and Resources

3

Colorado Achievement Assessments

TCAP vs. CoAlt

TCAP

CoAlt

ACCESS for ELLs

• Timed• Whole class• Scored only by CTB• No breaks during session

• Untimed• One-on-one• Partially scored by examiner• Students may take breaks

• One-on-one and grade span groups

• Partially scored by administrator

TCAP Assessment

PURPOSE: To measure student achievement relative to the Colorado Model Content Standards. The TCAP replaces the CSAP in order to support schools as they transition to curricula aligned with the Colorado Academic Standards.

26 tests Reading and Writing - Grades 3-10 (Grade 3 Reading separate)

Mathematics - Grades 3-10 (calculators certain sessions Grades 9 & 10)

Lectura/Escritura (Grades 3 & 4)

Personnel and Core Assessment Team

Core Assessment Team District Assessment Coordinator (DAC)

Carmen Williams

Assessment Specialist Ronda Hanson Andy Riley

English Language Administrator Clint Richards

Director of Exceptional Student Services Charlie Carter

Director of Elementary Education Dr. Paul Bankes

Director of Secondary Education Dr. Margaret Crespo

District Assessment Coordinator

What is a DAC? District Assessment Coordinator Appointed by the district superintendent Official point of contact with CDE

Responsibilities vary by district, but always include: Training School Assessment Coordinators (SACs) Ensuring all students are assessed Ensuring standardized administration of assessments Declaring, investigating and reporting misadministrations Managing and communicating assessment score results

Important Dates

Important Dates All testing, including make-ups, must be completed

within our district ’s testing window February 5 through March 14 – CoAlt testing February 12 – Housekeeping paperwork and training

verifications due/ 3 rd grade Reading Teacher Read Direction order forms due

February 19 –Test proctor training needs to be complete for 3 rd grade Reading Proctors

February 20 – Oral Presentation and Teacher Read Direction order forms due

February 20 – TCAP Testing Schedules due March 6 – Test proctor training completed

Important Dates(Continued)

All testing, including make-ups, must be completed within our district ’s testing window

3 R

February/ March24 25 26 27 28 3 4 5 6 7

M T W TH F M T W TH F

1 2

March10 11 12 13 14 17 18 19 20 21


3 W 1 2

3 M 1 2

4 R/W 1 2 3 4 5 6

4 M 1 2 3

5 R/W 1 2 3 4 5 6

5 M 1 2 3

6 R/W 1 2 3 4, 5 6

6 M 1 2 3

7 R/W 1 2 3 4, 5 6

7 M 1 2 3

8 R/W 1 2 3 4, 5 6

8 M 1 2 3

9 R/W 1, 2, 3 4, 5, 6

9 M 1, 2, 3

10 R/W 1, 2, 3 4, 5, 6

10 M 1, 2, 3

Grade

Total number of

Testing Days

3 64 95 96 57 58 69 310 4

Make-ups / Check-in Schedule

3 R

February/ March

24 25 26 27 28 3 4 5 6 7


MU MU MU √ in √ in

March April

14 17 18 19 20 21 24 25 26 27 28 7 8 9 10 11

F M T W TH F M T W TH F M T W TH F

3 W Make-Ups Check-ins

3 M Make-Ups Check-ins

4 R/W Make-Ups Check-ins


5 R/W Make-Ups Check-ins


6 R/W MU MU MU MU Make-Ups Check-in

6 M MU MU MU MU Make-Ups Check-in





9 R/W Make-Ups Check-in

9 M Make-Ups Check-in

10 R/W Make-Ups Check-in

10 M Make-Ups Check-in

TCAP Administration

Essential Understandings

Assessment results are part of the student’s permanent academic record per Colorado law [22-7-409(1.9)].

Every student enrolled in a public school shall be required to take the (state) assessments (in the content areas and grades administered) [22-7-409(1.2.a.1.d.I)].

“Standardized” Assessment

Standardization - All students will have the same test content, resources, directions, testing conditions, and scoring procedures

No students are provided an unfair advantage Standardization requires that all proctors administer

all test sessions in the same way A score obtained by one student in one part of

Colorado will mean the same as the same score obtained by another student in a different part of Colorado

Permitted Practices

Due to the complex nature of a standardized assessment process, any practice not specifically permitted should be presumed inappropriate until and unless specifically authorized by CDE.

“If it doesn’t say that you can, then you can’t.”

Standardization and the Media

Media presence at a school during testing can create a non-standard assessment environment by causing disruptions in the regular flow of the school day or by adding increased stress on the students being assessed

Media are expressly prohibited from taking pictures or video of test materials or students in the process of being tested. CDE notifies the media of this prohibition prior to testing time

TCAP School Schedule Deadline

TCAP school schedules are due to Ronda Hanson at the Department of Accountability and Accreditation per the following schedule:Grade 3 Reading – February 12 th, 2012All other Tests – February 20 th, 2012

Schedules must include start and end times for each grade and content

School TCAP schedule template will be posted on the AA Department wiki space

http://tsdaccountabilityaccreditation.wikispaces.com

TCAP Schedule Students in the same grade, at the same school, must

take the same session(s) at the same time Only one grade/content can be checked out to a test

proctor at a time Test sessions must be administered in the order they

appear within each test bookMake-ups administered for missed sessionsSession 1 Writing must happen before Session 2 Writing

SAC must keep on file a schedule of exactly when and where each proctor is administering which assessments to which students

Schedule should allow time for proctors to read the directions from the Proctor ’s Manual, to distribute materials before and to collect and secure materials after administration of each assessment

Considerations for accommodations should be included in the development of the schedule

TCAP Schedule

In General, The Test Environment

The testing environment must: Be adequately lit, quiet, free of distractions, and

heated or cooled Provide an adequate writing surface (free of other

materials) Be free of cell phones, music or other distracting

devices “Do Not Disturb” and “No Cell Phones” signs must be

placed on the door during test sessions No food or drinks are allowed on desks or near test

materials

The Test Environment CANNOT

Must be free of any content related posters or aids that suggest possible answers to students Word walls Steps for long division Multiplication tables Periodic Table of the Elements

Must be free of all reference materials that a reasonable person might conclude offers students an unfair advantage

***Name tags – must be writing resistant (laminated or clear plastic tape) and must not include reference materials

Posters that do not include content specific definitions, content related processes or solutions may remain on the wall A picture of the Earth The Alphabet Basic Number Lines

May include study carrels IF they are part of normal classroom environment

The Test Environment CAN

Forbidden Devices : No one (including test proctor) is allowed to have cell phones or any other device with the capability to reproduce, transmit or record secure testing materials or information about secure testing materials in the testing environment.

Authorized Personnel Only : Parents, the media, and anyone except authorized school personnel are prohibited from being in the test environment.

Open Desks : Student desks which are open and contain student materials should be organized. All instruments not allowed to be used on TCAP should be made inaccessible.

More on the Test Environment

Proctoring TCAP

Only one test session at one grade level and in one content area may be given in a testing environment

If a student finishes a test session early, they may read or sit quietly but must not write or do any other activityAll test materials must be removed before a student begins

reading All test proctor ’s must meet the qualifications for

proctoring as described in the Procedures Manual (pages 27-28)

All test proctors must undergo school or district training before administering the TCAP (proctor training is required annually)

Proctoring is active

Active Proctors SHOULD

Ensure that the testing environment complies with TCAP instructions

Ensure they have all necessary materials for each session Follow all scripts exactly as written Move throughout the room during testing Encourage on-task behavior by using proximity

Active Proctors Should NOT

Interact with students in any way that would impact student responses (verbal, gestures, or in writing)

Provide feedback Clarify test questions Answer content related questions Interfere with the students’ demonstration of skills Engage in other tasks during test sessions Read through test booklets or test items

Proctor Situations

Dealing with disruptive students Disruptive students should be removed from the testing environment. If

possible, they may continue the session in a designated area. The student’s test score may be invalidated if the session cannot be completed

Test Proctors should never leave testing students unsupervised while dealing with a disruptive student

Dealing with a concerning student response On rare occasion, students make concerning responses to test items, such

as responses that include information about threats, violence, abuse or other concerns

If discovered before returning test books, these items should be reported to school administration and to the SAC who in turn must inform the DAC

Unexpected Situations

If a student becomes ill during a test: The student’s needs are the primary consideration The student may make up the test The student may not change answers in a session that was

already begun The student may only have the amount of time that was left

when the test was suspended

If there is an emergency requiring the test to be stopped: Evaluate and respond to the emergency… safety first! If it is possible to do so without any risk to students, note the

time remaining in the test session Students will be allowed to use the remainder of the testing time

to complete the session at a later time Notify DAC as soon as possible Document situation in writing

Unexpected Situations

Misadministrations

A misadministration is any event that leads to the invalidation of one or more student test scores in one or more given test sessions DACs investigate, identify and declare misadministrations Major Misadministrations must be reported to CDE

Also reported to Office of Licensing if it involves a teacher or school (or district) administrator

Misadministrations

Any deviation from established standardized testing procedures A student moves on to a new test

session or takes the wrong session A student receives help from anyone

on a test item A student uses an unauthorized

instrument, such as a calculator, cell phone or notes, during a test session

A student is denied appropriate accommodations or given the wrong accommodations

Major Misadministrations(three main types)

Misadministrations affecting an entire class or group of students The wrong test session is administered to a class

Systematic unethical behavior A teacher, administrator or other person gives students hints,

prompts or answers to questions Students obtain or share secure test materials

Breach of secure test materials Discussing, reproducing, or transmitting, by any means, secure

test materials, or descriptions of secure test materials

All students in the same grade level must be given the same test session at the same time

The school is solely responsible for proper administrations Prevention: Create schedules and emphasize communication

Only one test book is checked out at one time Say, “Session 3 Reading” Do not say, “Reading Session 3”

Verification Call backs Classroom checks

Corrective Action Students must be kept separate Corrective action must be taken by the end of the school day or a

misadministration will be declared

Administering the Wrong Session

When a Misadministration Occurs The student must immediately stop working on that session, but

should still complete other test sessions as normal The Test Proctor must immediately inform the SAC, who must

inform and consult with the DAC The appropriate misadministration bubble should be filled in on

the student data grid The student receives a “no score” for the content area

District Policies for Misadministrations

Misadministrations are declared to preserve the reliability of test scores and assessment data

The declaration of a misadministration should not be used as a disciplinary measure

A record of misadministrations must be kept by the DAC Reports or suspicions of major misadministration must be

reported to CDE as soon as practicable CDE reserves the option to investigate and declare

misadministrations and apply its own disciplinary sanctions

Training

Training

SACs are responsible for training all staff involved in any aspect of TCAP administration

Training must include: Test Security including Chain of Custody Test Proctor Requirements Ethics

Required documentation Training Participation Agreement Confidentiality Agreement – all who come in contact with test

materials Certification due by March 6 th

Training

47

Accommodations

Standard Accommodations

• Students with a documented need

Nonstandard Accommodations

• Students with an IEP or 504

Linguistic Accommodations

• Students identified as English Learners

49

Planning for Reading/Writing

Accommodations – 2014 sessions

Session1 Session 2 Session 3 Session 4 Session 5 Session 6

Grade 4 Writing Writing Reading Reading Writing R/W







51

Oral Script

Small Groups < 15 students

Must be requested by February 20

Teacher Read Directions

English or native Language TCAP Reading, Writing, and Mathematics Follow directions exactly Separate environment Individually or small group

Grade 3 Reading due February 12th

All other must be requested by February 20

Oral or Written Student Response

Lectura/Escritura Grades 3 & 4

Must be administered in a separate environment by a Spanish-speaking test proctor

Administration instructions are provided in the English TCAP Test Proctor ’s Manual

Scheduling and administration are the same as for the English assessments

Student Data Grids on all test books are identical – both for English and Lectura/Escritura

STOP pages – must be printed and inserted before testing and removed before shipping and scoring

Oral Scripts will be available for Escritura

Assistive Technology

Assistive Technology is a Standard Accommodation iPads are allowed to be used on TCAP tests All Assistive Technology programs must have grammar checking

capability disabled for testing Assistive Technology devises are considered secure testing

materials and must be secured as such All materials must be printed and transcribed into a scannable

test booklet All student testing materials must be removed from the device

immediately following printing A Verification of Removal of Saved Data is required for each

school

Extended Time

Extended Time is NOT unlimited time. Extended time permits students 90 minutes in the testing environment

Test proctors can use their professional judgment and allow a brief extension past 90 minutes to allow students to finish their work (Accommodations Manual page 153)

Students in an extended time environment must remain in the test session, even if they complete the test within the standard time allotment

Only students who USE the extended time should have the Extended Time Used accommodation bubbled

Mark All Accommodations

Security

Test Security

Two main goals: Protect sensitive student information and data Ensure the validity of the state assessments

Materials –Secure – Not Reproducible

Test materials that may never be reproduced: Test Booklets Draft Booklets Student Answers

Materials – Secure but Reproducible

The DAC may reproduce the following materials. They must be returned with “Not to be scored” materials. Oral Scripts Translated Oral Scripts Teacher Read Directions Translated Teacher Read Directions

Materials –Non-secure Materials

These materials are non-secure and do not require documentation or chain of custody Procedures Manual Accommodations Manual and Accommodations Guide Test Proctor’s Manual / Test Examiner’s Manual DAC/SAC Manual Guide to Test Interpretation CSAP/TCAP Released Items (Assessment Unit website) Punch-out Tools (Not secure, may not be reproduced)

Chain of Custody

Well documented before, during, and after testing Materials must be accounted for and stored in a

secure and locked location before, during, and after testing

Materials delivered to schools no more than 1 week in advance

SAC responsible for distribution and collection of materials each day of administration

SAC must be able to assure the DAC that every step to ensure security has been taken

Chain of Custody Document

Test materials must be stored in a secure and locked location and must only be checked out to authorized individuals at the time of testing

Test materials should be tracked on the Chain of Custody document to ascertain who had what materials at what time (who, what, and when)

Chain of Custody document should be reviewed at end of each testing session to ensure that all test materials have been returned

Risk of major misadministration without proper documentation

65

Chain of Custody & Materials

All test books must be kept in a secure, designated area Test books and other secure test materials must not be kept

in classrooms Proctors must check out materials each day and check

them back in at the end of each testing day Proctors should ensure they have enough test books for

the scheduled sessions When two content areas are tested on the same day,

both sets of books must not be checked out to proctors at the same time

Basic Principles inMaintaining Security

Test books must be secure while in the TCAP Test Proctor ’s possession

No cell phones or other communication or recording devices are allowed in the test environment

Test items should never be discussed – verbally or in writing

Questions about Items• Proctors should not be studying items

• Items only accessed during test administration

• If a test proctor has a question about an item… Contact the SAC, who will contact the DAC

The DAC contacts the appropriate person at CDE in the Office of Student Assessment by phone

Include in proctor training instructions: Do Not copy the item Do Not discuss the item (with other teachers, etc.) Do Not email the content of the item

Housecleaning Every SAC must sign the School Verification of Training and

Housekeeping document: ensuring that there are no assessments or proprietary materials

related to the state assessments stored anywhere on the school premises (test booklets from previous years, etc)

Ensuring that any teacher-created materials (whether distributed or not) are free of actual assessment items Does not apply to TCAP released items

Due by March 6th

Data Quality

Use only a standard, wooden, graphite-based, #2 pencil on any part of the test book…. including the front cover!

Front Cover of Test Book

Student Data Grid

Mark All Accommodations

Mark ONE Test Invalidation

Transcribing Books

When transcribing is NOT an accommodation Damaged books Alternate formats (Large Print and Braille) Assistive technology used

When transcribing IS an accommodation When a student responds in a language other than English (Math and

Reading) Must be done in a scannable test booklet

Communication and Resources

Office of Student

Assessment

School Assessment Coordinators

CTB McGraw-Hill

DACand

Core Team

Test Proctors, Examiners and Administrators

Communication

Assessment Resources

2013-2014 Procedures Manual

2013-2014 Accommodations Manual and Guide

2014 DAC/SAC Manual

2014 Test Proctor ’s Manuals

Accountability & Accreditation wiki space

Ronda Hanson (613-5024)

Andy Riley (613-5019)

Assessment Binder

SignsKey Points to cover in trainingCopy of the letter sent to Superintendents regarding testingSchool Verification of Training and HousekeepingTCAP Security of Testing Materials CoAlt Security of Testing MaterialsTCAP Daily Accounting of Testing MaterialsCoAlt Daily Accounting of Testing MaterialsProctor Training Participation AgreementConfidentiality and Participation Agreement2014 TCAP/ CoAlt Timeline

Assessment Binder(continued)

TCAP SAC ChecklistCoAlt SAC ChecklistTCAP Check-in ChecklistCoAlt Check-in ChecklistOral Presentation & Teacher Read Directions RequestTCAP Parent RefusalTCAP Not Tested/ Test Invalidation FormCoAlt Not Tested/ Test Invalidation FormVerification of Removal of Saved Data2013-2014 Procedures Manual

Thank You!

Thanks for taking time to participate in this required training

Be sure that you have signed the record of attendance

2013-2014 tcap administration training for school assessment coordinators

Documents

tcap administration

test proctors

tcap proctors

proctor tcap

test books

test examiners

untimed test

test accommodations8