TELPAS 2015Rater & Verifier TrainingPresented by

*This training does not replace your responsibility to read and follow all manuals and supplements*

What is TELPAS?The Texas English Language Proficiency Assessment System assesses all students who are identified as LEP including parent denials in the domains of reading, writing, listening and speaking

K-1 2-12

ReadingHolistically rated

observational assessmentsMultiple choice online

reading test

WritingHolistically rated

observational assessmentsHolistically rated student

writing collections

ListeningHolistically rated

observational assessmentsHolistically rated

observational assessments

SpeakingHolistically rated

observational assessmentsHolistically rated

observational assessments

The Testing WindowWritingCollect Writing Samples

February 16 – March 3

Verifiers Verify Collections

March 3-6

Raters Rate Writing Collections and other Domains (Reading for K-1,

Listening & Speaking)

March 16-25

Online Reading March 16 – April 2March 16-20 (HS/MS/AC) March 23-27 (Elementary)

Who tests?ALL students K-12 who are identified as LEP including parent denialsparent denials are required to participate in TELPAS

Exceptions:•ARDCs in conjunction with the LPAC may determine that a student receiving Special Education services should not be tested in one or more domain of TELPAS. Participation should be considered on a domain by domain basis.•Students who enroll on or after March 16 will only take the online reading test (contact the Testing Office for confirmation if test is needed)

CTC Responsibilities Read and adhere to all manuals Attend required coordinator training sessions CTC and campus principal are responsible for test

security and confidentiality on the campus Supervise and actively monitor testing Be the campus contact for all questions about testing Report testing irregularities and security violations

immediately to the district testing coordinator Compile required documentation and turn in to testing

office

CTC Responsibilities Obtain CTC access in Texas Training Center Identify and train a writing collection verifier(s) by February 16 Identify and train Raters by February 16 Identify and train Online Test Administrators by March 6 Train appropriate personnel to enter/verify student information

in TAMS (data can be entered by the CTC) Ensure computers have been checked for TestNav capability Ensure that all students are identified and tested Oversee the implementation of validity and reliability

CTC Responsibilities for Training Raters Identify Raters and the type of online training they need

to complete Ensure Raters understand the importance of being

properly trained on the holistic rating process Ensure Raters have obtained access to Texas Training

Center Ensure new and returning Raters know which training to

take and for which grade cluster Monitor that all Raters complete their training

requirements and receive supplemental training support if needed

Rater Responsibilities Attend training on administrative procedures Obtain access to Texas Training Center Complete holistic rating training requirements (online calibration

sets) Print Certificate of Completion and turn in to CTC (page

75 Rater Manual)

Assemble and verify student writing collections Rate writing collection and the listening & speaking domains Complete required documentation Maintain security and confidentiality of student writing collections

and ratings Report test irregularities Sign all oaths

Rater Credentials Must have student on spring roster during the

assessment window (train PE or Fine Arts teacher as a backup rater)

Be knowledgeable about the student’s language ability Be trained in the rating process Complete online rating calibration set with a 70% or

better (3 attempts) Attend mandatory supplemental training if the calibration

set was not passed after the first two attempts

Online Training- Raters Raters need to self register for an account in the Texas

Training Center https://texas.pearson.desire2learn.com Site Code: see CTC

Returning users that have forgotten their login information click on the “Login Help” link

New Users click on the “Self Registration” link Page 57 in Rater Manual

Online Training- Raters Raters will access the online training and calibration

activities through Texas Training Center Raters need to view the training in their grade cluster

• K-1• 2• 3-5• 6-8• 9-12

Raters will complete both the Online Basic Training Course & Calibration Activities OR the Calibration Activities

Online Training- Raters

Note: If a rater has not calibrated in the last three years they are considered “new” and must complete the Basic Online Training Course AND Calibration Activities

Online Training- Raters Access the Basic Training Course for grade cluster New, new to grade cluster or a rater who has not passed the

Calibration Activities in the last 3 years Basic Training Course is open from January 26 - February 7 Click on TELPAS 2015 under the “My Texas Training Center

Program” header then:

Page 62 in Rater Manual

Online Training- Raters Access the Calibration Activities Calibration Activities are open from February 17-21 ALL Raters must complete and pass the Calibration

Activities with at least a 70% Raters have 3 attempts to pass and must attend

Supplemental Support Training before the 3rd attempt

Page 62 in Rater Manual

Teacher collaboration is NOT allowed on

Calibration Activities

Online Training- Raters

As per the TEA Security Supplement:

Test Administrators must independently complete TELPAS calibration activities.

Answer sheets or notes that contain responses to TELPAS calibration activities are not allowed to be shared

Motes created by raters during practice and calibration activities must be destroyed

Supplemental Support Training

• If a Rater has not passed the Calibration Set with a 70% or higher after the second attempt, the Rater is required to attend Supplemental Support Training

• CTCs submit a list of Raters needing Supplemental Support by February 23 February 23 (use form provided by Testing Dept)

• A Rater will not be able to attempt the 3rd Calibration Set until they have attended Supplemental Support Training

• Supplemental Support Training is scheduled for:

February 25, 26, or 27 from 4:00-6:00 February 25, 26, or 27 from 4:00-6:00 at the Johnson Center at the Johnson Center

Assembling the Writing Collection K-12th teachers will collect writing samples during the

window and turn in to the CTC Writing samples are considered to be secure materials Writing Sample Collection window is

February 16-March 3 All Writing Sample Collections must be returned to the

CTC by March 3 Must follow the guidelines for collecting writing samples

Writing Collection Guidelines

At least 5 writing samples (Rater Manual page 19 for examples of writing)

At least 1 narrative about a past event At least 2 academic samples from math, science, social

studies Samples must include the student name and date Cover sheet must be used and stapled to each writing

collection (page 53 of Rater Manual)

Raters will initial the Verification Checklist (page 54 of Rater Manual)

Samples may be typed but spell and grammar check must be disabled

May be a “clean” photocopy from a journal Must only be collected during the window

Writing Collection Guidelines

What NOT to include…

Papers done with heavy use of dictionaries/thesaurusPapers with teacher, peer, or parent correctionsWritten primarily in another language other than EnglishWorksheets or Q&A writing assignmentsBrief, incomplete or rushed papersWritings that have been copied from a textbook, lesson or other sources. Must be in students own words.

Cover Sheet

FRONT BACK

Cover Sheet must be

attached to each Writing

Collection

Rater initials each box while

collecting writing

samples

Verifier initials each box

during verification

window

Verifier Responsibilities Attend training on administrative procedures Obtain access to Texas Training Center Complete online training “Assembling and Verifying

Writing Collections” Print Certificate of Completion and turn in to CTC (page

75 in Rater Manual) Verify student writing collections in accordance with the

checklist Sign Oath Verifier cannot be a TELPAS Rater or the CTC

Online Training- Verifiers Verifiers need self register for an account in the Texas

Training Center https://texas.pearson.desire2learn.com Site Code: telpas2015

Returning users that have forgotten their login information click on the “Login Help” link

New Users click on the “Self Registration” link Page 57 in Rater Manual

Online Training- Verifiers Verifiers need to self register for an account in the Texas

Training Center https://texas.pearson.desire2learn.com Site Code: telpas2015 Click on TELPAS 2015 under the “My Texas

TrainingCenter Program” header then:

Verifying the Writing Collection Verifiers ensure that the writing collections meet all of the

criteria based on the Verification Checklist by initialing each item (checklist is on the back of the Writing Collection Cover Sheet) page 54 of Rater Manual

Verifiers sign the bottom of checklist then return the collection to the CTC

Writing Collections are secure materials Verification Window is March 3 - March 6

Rating Students Raters should review pages 24-30 in Rater Manual prior

to rating students in each domain (writing, listening, speaking)

CTCs will return the Writing Collection to the Raters during the Rating Window March 16 – March 25

Raters must use the district required Rating Roster form to indicate the student’s rating

While rating students, Raters must refer to the PLDs (Proficiency Level Descriptors) pages 33-37 in Rater Manual

Rating Roster must be signed by Rater, Additional Rater (if needed) Teacher Collaborator and Campus Principal

All student ratings and Writing Collections must be returned to the CTC by March 25

Rating Roster

There must be teacher collaboration on ratings as part of the Validity & Reliability process

An Additional Rater is required if the Rater did not pass the Calibration set with a 70% or higher after the 3rd attempt

• TEA requires districts to implement one or more rating support and verification procedures for every campus with TELPAS-rated students

• SAISD has developed a TELPAS validity and reliability plan that meets TEA requirements

• Campus test coordinators will be expected to submit required documents to the district test coordinator by the requested deadlines in order to comply with our district validity and reliability plan

• An internal audit will be conducted by the Testing Department

Validity and Reliability

• For all language domains – grades K-12: Raters will collaborate with other teachers of the students in determining the student’s rating. One collaborator will initial the TELPAS Rating Roster for each child

• For Writing– grades 2-12: 5% of all writing collections will be randomly pulled for assembly verification by the campus test coordinator

SAISD TELPAS Validity and Reliability Procedures

CTCs must create Rating Entry Groups in order to enter ratings for students

CTCs, Rater or Rater Assistants may enter ratings

Enter Ratings

Must be entered in TAMS

DOMAINS GRADES

Listening K-12

Speaking K-12

Reading K-1

Writing K-12

LEVEL RATING

Beginning B

Intermediate I

Advanced A

Advanced High H

If a student will not be rated in a domain, one of the following codes must be used:

Enter Ratings

Code What it Means Information

E Extenuating CircumstanceDocumentation of the reason

must be maintained on campus

X ARD Decision

Decision is made student by student and domain by domain

*decision for a student not to participate in all 4 domains

should be rare

Enter Ratings

Campus Procedures

Input Campus Specific Procedures HereInclude any information that is campus specific for collecting, verifying and rating writing samples

Security Training Modules

Web-based Texas Test Administrator Online Training Modules

www.TexasAssessment.com/TAonlinetraining

All three modules must be completed by certified campus personnel involved if they:- are new to the district- took an incident in the 2013-14- have not completed the Modules in the

last 3 years

The following govern test security

Texas Education Code (TEC) Chapter 39, Subchapter B

Texas Administrative Code (TAC) 19 Subchapter 101, Assessment Family Educational Rights and Privacy Act of 1974 (FERPA) Texas Penal Code 37.10 - Tampering

General Security Information

Penalties for Prohibited Conduct• Placement of restrictions on the issuance, renewal, or

holding of a Texas educator certificate, either indefinitely or for a set term

• Issuance of an inscribed or non-inscribed reprimand• Suspension of a Texas educator certificate for a set term• Revocation or cancellation of a Texas educator certificate

without opportunity for reapplication for a set term or permanently

• Release or disclosure of confidential test content is a Class C misdemeanor and could result in criminal prosecution under TEC ξ39.0303, Section 552.352 of the Texas Government Code, and Section 37.10 of the Texas Penal Code.

Security Oaths • TEA requires that everyone be trained and sign a

security oath before handling secure testing materials or participating in the administration of a state test

• A person who has several roles during TELPAS testing must sign all oaths (Test Administrator, Rater, Verifier, etc)

• Paper test require that the test administrator has signed the bottom box of the oath

Security and Confidentiality

Account for all secure materials before, during, and after each test administration (including Student Authorizations)

Protect the contents of all tests booklets and student answer documents (including writing samples)

No person may view, reveal, or discuss the contents of a test or answer document unless specifically instructed to do so by the procedures in the test administrator manuals and the bottom portion of the oath has been signed

No collaboration allowed during online calibration sets

Secure TELPAS Information• Content of the online reading test is secure and

confidentiality must be maintained• Student Authorization Letters are to be kept

secure All student data in the TELPAS system or entered into the system is confidential

• Passwords and user IDs are secure and should be kept confidential. DO NOT SHARE!

Testing IrregularitiesIncidents resulting in a deviation from documented testing

procedures are defined as testing irregularities

Procedural Examples Serious Examples

Improper accounting for secure materials

Directly or indirectly assisting students with responses to test questions

Eligibility Error Tampering with student responses

Monitoring Error Viewing secure content

IEP Implementation Issue Discussing test content, student responses or student performance

Procedural Error Formally or informally scoring student tests

Duplicating or recording test content

Active Monitoring Monitoring during test administrations is the responsibility of

the test administrator, the campus test coordinator and administrators.

What is NOT active monitoring?Anything that takes the test administrator’s attention away from the students during testing. Examples include:

• Working on the computer or checking email• Reading a book, magazine, or newspaper• Grading papers or working on lesson plans• Leaving the room without a trained substitute test administrator in

the room• Leaving students unattended during meals or breaks

Emphasize importance of training and reading manuals Account for all test materialsEnsure proper storage of materialsVerify accuracy of student information Verify testing requirements for students receiving special education services

Reduce the risk of having an irregularity!

Before the test administration:

Ensure each student receives the correct assessment and testing material(s)Emphasize and verify active monitoringEnsure all testing personnel understand that they may NOT:

provide assistanceview the tests without authorizationdiscuss confidential student informationcheck for strategies

Reduce the risk of having an irregularity!

During test administration:

Reduce the risk of having an irregularity!

After test administrationNo unauthorized viewing (only TEA may permit)No scoring of student responsesShred Student Authorization forms (ASAP)No discussion of confidential student informationNo erasing stray marks or darkening response

ovalsNo copying of students answer documentsAccount for all test materialsNo opening answer documents to view answers

Pitfalls to avoid:Assigning untrained staff to administer and/or

monitor tests (must be certified personnel)Failing to inventory testing materialsHaving insufficient test booklets and/or test

administrator manualsNot accounting for all test booklets and answer

documents each dayTeacher violates time limit Testing beyond the normal school day

Pitfalls to Avoid:Assigning a student the incorrect assessment or

student authorization formFailing to give appropriate accommodation(s) Improper (or nonexistent) monitoringLeaving secure materials unattended (to include

Student Authorizations and Writing Samples)Stopping an on-line session prior to removing

absent students

Reporting & Documentation

If you become aware of any irregularities, contact the DTC immediately with the who, when, where,

what and why.

If the irregularity is deemed an incident the following documentation is required:

Incident report including a plan of action (local form) Signed statement(s) from individual(s) involved (typed, printed

and signed name and dated) Statements should not include names of students The local disciplinary referral form used to report disciplinary

actions taken against students for cheating, cell phone issues or disruptions

Plan of Action

Questions?