developed by the center for applied linguistics access for ells® tier placement, test ordering...

Developed by the Center for Applied Linguistics

ACCESS for ELLs® Tier Placement, Test Ordering & Security, and Score Reports

Jessica Nelson, Center for Applied Linguistics

Harrisburg, Pennsylvania – December 2006


For more information, please contact the WIDA Hotline:1-866-276-7735 or www.wida.us/helpform

World Class Instructional Design and Assessment, www.wida.us

Center for Applied Linguistics, www.cal.org

Metritech, Inc., www.metritech.com

ACCESS for ELLs® Tiers, Ordering, Security, and Score Reports 3

Workshop GoalsWorkshop Goals

To appropriately place students into tiers for the ACCESS for ELLs® test

To be equipped to handle test operations, such as test ordering and test security

To understand and begin to interpret ACCESS score reports


Workshop Objectives (1)Workshop Objectives (1)

To review roles of staff in administering the ACCESS for ELLs® test

To learn the guidance for placing students in tiers (A, B, C) of the ACCESS for ELLs® test

To understand how to order test booklets

To understand what to do when a test shipment arrives How to track test materials at the district and school How to order additional materials

To understand test security procedures How to draft district and school security checklists


Workshop Objectives (2)Workshop Objectives (2)

To understand and interpret ACCESS for ELLs® score reports Domain, Raw, Scaled, Oral Language, Literacy,

Comprehension, and Overall (Composite) Scores Parent/Guardian Report Teacher Report Student Roster Report School Frequency Report District Frequency Report

To explore the WIDA Can Do Descriptors

To discuss programmatic implications of ACCESS results


Roles & Responsibilities


Roles and Responsibilities (1)Roles and Responsibilities (1)

District Assessment Coordinator Serves as main contact with MetriTech, Inc. Takes Inventory of materials immediately upon arrival and

places them in locked storage. Prepares list of grades to be tested in each school, list of testing

materials required by each school, and testing schedule of each school

Coordinates ordering and distribution of test materials to the schools and returning of test materials to MetriTech

Takes inventory of materials distributed and returned Responsible for training School Assessment Coordinators &

Test Administrators on test administration and security Anticipates and answers questions from school assessment

coordinators at your schools



Test Coordinator (at school-level) Takes inventory of boxes immediately upon their arrival from the

District Assessment Coordinator Verifies that there are enough testing materials Requests additional needed testing materials from MetriTech if

necessary Coordinates and distributes test materials in your school; Takes inventory of materials that are returned to District Test

Coordinators Reminds Test Administrators that all test materials are to be

kept secure and confidential



Test Administrators Complete online training course and certification through

Desire2Learn system (www.uwosh.edu/d2l) Become familiar with procedures in Test Administration Manual

for test accommodations for ELLs with disabilities Fill out the demographic student information on each test

booklet (if pre-ID labels were not ordered) Administer the components of ACCESS for ELLs® for which you

are certified (Kindergarten/Group Components/Speaking)


Tier Placement


Importance of Tier PlacementImportance of Tier Placement

Appropriate tier placement maximizes the accuracy and validity of the ACCESS for ELLS® test results.


Grade Leveland Tier

K

1-2

3-5

6-8

9-12

A (adaptive – no tiers)

A B C

A B C

A B C

A B C

100 (roll-out Spring 2005)



Listening — group admin, machine scored

Reading — group admin, machine scored

Speaking — individual admin, adaptive, TA scored

Writing — group admin, rater scored

Domains

Series

Structure of ACCESS for ELLs® Test Structure of ACCESS for ELLs® Test

1 Kindergarten form + (4 grade level clustersx 3 tiers) = 13 test forms

Each letter represents one tiered test form


Tier Alignment with Proficiency LevelsTier Alignment with Proficiency Levels

ENTERING BEGINNING DEVELOPINGEXPANDING BRIDGING

1 2 3 4 5

Annual ACCESS for ELLs®ACCESS for ELLs®

Tier A

Tier B

Tier C


Tier Placement GuidelinesTier Placement Guidelines

Use previous test scores, if available

Use teacher judgment, if available

Keep in mind that 70-80% of students will take Tier B

Use reading/writing levels over oral skills

When in doubt, place the student in the higher tier

At intake use W-APT™ or equivalent English language proficiency test


Tier Placement “Rule of Thumb”Tier Placement “Rule of Thumb”

Assign an ELL the Tier B test unless you have compelling evidence that items in this tier, given their intended proficiency level, would be way too hard or way too easy for the student.

TIER B is appropriate for most ELLs - those who:have social language proficiency and some, but not

extensive, academic language proficiency in English OR

have acquired some literacy in English, though have not yet reached grade level literacy.


Profile 1: FatimaProfile 1: Fatima

Fatima shows developing language skills in most classroom subjects. Despite her academic interests, she is not yet approaching grade-level literacy in the core content areas.

Which tier is most appropriate for Fatima? Tier B


Profile 2: MohammedProfile 2: Mohammed

Mohammed is in his first year of instruction in English. He is comfortable with basic conversations outside the classroom, but struggles with even low-level reading tasks.

Which tier is most appropriate for Mohammed? Tier A


Profile 3: EstherProfile 3: Esther

Esther is approaching grade level literacy in the core academic content areas. Her teacher feels she will likely meet the state’s exit criteria for ELL support services by the end of the academic year.

Which tier is most appropriate for Esther? Tier C


Profile 4: LilyProfile 4: Lily

Lily shows developing proficiency in academic English, but does not yet reach grade level. She seems comfortable interacting with her monolingual English-speaking peers. Lily’s teacher characterizes her as a beginner.

Which tier is most appropriate for Lily?

A/B line Tier B


Profile 5: ByungProfile 5: Byung

Byung’s intake tests reveal expanding oral skills in English, as well as grade-level literacy in his native language. His English reading and writing skills are lower. A portfolio of his work provides evidence that his literacy skills are not yet on grade level.

Which tier is most appropriate for Byung?

B/C line Tier B


Ordering Test Materials


Pennsylvania’s Testing WindowPennsylvania’s Testing Window

TaskStart Date End Date

Duration (Days)

Test Ordering 01/08/07 02/05/07 29

Pre-ID Label Ordering 01/08/07 02/05/07 29

State Receives Test Materials 03/19/07 03/19/07 1

Test Window 04/02/07 04/27/07 26

Additional Materials Deadline 04/16/07

Districts Ship Completed Material to MT 05/03/07 05/03/07 1

Reports Shipped to State 07/02/07 07/02/07 1


Ordering Test BookletsOrdering Test Booklets

Test booklets are ordered from MetriTech online at www.metritech.com/wida.

You will be asked to log on. Use the ID wida and the password access.

Carry out the following step-wise procedure: Select your state Select your district Verify the contact information for the district facilitator who will

receive the booklets Enter the number of test booklets by grade level and tier for

each of the schools within the district Submit the order


Booklet Ordering Screens (1)Booklet Ordering Screens (1)

Select your state

Select your district

Verify your contact information and shipping address



For each school, indicate the number of booklets for each grade level cluster and tier.



When your order is completely specified, click on the ORDER button, verify your order, and then submit it.

When you verify your order, click on the SUBMIT ORDER button to send your order to MetriTech.


Pre-ID Labels for Test BookletsPre-ID Labels for Test Booklets

Pre-printed ID labels for test booklets are available from Metritech. Call 1-800-747-4868 or email [email protected] to get ordering procedures.

Only designated district facilitators can order labels.

ID labels are not automatically included with a booklet order. Prepare a text or Excel file with pertinent demographic

information for each student. Format the file appropriately so it can be computer read.

(MetriTech will supply instructions.)

If you do not order pre-id labels, the test administrator must bubble-in the front and back covers of each test booklet with relevant student data.


Maintaining Test Security


Materials Received by the DistrictMaterials Received by the District

District Packing List

Each school’s Packing List

Your state’s schedule

Test Administration Manuals (1 per set of 20 test booklets)

Test Administration Scripts and Speaking Test Picture Cue Booklet/Script (1 per set of 8 booklets ordered per grade-level cluster)

10% overage of Listening, Reading and Writing test booklets

Pre-ID labels and return instructions


District Procedures Upon Receipt of Test MaterialsDistrict Procedures Upon Receipt of Test Materials

Locate and verify district and school packing lists (in Box 1).

Divide materials by school.

Note discrepancies between the Packing List and material received; record on the Documentation of Materials Not Returned form; fax this form to MetriTech (217-398-5798).

Deliver the test assessment materials to the School Assessment Coordinator.

Create sign-in and sign-out forms for test materials (see sample on next slide).

Order additional materials if necessary using the Additional Materials form; fax this form to MetriTech (217-398-5798).


Sample District Security Checklist (Part 1)Sample District Security Checklist (Part 1)

DIRECTIONS: The District Test Coordinator must be sure that the School Test Coordinator has signed the Agreement to Maintain Confidentiality before issuing secure test materials. The School Test Coordinator must sign the District Security Checklist form when secure test materials are issued. The School Test Coordinator’s signature confirms that he/she has received the materials listed below and that he/she will require test administrators to sign the school security checklist and the Agreement to Maintain Confidentiality before issuing secure test materials.

School Test Coordinator Signature: ___________________

Date: ___________________________________________

NOTE: The District Test Coordinator should keep a copy of all signed forms, including the agreements to maintain security, the District Security checklist, and the School Security checklist. The School Test Coordinator should keep one copy of all completed forms and return the original to the District Test Coordinator with the secure test materials.


General Security GuidelinesGeneral Security Guidelines

Only test facilitators, coordinators, and test administrators (or other authorized staff) may handle secure test materials.

Place all secure materials in locked storage.

Do not leave materials unattended before or after testing.

Do not share any specific test information with students prior to or after testing.

Do not copy any test booklets or other secure materials.


Pennsylvania System of School Assessment Test Security Procedure (1/5)Pennsylvania System of School Assessment Test Security Procedure (1/5)

Test Booklet Security

District assessment coordinators, administrators, and teachers are responsible for the security of the assessment materials from the time the district receives them to the time they are packed and shipped back to the contractor for scoring. When not being used, the assessment materials must be stored in a secure location with access given to authorized personnel only. The following is a list of cautions to remember:

[Continued on next slide]


1. Assessment materials must not be in any way reproduced, in whole or in part (including paraphrasing).

2. No one may have test booklets without authorization from the district assessment coordinator.

3. Teachers or other staff members may not show any items in the test booklets to anyone not administering the assessment.

4. Teachers may not use any specific items in the secure booklets to help prepare the students for the assessment.

5. Test administrators may not return any test booklet to any student after it has been turned in except in the following cases: Make up sessions for absences Students going to another testing site for extended time

(Note: If, after the student returns the test booklet, it is noticed that not all of the test items were answered, the booklet can not be returned to the student to complete.)



Test Administration Security

Building administrators and/or district assessment coordinators are to review the testing procedures with the teachers and other staff members who will be administering the assessment. During the administration of the assessment, the following are allowable:

1. Teachers and other staff members who are administering the test may give words of encouragement and general instructions to the students.

2. Teachers and other staff may answer questions concerning ONLY the directions for each test session.

[Continued on next slide]



3. Teachers and other staff members may give accommodations or modifications that are consistent with the student’s IEP, 504, or LEP Plan and that are used in the normal delivery of instruction with the exception of those indicated as not allowable in the state Accommodations Guidelines.

4. Breaks may be given at the discretion of the district as long as the length of the break does not affect the integrity of the assessment.

5. Teachers and other staff members may point out to a student items that were left blank (i.e., accidental skipping of an item response grid or response space).



During the administering of the assessment, the rules listed below must be adhered to:1. During the testing period, teachers and other staff

may not assist any student with any item in any way that would aid the student in answering the question.

2. Teachers and other staff may not encourage any student to edit their responses in any way.

3. Do not leave any student alone while taking the assessment.

4. Do not allow any student to take a test booklet and/or answer document out of the testing area without proper supervision.

Source: http://www.pde.state.pa.us/a_and_t/cwp/view.asp?a=108&q=95357&a_and_tNav



Sample District Security Checklist (Part 2)Sample District Security Checklist (Part 2)

Grade Materials # of copies

Kindergarten Test Booklets

Administrator’s scripts

1-2 Tier A Test Booklets


1-2 Tier B Test Booklets


1-2 Tier C Test Booklets


1 -2 Speaking Tests


Materials for Each SchoolMaterials for Each School

School Packing List

Grade/Tier Header Sheets

Documentation of Materials Not Returned form

District and School Test Administration Manuals (1 per set of 20 test booklets)

Speaking Tests for each grade-level cluster; Listening, Reading and Writing test booklets and scripts for each grade cluster & tier tested


School Procedures Upon Receipt of Test MaterialsSchool Procedures Upon Receipt of Test Materials

Divide test booklets into groups for each scheduled testing session. Test booklets may not be distributed prior to testing session Students must use Number 2 pencils; the scanning equipment

used to score answer documents will not read anything but Number 2 pencil marks.

Place pre-ID labels containing the student demographic information in the box on the front cover of the test booklet.

If pre-ID labels were not ordered or if any label contains incorrect information, the student demographic information must be filled in by hand; school test coordinator will give District Code number and School Code number to test administrators.


Sample School Security Checklist (Part 1) Sample School Security Checklist (Part 1)

School Security Checklist

ACCESS for ELLs® Check List

Spring 2007

District:________________ School:___________________

DIRECTIONS: The School Test Coordinator must make sure that the test administrator has signed the Agreement to Maintain Confidentiality before issuing secure test materials. The test administrator must date and sign this form when secure test materials are issued. The School Test Coordinator must sign this form when secure test materials are returned.

NOTE: The School Test Coordinator should keep one copy of all completed forms and return the original to the District Test Coordinator with the secure test materials.


School Security Checklist (Part 2)School Security Checklist (Part 2)

Grade Materials # of copies

Receiving Test Administrator’s Signature

Date & Time Received

School Test Coordinator’s Signature

Date and Time Returned

Kindergarten Test Booklets

Administrator scripts

1-2 Tier A Test Booklets


1-2 Tier B Test Booklets



Score Reports


ACCESS for ELLs® Interpretive GuideACCESS for ELLs® Interpretive Guide

The ACCESS for ELLs® Interpretive Guide for Score Reports (M. Gottlieb, April 2006) contains detailed information on the use of scores from this assessment.

Recommendation: Download the full document (34 pages) from www.wida.us.


ACCESS Component Test Weights as Percentage of Overall Composite ScoreACCESS Component Test Weights as Percentage of Overall Composite Score

Test Weights(Percent)Listening,

15%

Reading, 35%

Writing, 35%

Speaking, 15%

Listening

Reading

Writing

Speaking


Scores Received: Student LevelScores Received: Student Level

Each student receives a scale score and a proficiency level for: Listening Speaking Reading Writing

Scale scores are out of a possible 100 – 600 Scale scores are converted to proficiency scores

using the cuts established during standard setting Proficiency levels range from 1.0 – 6.0


Scores Received: Student Level (cont’d.)Scores Received: Student Level (cont’d.)

Each student also receives a scale score and a proficiency level for Oral Language, Literacy, Comprehension, and Overall Composite.

Contribution of Language Domains to Composite Scores

Contribution of Language Domains (By Percent) Type of Composite Score

Listening Speaking Reading Writing

Oral Language 50% 50% -- -- Literacy -- -- 50% 50% Comprehension 30% -- 70% -- Overall 15% 15% 35% 35%


ACCESS for ELLs® Score ReportsACCESS for ELLs® Score Reports

There are 5 ACCESS Score Reports:

Teacher Report Parent/Guardian Report Student Roster Report School Frequency Report District Frequency Report


What does the Teacher Report tell us?What does the Teacher Report tell us?

Score Report

Audience or Stakeholder

Types of Information

Teacher

Teachers Administrators

Individual student’s scale scores and proficiency levels for each language domain, Oral Language, Literacy, Comprehension, and Overall Score; raw scores for Comprehension Tasks, Speaking, and Writing Tasks by English language proficiency standard

The Teacher Report contains individual data for one student.


TeacherReportTeacherReport

Report for one individual student


Teacher Report (cont’d.)


Teacher Report (cont’d.)

Raw Scores by Standard


What does the Parent Report tell us?What does the Parent Report tell us?

Score Report

Audience or Stakeholder

Types of Information

Parent/Guardian Students Parents/ Guardians Teachers School Teams

Individual student’s Overall Score and levels of English language proficiency for language domains (Listening, Speaking, Reading, and Writing) and Comprehension

The Parent Report, like the Teacher Report, contains individual student data.


Interpretive Guide and Parent/Guardian Report Translations available on the WIDA website (www.wida.us)

Special thanks to Milwaukee and Wausau Public Schools for several of the translations!

Parent Report


Parent Report (cont’d.)

The Parent Report is currently available in:

Amharic

Bosnian-Croatian

Creole

French

Gujarati

Hmong

Korean

Lao

Polish

Portuguese

Russian

Serbian (Cyrillic)

Somali

Spanish

Swahili

Traditional Chinese

Urdu

More translations coming soon!


Parent Report (cont’d.)

May 2006

Dear Parent or Guardian,

This past winter, ELL students in grades kindergarten through twelfth grade participated in the administration of the ACCESS for ELLs® language proficiency test. ACCESS now provides a standardized measurement of academic language proficiency for English Language learners (ELL) students throughout the state of Wisconsin. With this information, we will also be able to monitor individual ELL student progress on an annual basis.

Enclosed you will find your child’s results on ACCESS. The Parent/Guardian Reports provides information about your child’s English Language Proficiency Level. This information is for you to review and keep.

If you have any questions regarding these tests or the information that is being sent to you about how your child did on these tests, please contact your child’s ELL teacher, building principal, or me.

Sincerely,

__________________________________(School ELL coordinator, principal, or teacher)

Parent letter template in English, Spanish, and Hmong that the school can customize - available at www.wida.us


What does the Student Roster Report tell us?

The Student Roster Report lists the scale scores and proficiency levels for a group (or class) of students.


Student Roster Report


What does the School Frequency Report tell us?

The School Frequency Report lists the numbers of students tested in each domain of ACCESS by grade level within a school.


School Frequency Report


What does the District Frequency Report tell us?

The District Frequency Report lists the numbers of students tested in each domain of ACCESS by grade level within a district.


District Frequency Report


Kindergarten Scores

The maximum Overall (Composite) English language proficiency level that a student taking the Kindergarten form of ACCESS for ELLs® can receive is 3.4.

Language Domain Scale Score

(Possible 100 - 600)*

English Language Proficiency Level

(Possible 1.0 - 6.0)* Listening 308 4.0 Speaking 400 6.0 Reading 284 3.0 Writing 255 2.4 Oral Language (Listening & Speaking) 354 5.4 Literacy (Reading & Writing) 270 2.7 Comprehension (Listening and Reading)

291 3.5

Overall Score (Composite—all language domains) 295 3.4


6-

Reaching

specialized or technical language reflective of the content area at grade level a variety of sentence lengths of varying linguistic complexity in extended oral or written

discourse as required at the specified grade level oral and written communication of English comparable to that of English proficient peers

5-

Bridging

the technical language of the content areas; a variety of sentence lengths of varying linguistic complexity in extended oral or written

discourse, including stories, essays, or reports; oral or written language approaching comparability to that of English proficient peers

when presented with grade level material

4-

Expanding

specific and some technical language of the content areas; a variety of sentence lengths of varying linguistic complexity in oral discourse or multiple,

related paragraphs; oral or written language with minimal phonological, syntactic, or semantic errors that do

not impede the overall meaning of the communication when presented with oral or written connected discourse with occasional visual and graphic support

3-

Developing

general and some specific language of the content areas; expanded sentences in oral interaction or written paragraphs; oral or written language with phonological, syntactic, or semantic errors that may impede

the communication but retain much of its meaning when presented with oral or written, narrative or expository descriptions with occasional visual and graphic support

2-

Beginning

general language related to the content areas; phrases or short sentences; oral or written language with phonological, syntactic, or semantic errors that often impede

the meaning of the communication when presented with one to multiple-step commands, directions, questions, or a series of statements with visual and graphic support

1-

Entering

pictorial or graphic representation of the language of the content areas; words, phrases, or chunks of language when presented with one-step commands/directions,

WH-questions, or statements with visual and graphic support

Performance Definitions

At this level, English language learners process, understand, produce or use:


Can Do Descriptors

Language Domain

Level 1- Entering

Level 2- Beginning

Level 3- Developing

Level 4- Expanding

Level 5- Bridging

Listening

Point to stated pictures, words, phrases

Follow one-step oral directions

Match oral statements to objects, figures, or illustrations

Sort pictures, objects according to oral instructions

Follow two-step oral directions

Match information from oral descriptions to objects, illustrations

Locate, select, order information from oral descriptions

Follow multi-step oral directions

Categorize or sequence oral information using pictures, objects

Compare and contrast functions, relationships from oral information

Analyze and apply oral information

Identify cause and effect from oral discourse

Draw conclusions from oral information

Construct models based on oral discourse

Make connections from oral discourse

Level 6

- Reaching

Listening

For the given level of English language proficiency level, English language learners can:


Can Do Descriptors

Speaking


Language Domain

Level 1- Entering

Level 2- Beginning

Level 3- Developing

Level 4- Expanding

Level 5- Bridging

Speaking

Name objects, people, pictures

Answer wh- questions

Ask wh- questions

Describe pictures, events, objects, people

Restate facts

Formulate hypotheses, make predictions

Describe processes, procedures

Re/ tell stories or events

Discuss stories, issues, concepts

Give speeches, oral reports

Offer creative solutions to issues, problems

Engage in debates

Explain phenomena, give examples, and justify responses

Express and defend points of view

Level 6

- Reaching


Can Do Descriptors

Reading



Can Do Descriptors

Writing


Language Domain

Level 1- Entering

Level 2- Beginning

Level 3- Developing

Level 4- Expanding

Level 5- Bridging

Writing

Label objects, pictures, diagrams

Draw in response to oral directions

Produce icons, symbols, words, phrases to convey messages

Make lists Produce

drawings, phrases, short sentences, notes

Give information requested from oral or written directions

Produce bare-bones expository or narrative texts

Compare/ contrast information

Describe events, people, processes, procedures

Summarize information from graphics or notes

Edit and revise writing

Create original ideas or detailed responses

Apply information to new contexts

React to multiple genres and discourses

Author multiple forms of writing

Level 6

- Reaching


How do we use this information?How do we use this information?

Standards-based results help inform curriculum, instruction and assessment of ELLs

The Overall Composite Score summarizes student’s global language proficiency

Domain subscale scores allow for examination of strengths and weakness by domain

Raw scores by standards allow for examination of strengths and weakness by content area language

Individual report components offer a starting point for differentiating instruction and assessment

Writing and Speaking Rubrics in Interpretative Guide - criteria within rubrics scaffold across the levels of language proficiency and may be used in assessing classroom tasks and projects throughout the year


Programmatic Implications (1)Programmatic Implications (1)

If it’s appropriate to exit the student from ELL services? Does this student have the language skills necessary to access the content in the mainstream classroom without additional language support services? What additional evidence is needed to make a determination?

If the student’s English proficiency is weak in a particular language domain (e.g., Writing)?

If the student’s English proficiency is weak in a particular standard area (e.g., the language of Social Studies)? If so, consider additional content language support.

High scores (Levels 5-6) may indicate a need for Monitoring or Targeted Support. School teams should consider:


A balanced, long-term approach that focuses on grade-level academic standards and English proficiency standards, and utilizes strategies that increase comprehension and communication in English (e.g., sheltered instruction)

Enhancement of both oral language and literacy development

Providing L1 instruction (bilingual education) and/or support where feasible

Programmatic Implications (2) Programmatic Implications (2)

Mid-level scores (Levels 3-4) may indicate a need for 1-3 more years of ELL support services. School teams should consider:


Providing targeted communicative / social & instructional English briefly

Enrolling student in “newcomer” program if available and appropriate

Using content-based strategies (e.g., sheltered instruction) and L1 instruction, if possible

Scaffolding within programs and schoolGraphic supportPeer supportSupplemental and modified materials

Programmatic Implications (3) Programmatic Implications (3)

Beginner level scores (Levels 1-2) may need 5 or 6 more years of ELL support services. School teams should consider:


For more information, please contact the WIDA Hotline:1-866-276-7735 or www.wida.us/helpform

World Class Instructional Design and Assessment, www.wida.us

Center for Applied Linguistics, www.cal.org

Metritech, Inc., www.metritech.com

Questions

or

Comments?

developed by the center for applied linguistics access for ells® tier placement, test ordering...

Documents