office of accountability research, evaluation & assessment school assessment coordinator meeting...

Office of AccountabilityResearch, Evaluation & Assessment

School Assessment Coordinator MeetingFebruary 9, 10 and 11, 2015

The Minnesota Comprehensive Assessments (MCA) and

Minnesota Test of Academic Skills (MTAS)

March 9 – May 15, 2015


Pearson Online Assessment System

Spring 2015 Assessments

Test Administration

Roles and Responsibilities for Testing

Preparing for MCAs

Test Results

Test Security

3

2

5

6

Sections:

4

1


• Testing Windows

• Purpose for Minnesota Assessments

• Reading, Mathematics and Science MCAs

•MTAS Reading, Mathematics and Science

• Title 1 MCA Accountability Windows

• Dates for PearsonAccess Tasks

3

1

Spring 2015 Assessments


• MCA Reading – online 3/9 – 5/8/2015

• MCA Math – online 3/9 – 5/8/2015

• MCA Science – online 3/9 – 5/15/2015

• MTAS Reading – paper pencil 3/9 – 5/8/2015

• MTAS Math – paper pencil 3/9 – 5/8/2015

• MTAS Science – paper pencil 3/9 – 5/8/2015

MCA Modified tests have been discontinued.

4

Testing Windows


5

Purpose for Minnesota Assessments

The Minnesota Assessments program has three purposes:

1. To measure achievement towards meeting the Minnesota Academic Standards and to measure progress towards meeting Minnesota’s standards for English language development

2. To provide information for Minnesota graduates related to career and college readiness

3. To measure the academic progress of students over time

MDE Procedures Manual 2014 – 2015


6

Reading MCA

Grades 3–8 and 10 Reading MCA

• Available only in the online administration mode

• Paper accommodated test materials are available for eligible students

• On-line entry system via Pearson, Data Entry form in Data Entry Session

‘DES’


7

Mathematics MCA

Grades 3–8 Mathematics MCA

• Grades 3–8 Mathematics MCAs are adaptive tests



• On-line entry system via Pearson, Data Entry form in Data Entry Session ‘DES’

• Access to an online calculator when allowed

• Formula sheets are available for students in grades 5–8o REA web site: rea.spps.org>minnesota_comprehensive_assessment

o PearsonAccess: www.pearsonaccess.com>MN>Resources

http://www.pearsonaccess.com/


8

Mathematics MCA

Grades 11 Mathematics MCA

• Grades 11 Mathematics MCA is a fixed-form test




‘DES’

• Access to an online calculator or handheld calculator for the entire test as well as an online formula sheet

o REA web site: rea.spps.org>minnesota_comprehensive_assessment

o PearsonAccess: www.pearsonaccess.com>MN>Resources

http://www.pearsonaccess.com/


9

Science MCA

Grades 5, 8 and High School Science MCA




‘DES’

High School Science MCA

• Page 10 in your 2014 –2015 MCA and MTAS Assessment Manual• Only for students taking Biology


10

MTASMinnesota Test of Academic Skills

MTAS Reading , Math and Science

• Alternate assessment based on alternate achievement standards for students with the most significant

cognitive disabilities • Special Ed. teachers administer, required training via

Special Ed.

• On-line entry system via Pearson, Data Entry form in by Special Ed. Data Entry Session ‘DES’

• SAC receives and inventories secure materials, hand-off to Special Ed.

• SAC receives materials back from Special Ed., secure return to Pearson


11

Title 1 MCA Accountability Windows

Accountability windows are different than testing windows

• AYP Participation windows:Reading and MathematicsApril 14 – April 25

Science MCA and Science MTASMarch 17 – May 9

• AYP Proficiency windows:Students enrolled on October 1 and continues to be enrolled during the accountability window


12

Dates for PearsonAccessTasks

• MCA Reading and Mathematics

˗ Testing Dates: March 9–May 8

˗ Create Test Sessions: TBD

˗ Proctor Cache Tests: TBD

• MCA Science

˗ Testing Dates: March 9–May 15

˗ Create Test Sessions: TBD

˗ Proctor Cache Tests: TBD


• Preparing Schools for Testing

• Available Resources

• Test Administration Times for Reading, Mathematics and Science

• Test Administration with Paper Accommodated Test Materials

• MCA Valid Score Rules

•New to Country EL

13

2

Test Administration


14

Choice of Testing Rooms

• The best location is typically a computer lab. A regular classroom with laptop computers will also work, MUST BE WIRED.

• Computer monitors or laptop screens should be adequately spaced.

• A computer mouse must be provided.

• Headphones (mathematics and science only) must be provided.

• Because of the need for security, only school owned computers can be used for online tests; student-owned computers are not permitted.

• All instructional posters and graphics in the computer lab or where students will be taking the MCAs must be covered or removed.

Freedom from Distractions

• Only people involved in taking or administering the test are allowed in the room.

• The location must be a closed room that is being used only for testing, not a public venue.

• Take appropriate actions to reduce noise such as limiting the use of alarms (not fire alarms), bells and announcements. Talk to your principal.

• The doors of each testing room are to be closed during the test. Post signs on the doors reading “Testing—Do Not Disturb.” Do not allow visitors to enter the room while testing is in progress.

Seating Arrangements

• Separate monitors as much as possible.

• Spread students around the computer lab to the extent possible.

• Consider using commercial privacy screens (or something similar created at the district) or tape construction paper to the sides of monitors (extending out from the of the screen) to serve as blinders.

Preparing Schools for Testing


15

Available Resources on PearsonAccess

• Select the Go to Resources link on any of the landing pages.• Items that are or will be available include:

Student Resourceso Student Tutorials (4min 3sec)o Formula Sheetso Calculators

Manualso Procedures Manualo All assessment manuals

Directionso Student Login Information Instructions and Templateo Test Monitor and Student

Directionso Student Directions

Presentations

User Guides & Technologyo PearsonAccesso TestNavo SystemChecko Proctor Caching

MDE Trainingso Training Module Io Test Security Trainingo MTAS Trainings

Pearson Trainingso Online Test Monitor

Certification Courseo All recorded trainings and

PowerPoints


16

Test Administration Times

Test Administration Times for MCA Reading

• Total Test Administration includes:

1. Time needed to review the Test Monitor and Student Directions

2. Students logging in to the test

3. Answer student questions

4. Student work time

The estimated times do not include time for completion of item samplers or viewing of the online student tutorial.

Estimated Test Administration Times for Reading MCA


17


Test Administration Times for MCA Mathematics







Estimated Test Administration Times for Mathematics MCA


18


Test Administration Times for MCA Science







Estimated Test Administration Times for Science MCA


19

MCA Valid Score Rules

For all grades and all subjects:

• Valid Score (VS)

‒ Respond to 90 percent of the total numbers of items in the assessment to receive a VS code.

• Not Complete (NC)

‒ Respond to six or more items but fewer than 90% receive a NC score code.

• Not Attempted (NA)

‒ Respond to fewer than six items receive a NA score code.

Rules for determining how many responses are required to generate a valid score that can be reported for a student vary with the test.


20

New to Country ELEnglish Learners Who Are New to U.S. Schools (New to Country) A new-to-country English learner is a student who is identified as EL in MARSS and first enrolled in a U.S. school within 12 months of the first day of the Title I accountability window. New-to-country status will be verified using the student’s enrollment date in MARSS as the beginning date and the first day of the accountability window as the end date to calculate less than 12 months. For example:

• A student who entered a U.S. school on April 19, 2014, is considered new to country through April 18, 2015.

• A student who entered a U.S. school on April 14, 2014 would be considered new to country until April 13, 2015; this student could not be indicated as “New-to-Country” for the Title I assessments since the accountability window starts on April 14, 2015.


21

New to Country EL, cont.For Title I assessments, new-to-country students can be removed from AYP proficiency and MMR calculations but they are included in participation calculations.

For reading, new-to-country students do not need to take a Title I reading assessment if they have met one of the following conditions.

• Valid score (VS) or score code for the ACCESS for ELLs or Alternate ACCESS for ELLs Reading test

• Invalid (INV) score for the ACCESS for ELLs or Alternate ACCESS for ELLs Reading test

• Student did not have the opportunity to take the ACCESS for ELLs or Alternate ACCESS for ELLs Reading test because the student enrolled after the end of the testing window

For mathematics and science, new-to-country students must take the Title I assessments. Districts can indicate that these students are new-to-country, and their scores will be removed from the AYP proficiency and MMR calculations for mathematics (there are no proficiency calculations for science). However, these students will be included in the AYP participation calculation for mathematics and science.


• SAC Responsibilities for Paper/Pencil Testing

• SAC Responsibilities for Online Testing

• Schedule MCA Segments

• Selecting Test Monitors and Test Administrators

• Test Monitor Guidelines

•MTAS Reading, Mathematics and Science

22

3

Roles and Responsibilities for Testing


23

SAC Responsibilities for

Paper/Pencil Testing - Accommodations• Manage Paper/Pencil Testing

‒ Inventory test materials immediately upon receipt and returns all test materials to Pearson.

‒ Consolidates and request additional materials needed for testing.

› Request additional materials from [email protected] or [email protected]

‒ Organize and place accommodated test materials in a secure , locked location until they are distributed for testing

‒ Test administrations are performed by trained test proctors

‒ Distributes Non-secure test materials, such as the Test Monitor and Student Directions for MCA Paper Accommodations , to Test Monitors upon receipt to allow them sufficient time to prepare for the test administration

‒ Schedules paper/pencil test sessions

‒ Completes and monitors the Accommodated Materials Security Sign-out Forms

‒ Distributes test materials on each day of the test, including Red Seals

‒ Reminds test proctors that all test materials are to be kept secure and confidential at all times

– Knowing which students have the Math Accommodated Text-to-Speech (ATTS) - Audio CD or Script

– Having head-phones available for Text-to-Speech/Audio CD– Establishes who will enter student responses in the Data Entry form in

TestNav/DES.

mailto:[email protected]



24

SAC Responsibilities for

Online Testing

• Manage online testing– Schedule MCA Test Administrations– Proctor Cache Test Sessions– Create and Start Test Sessions– Monitor Student Progress– Know which students have the Math

Accommodated Text-to-Speech (ATTS)– Have head-phones available for Text-to-Speech


25

Scheduling MCA Segments

• Elements of MCA by Subject− Allows students to exit their test at different points and to

return to complete it later− Students responses are saved in the system− Students are brought back to the item last viewed before exiting

• MCA Reading

‒ Divided into sections by passage and test items‒ Number and placement of passages will vary by form ‒ Each section begins with a title page‒ Students will know they have completed all items for the

passage when they reach the Review Screen for the section‒ Before exiting

› students must complete a section› review items› select Next Section to go to the next section› students must exit the test on the title page for the next

passage

A dialog box will ask students if they want to close the section. Once the section is closed it cannot be reopened.


26

Scheduling MCA Segments, cont.

• Grades 3–8 MCA Mathematics‒ Is divided into four sections‒ Before exiting

› students must complete a section› review items› select Next on the Review screen to go to the next section› students must exit the test on the first item in the next

section

• Grades 11 MCA Mathematics› students must complete a section› review items› select Next on the Review screen to go to the next section› students must exit the test on the first item in the next

section

• MCA Science

‒ Is divided into two sections containing multiple scenarios and each scenario begins with a title page

‒ Before exiting

› students must complete the first section› review items› select Next Section to go the second section› students must exit the test on the title page of the first

scenario on the second section

A dialog box will ask students if they want to close the section. Once the section is closed it cannot be reopened.


27

Selecting Test Monitors andTest AdministratorsTest Monitors and Test Administrators for Title I AssessmentsTest Monitors should be selected from the highest possible ranking from this list. If a school has exhausted the availability of persons in category “1,” it should select staff in category “2” and so on. There should be at least one Test Monitor for every 30 students, if possible.

1. Licensed teachers or administrators who work in the school

2. Licensed teachers or administrators who work in the district

3. Paraprofessionals who work in the school

4. School district personnel employed by the school district

5. Licensed substitute teachers who are employed by the district for the purpose of proctoring the test (Not Recommended)

For the MTAS, Test Administrators may be any district employee who has completed the MTAS Training for Test Administrators for that year. However, the Test Administrator should be someone who is familiar with the student’s response mode and with whom the student feels comfortable.


28

Materials Allowed when taking The MCAs• The following materials are to provided for students for the administration:

• Student login information• Scratch paper (students must write their name on it)• A pen or pencil • Head phones for Text-to-Speech (mathematics and science only)• Hard-copy mathematics formula sheets for grades 5-8 and 11;

located on the Student Resources page of PearsonAccess• Handheld calculators (grade 11 mathematics only)

Materials Not Allowed when taking The MCAs

• Dictionaries, thesauruses and other reference materials are not permitted (see word-to-word dual-language dictionary in Table 12 for the exception for English learners). Calculator manuals are also not permitted.

• Handheld calculators are not allowed for the Science MCA.

• Handheld calculators are not allowed for the grades 3–8 for the Mathematics MCA, unless specified in IEP. Students who require a handheld calculator as an accommodation must use a paper accommodated test book.

• Test Monitors must actively monitor the student(s) so their handheld calculators are not used on non-calculator test items.

• Cell phones and other electronic devices must be turned off and put out of reach before the start of testing and throughout test administration and breaks.

• All instructional posters and graphics in the classroom or on a student’s desk that are related to the subject being tested must be covered or removed during test administration. There is not a list of materials that can remain visible to students during testing. If the materials are related in any way to the subjects being tested, they must be covered or removed. If you have any questions or concerns about materials in the classroom, cover or remove them.

Test Monitor Guidelines


29

What Help Can Test Monitors Give to Students?

• Students must be familiar with how to answer different types of items and Test Monitors cannot provide assistance.

• Repeat the Test Monitor and Student Directions as written. Scripted directions that Test Monitors can use during online test administrations are included to the Test Monitor Directions.

o Test Monitors may not paraphrase Test Monitor and Student Directions, define or pronounce a word within a test item, discuss a test item or give an opinion

regarding a test item. The only directions Test Monitors can give students are those that are scripted in the Test Monitor and Student Directions.

• Students must be familiar with the functionality of calculators or online tools they are using prior to the start of testing; Test Monitors cannot provide assistance with calculators or online tools.

• If a student points out a test item that appears to have an error, the Test Monitor should instruct the student(s) to continue the test and note the grade, subject, form (if applicable), test session ID (online assessment only), student MARSS number (online assessments only), and item number.

o Following the testing session, the Test Monitor should notify the School Assessment Coordinator, who will notify the District Assessment Coordinator.

o The District Assessment Coordinator will contact MDE and give the information collected by the Test Monitor.

o The content of the item itself should never be referenced, copied or emailed within the school or district or in a communication to MDE.

• If a student points out an issue that appears to be a display error or a technical issue, the student should pause the test and the Test Monitor should immediately contact the District or School Assessment Coordinator or the Pearson Help Desk as directed by the district.

Test Monitor Guidelines, cont.


30

Accessibility Panel Tools

Accommodated text-to-speech is available for mathematics andscience as an accommodation only. Please see more information about accommodated

text-to-speech in the next section.

• The Accessibility Panel tools within the test may be adjusted by students to modify the online testing experience for reading, mathematics, and science. See Table 1 on page 21 in your 2014–2015 MCA and MTAS Assessment Manual.

Script

• For the Mathematics MCA, the script accommodation can only be used in conjunction with the paper accommodated test book. The online Mathematics MCA test will not match the script.

• For the Science MCA, the script accommodation can be used in conjunction with the online Science MCA or a large print/braille test book and must be administered in a small group or individual setting.

• When using the science script in conjunction with the online test, the Test Monitor reads from the script while the student sites at the computer and enters responses directly into the online test.



31

Standard Text-to-Speech

• Standard text-to-speech:

‒ is available for all students taking the Mathematics and Science MCA

‒ is student-initiated

‒ reads everything except the charts and graphs

‒ is available in the Accessibility Panel for all students taking Mathematics and

Science MCA

Accommodated Text-to-Speech

• Accommodated Text-to-Speech:

‒ only available for students with an IEP or 504 plan or who are EL-identified in

MARSS

‒ is available for mathematics and science as an accommodation only

‒ all graphics and tables will have text-to-speech

Encourage the EL teacher to look at the script for the item sampler. Practice the item sampler with the script and then do the online item sampler with the text-to-speech to see what is the best fit for the student.

Detailed information about how the Accessibility Panel works is found in Appendix A in the back of your 2014–2015 MCA and MTAS Assessment Manual.



32

Test Monitor Guidelines, cont.on REA web siterea.spps.org>Test Coordination>Minnesota Comprehensive Assessment MCA


33

4

Preparing for MCAs• Item Samplers • Training Center


34

• Grades 3–8 and 11 Math MCA Online Item Samplers

• Grades 3–8 and 10 Reading MCA Online Item Samplers

• Grades 5, 8, HS Online Science Item Samplers

• MTAS Reading, Math, Science Paper Item Samplers

• Paper Item Samplers are available via PDF for students requiring accommodations.

• Access Item Samplers from ANY of the tabs on www.pearsonaccess.com/mn.• Use Item Samplers to familiarize students with the format of the test. • Item samplers are not practice tests and cannot be used to predict how well

students will perform on the assessments. • Two ways to access item samplers:

Select the Go to Item Samplers link on any of the landing pages.o From the Item Samplers page in PearsonAccess without logging in.

See Accessing Online Item Samplers on PearsonAccess on the General tab on the Item Samplers page of PearsonAccess.

o PDFs will also be available for download for any applicable paper accommodations.

Sign in to the Training Center.o After creating sample students and a test session, students can log in

to TestNav to take an online item sampler.o Use this to provide both Test Monitors and students familiarity with

PearsonAccess functionality and the online testing format.

Item Samplers and Training CenterOn PearsonAccess


35

Test Results

• Viewing Test results in PearsonAccess• On-Demand and Longitudinal Reports• Available Reports

5


Viewing Test Results in

Pearson Access

36

• Three components:

• On-Demand Reports – Preliminary results available during the testing window

• Published Reports – PDF versions of final paper reports delivered to districts, available following the Posttest Edit Window

• Longitudinal Reporting System – Results following the Posttest Edit Window loaded to the system for districts to analyze trends and patterns over time


37

On-Demand and Longitudinal Reports

• On-Demand Reports – access to preliminary online rosters and ISRs– Users can drill down to an individual student to see preliminary

student-level results.

• Longitudinal Reports - analyze results from a specific administration, multiple administrations within a year, or assessment results from year to year

– Drill-down, filtering, and sorting capabilities– Aggregate and disaggregate data all the way down to individual

student-level results– District, school, and teacher access to student results


38

Which Reports are Available for Which Tests?

Test Administration On-Demand Reports

Published Reports

Longitudinal Report

MCA Math, Reading, Science a a a

MTAS Math, Reading, Science

a a

OLPA Math and Reading a a

GRAD Math and Reading a a

GRAD Written Composition a


39

When will Results Be Available?

Test AdministrationOn-Demand

ReportsPublished Reports Longitudinal Report

Historical Data - - 10/14/14

OLPA Math and Reading Within 1 hour -Loaded weekly beginning10/24/14

GRAD Math and Reading Within 1 hour4 weeks after test window

-

GRAD Written Composition -12/15/14 5/22/15

-

MCA Math, Reading, Science Within 1 hour 7/29/15 7/29/15

MTAS Math, Reading, Science - 7/29/15 7/29/15


40

Test Security

• Test Monitor Certification • Test Security Policies• Test Security and Training Proctors• Topics to Cover during the SAC Training• New Security Measures for 2014-2015

6


41

Test Monitor Certification

• Test Monitor Certification Training Course

– Required for all Test Monitors and SACs– Required once a year– Learn to create, start and monitor test sessions– PearsonAccess ‘Resources’, Pearson Training’

Online Test Monitor Certification Course (2014-2015)

https://www.pearsonaccess.com/media/MN/Online_Test_Monitor_Certification/launch.html


42

Test Security Policies

• Nondisclosure Agreement

– Everyone with secure access to Pearson must complete the Nondisclosure Agreement.

– Test Monitors & data entry personnel must accept the agreement in PearsonAccess or they will not be able to monitor tests or enter student responses.

– Everyone with access to the test items must sign the Nondisclosure Agreement.

– Anyone who may come in contact with or see test items, can be asked by a student about an item or handle test materials must sign a Nondisclosure Agreement.

– Paper Nondisclosure Agreement available here on REA website: http://rea.spps.org/Test_Coordination2

http://rea.spps.org/Test_Coordination2


43

Test Security & Training Proctors

• School SAC Meeting• You must

˗ hold a training meeting for your monitors,˗ have them sign-in to prove they attended the meeting,˗ allow time for monitors to read the non-disclosure

agreement,˗ have them sign one Non-Disclosure Agreement per school

year collect the Non-Disclosure Agreements and file, and ˗ decide what to do when students finish their test, remain

seated or leave the classroom.

• Topics to Cover during your Training Session• Proctors must be impartial observers during the test

administration.• Reproducing or paraphrasing an item in an email or in writing is

a breach in security.• Cell phones, PDAs or translation devices may not be used.• Dictionaries may not be used.• Scratch paper must be collected from students before leaving the

testing room.• All instructional posters and graphics in the computer lab or

where students will be taking the MCAs must be covered or removed.


44

Test Security & Training Proctors, cont.

• Test Administration Report (TAR)• The TAR is used to record test misadministrations and reasons

for invalidations.• The SAC completes the TAR and forwards a copy to Robin or

Michael in the testing office.• The TAR must be kept on file for 12 months.• The TAR is available here on REA website:


• Examples of misadministrations• A student uses a calculator during a non calculator segment.

• A student engages in inappropriate behavior or action that results in a test being invalidated.

• A student refuses to take an assessment or cheats on an assessment, requiring the assessment to be invalidated.

• A student’s responses are transferred by staff from an answer document to the Data Entry Interface incorrectly.



45

Test Security Policies, cont.

• Paperwork to REA

– Send paperwork through district school mail to the REA Testing Office, 360 Colborne

– Copy of Monitors Training Sign-in Sheet

– Nondisclosure Agreement; one for every person having access to MCAs

– Classroom Security Checklist for Accommodated Test Materials

– Copy of Test Administration Report (TAR)


46

Test Administration Report (TAR)

2014-2015


47

Security Measures for 2014-15

Tip Line

• MDE has launched a Test Security Tip Line.

• Educators, parents, students and otherscan report suspected incidents of cheating, improper or unethical

behavior on any Statewide assessments.

• Links to the Test Security Tip Line are available in two places on the

MDE web site.

˗ View the Test Administration page(MDE>School Support>Test

Administration)

˗ View the Testing Information page(MDE>Just for Parents> Testing

Information)


48

PEARSONhttp://www.pearsonaccess.com/cs/Satellite?pagename=Pearson/QuickLink/PearsonAccessSAC’s site for all assessment tasks

Help DeskPhone: [email protected] – Friday 7:00 am – 6:00 pm (CST)

MDEhttp://education.state.mn.us/mde/index.html

REA Test Coordinator Sitehttp://rea.spps.org/Test_Coordination2

Contact Information

Robin LaneOffice; 651-767-8257Cell: 651-335-3594Email: [email protected]

Michael WirtzOffice: 651-632-3776Cell: 651-261-0045Email: [email protected]

http://www.pearsonaccess.com/cs/Satellite?pagename=Pearson/QuickLink/PearsonAccess

http://www.pearsonaccess.com/cs/Satellite?pagename=Pearson/QuickLink/PearsonAccess



office of accountability research, evaluation & assessment school assessment coordinator meeting...

Documents

mca science

mathematics mca grades

reading mca available

science mca grades

reading mca grades

mca math

assessments slide

assessment o pearsonaccess