Kansas Large District MAP Members Mtg. - Oct. 17, 2013

Intro & WelcomeUpdates Technology, Curriculum Partners, Product Updates - Skills Pointer, DesCartes, Science, Field Testing, Knowledge AcademyMinnesota Updates: MCA Alignment, New Norms 2015Special Feature: MAP & STAR: Whats the Difference?LUNCHAfternoon Breakout sessions

Featured Guest: Matt Hicks, NWEA Partner Advisory Roundtable Discussion

Minnesota MAP Members Mtg. Oct. 22 & 24, 2013

1Afternoon Sessions

12:45 to 1:45S1 Using RIT Norm Calculators: Setting Goals at All Levels Dan HendersonS1Bring Your Data Bring Your Questions Eric Merchant & Lynn Lamers1:50 to 2:50S2College and Career Readiness: Tracking Growth toward Post H.S. Success Les PerryS2MPG/CPAA Assessing the Learning Needs of Primary Age Students Lynn Lamers & Eric Merchant

Minnesota NWEA TeamLes Perry Account Executive for Kansas, Missouri, MinnesotaMinnesota based (Detroit Lakes) les.perry@nwea.org Lynn Lamers Account Manager for Minnesota & Wisconsin lynn.lamers@nwea.org (Minneapolis)Eric Merchant Account Manager for Minnesota & Wisconsin eric.merchant@nwea.org (Modesto, Ca)Linda Andres - Account Manager for Minnesota & Wisconsin linda.andres@nwea.org (Portland, Or)Dan Henderson Central Region Accounts Manager (Iowa, Wisconsin, Minnesota, Illinois, Missouri, Kansas, Nebraska)(Washington, Ia.) dan.henderson@nwea.org General Number: 503.624.1951 Tech Support: 877-469-3287

Special Guest: Mr. Matthew HicksDirector of Product StrategyForming a Partner Advisory Panel

Matt previously served as the Senior Product Manager for NWEAs MAP assessment product line. In his current role Matt is responsible for leading the strategy for NWEAs growing product portfolio, ensuring that each of NWEAs products is aligned with NWEAs mission of helping all kids learn. Before joining NWEA, Matt worked in high-tech, market research, secondary education and eCommerce where he successfully developed and launched over a dozen products. He developed his deep interest in education from his wife, a first grade teacher of 12 years. Matt graduated from Sheffield Hallam University in England and now lives in Portland Oregon with his wife and two dogs.Special Guest: Mr. Jeff Strickler,Executive Vice President and Chief Operating OfficerJeff joined Northwest Evaluation Association (NWEA) in 2007. In addition to his role as Chief Operating Officer, Jeff also serves as NWEA's CFO and Vice President of Corporate Services. Jeff brings to NWEA more than 20 years of experience helping organizations attain their goals, first as an attorney at Perkins Coie and later in various leadership roles at notable Northwest companies, including CFI ProServices/Harland Financial System, Vesta Corporation, Centrisoft Corporation and CakeBoxx LLC.At CFI ProServices, Jeff was Vice President and General Counsel from 1994 through its acquisition by Harland Financial Services Systems in 2000. He was the Vice President of Finance and Administration at Vesta Corporation during a critical time in that company's expansion. Jeff has served as the Chief Operating Officer at both Centrisoft Corporation and CakeBoxx LLC, a startup company engaged in developing a new, more secure design for international shipping containers.Jeff received a B.S. in Business Administration from Oregon State University in 1982 and J.D. degree from the University of California at Berkeley (Boalt Hall) in 1985.

Technology: The Good, The Bad & The Ugly (and the beautiful) CLIENT-SERVER MAP - GoodSystem in use since early 2000s

Archive your previous term data located in the NTE folder.

Submit your Class Roster File (CRF) at least two weeks in advance of testing. If applicable, also submit your Special Programs File (SPF)

Client-Server performance has been good this year tech calls are mostly of the how-to variety

Process for transitioning to Web MAP contact Julia (currently a waiting list)

Susan6INTERNET ACCESS

Technology HappensMobile Devices? Performing Well - Good

iPad Implemented this Fall (Not MPG)Chromebooks? YESSusan8Web Based MAP UpdateUgly Start This YearMany of you experienced interrupted or suspended testing because of multiple technical problems in MAP.hardware failures in network equipmentresiliency failuresdegraded performance due to anomalous traffic generated by the changing patterns of testing

Long wait time on tech support line

Heavy concurrency load at certain times of the day 8:30 to 10:00 am

NOT the optimum partner experience that we want to provide

Hard to determine issues locally when we are having issues

Resulted in delayed testing; drawn out window; stops and starts; less than stellar resultsSusan9What We Have Done What We Will DoWHAT STEPS HAVE WE TAKEN?We replaced hardware and audited our network configuration; we then asked Cisco to confirm the configuration and health of our system

We strengthened our solutions to automatically switch to redundant systems in case of failure; this work was confirmed with third party experts.

We expanded our hardware and systems infrastructure to increase the overall processing power available to MAP assessment

We temporarily delayed end-of-test reporting until end of day to optimize students testing experience. Students will still immediately receive their reports at the end of the testing session.

As we made these changes we asked many of you to suspend or delay testing for one or more days. This allowed us to ensure the changes we made fully solved the problemsNEW! Web MAP Update Website Better Communication Processhttp://www.nwea.org/node/18860/ Latest Tech Updates (no calling required)Hour by Hour Updates on the Technical System at NWEAAlerts will be posted in real timeTechnical reminders for local configurationNotices of future maintenance windowsQuestions from Partners ?Are Results Valid when students had to stop and restart? (see letter from research team)

Research team analyzed about 100,000 test results of students that were interrupted.

From our findings, we are confident the scores are valid and that the MAP test behaved as designed in spite of unintentional interruptions

CRITERIA TO DETERMINE SCORE VALIDITY

Time on test less than 30 minutes are suspectStandard Error of Measure (St Err) more than 4 points is suspectPercent correct for the test:Normal = 45% to 55% correctStudents getting less than 45% to 40% correct will be suspectThese data are all on the Comprehensive Data File aka, the Data Export SchedulerPartner QuestionsIf we had to delay our test window to later in the fall, will the norms still apply?(see letter from Mike Nesterak Director of NWEA Research Services)

It is recommended that districts not delay beyond the eighth week of instruction for their fall MAP test administration. Testing within these first eight weeks should have no material impact on the fall-to-spring RIT growth.Partner QuestionsSince the Web MAP system had problems this year, will we receive any compensation?

Yes, we will work with districts one on one to determine a fair compensation.Check List for Technology on District SideCheck the Whitelist: If your district/school is experiencing intermittent slowness, white screens, or the inability to click on various buttons when testing, please ensure that you have whitelisted *.mapnwea.org on equipment that sits between the testing machine and NWEA servers.Double Check That You Are Using the LATEST Version of the Lock Down Browser - Latest version was released on Aug. 28. If you did not download that browser, Web MAP will not be compatibleCheck Internet Usage During Testing Hours - your district may have plenty of bandwidth, but if people are using the internet during testing hours, the bandwidth requirements may not meet the need for testing.Check the Tech Requirements for Using Compatible Browsers Chrome, Safari (NO INTERNET EXPLORER)Use the Proctor Console Issues Quick Guide - If students are stuck dont panicuse the guide and a few key strokes.BIGGEST CHANGE..No New Migrations to the Web Based System This School Year will open again for Fall 2014Product Updates: Skills Pointer for Progress Monitoring & DesCartes (Fall 2014)What Will Be New with DesCartes?Interactive learning continuumStudent-Teacher Friendly languageAbility to create Data-Ladders for Differentiated Instruction

What are our Partners Saying?Our teachers give up on it pretty quick.

Its become a hurdle. Its too intense and stymies the teachers and detracts from the value of MAP.

intimidating and takes too much time.

The statements dont appear to be directly based on the CCSS.The vision for a future DesCartes addresses the specific limitations of the current offering, including the following:Teacher and student friendly content that clearly describes a learnable skill or understandingImproved organizational structure providing options of sorting Learning Statements by Topic and Standard in addition to Sub-GoalInteractive and customizable reporting capabilities & features support instructional planningAbility to link to external resources including lesson plans, interactives, and student learning resources

DesCartes Vision

Advanced options for report display, filtering & contentImproved content for instructional useLinking to additional resourcesAbility to view statements by standards language21

Automatically generated class-based ladders for instructional groupingsView Learning Statements by State Standards22NEW! Skills Pointer diagnostic tool (gr. 3-9)What Will Be New in Skills Pointer?User-friendly design & interfaceEasy-to-use reportsRich digital learning contentHigh-quality test items & larger item bankRevised CCSS skills frameworkIntegration with the MAP platform

Improved UI

Two years ago NWEA purchased a skills mastery assessment from a small assessment company.Over the last few years weve made the assessment available to a handful of partners with mixed results. This experience has helped inform us with what is needed from our partners for an assessment such as Skills Pointer.

Before I jump into how this feedback is informing our plans. Lets take a look at what skills pointer is today.24Skills Pointer TodaySkills mastery assessment for: Grades 3-9: Math, Reading, Language Usage and Sciencealigned to the Common Core and state standardsIntended for frequent use, primarily for remediation

Provides access to printable learning plans based on individual skills

Standalone system, independent of MAP What will the new Skills Pointer provide?Ability to leverage MAP as an RTI Screener to inform remediation plans for students with Skills Pointer

CAT based Skills Locator Assessment determinesa student's actual skill level, independent of grade to find the exact learning objectives/skills each student needs to learn

Track frequent progress to plan with Skills Pointers short skills mastery tests

Accesslearning and instructional resources based on each students plan, saving teachers time and giving every student resources to learn

Developed from the ground-up based on the Common Core State StandardsBased on the feedback of our partners weve invested in an effort to completely re-build Skills Pointer from the ground up.

This new version will leverage NWEAs research to provide evidence based skills assessment and mastery

Will be available via MAP rather than requiring a separate system.26

Display Resources for:InteractiveLesson Plans HandoutsQuestionsArticles

Science Test UpdateChanges to Science:

One test instead of two

Concepts and Processes no longer a stand alone test

Incorporated into the General Science test

64 to 45 items saving time

Current General Science Norms are applicable

Best Practices - SharingWhat Is Your District Doing to Help Parents Understand the Data?

Group into 2 or 3 different districts

Each share a strategy or practice that you use in your district

Talk about new ideas that youd like to try

Be prepared to share out one current practice from the group (or more) and one new idea to tryParent Resources from Other SchoolsParent Resources:

http://www.greenville.k12.sc.us/lforest/parents/map.asp http://lmms.acps.schoolfusion.us/modules/groups/homepagefiles/cms/2361977/File/MAPSCORESPARENTS.pdf http://www.ccsd89.org/index.php?option=com_content&task=view&id=1699&Itemid=256 http://www.edgerton.k12.wi.us/thedistrict.cfm?subpage=831873 http://www.pisd.edu/parents/assessment.accountability/map.shtml Parent Videoshttp://www.youtube.com/watch?v=9bPQOXMle00 Overview

http://www.youtube.com/watch?v=y5JsAoy8EVY Reading a Parent Report

http://www.youtube.com/watch?v=KHTkM-io27Q (This on is ours)

http://www.youtube.com/watch?v=k2oAbzEvAbk Cool kids talking about MAP

http://www.youtube.com/watch?v=C9BHHm5ZgjY This One Is Just for FUNField Testing OpportunitiesNWEA Needs YOUR Help!

3 Areas for Field Testing This Year:Spanish Math grades 2-5Upper Level Math (high school)

Standalone Field Testing (SFT) is NWEAs means of introducing and testing new items with a goal of increasing the breadth and depth of assessment knowledge in future Measures of Academic Progress (MAP) testing. The field-testing of items also allows NWEA to maintain the stability and accuracy of our measurement scale.Dan34What is required of a field test partner?ANY District is eligible:

No limit to the number of students that can participate (just limited to grade levels mentioned)

Each test will take about an hour students can take multiple tests on different subjects

Workstations must meet current technical requirements web based testing.

Either Server based districts or Web Based MAP districts can participateDan35Whats in it for YOUR District?NWEA will give renewal or new product credit up to $5.00 per completed test

You will expose your students to new item types that will eventually become part of the NWEA standard item bank

Your district will help to create the Next Generation of MAP AssessmentsDan36Knowledge Academy

Knowledge Academy

Knowledge AcademyKnowledge Academy Nomination

Minnesota Updates: MCA AlignmentKingsbury Center is Collecting Data NOW for a new MAP/MCA Cut Point AlignmentPlease send Don Draper your MCA dataNeed at least 1000 students per gradeOnce data are collected, linking will take about 2 weeksWrite and publish the reportLook for a new alignment by Jan-Feb this year

Norm Study - New Norms?Norm Study Conducted every 3 yearsLast norm study was in 2011Due to implementation of the Common Core SS in many states, research has decided to wait until 2015

STAR Claims:Cheaper- FasterBetterMAP & STARWhats the Difference?Lets look Under the Hood to see if these claims are validBy only looking at the shiny new outside???Buying a New Car..

..not all assessments are equal under the hoodYou Have to Look Under the Hood..

NWEA is the only adaptive testAdapts across all grades on a single scale (pre-k through 12)Based on 30+ years of solid researchHas the most stable-predictive scale in the industryUses norms that are truly nationalIs content neutralHas the largest item bank of any assessment programIs a non-profit, mission driven organizationSetting the Standard in Adaptive Testing for 30+ yearsHow stable is the scale?How are norms created, and how often updated?Is the assessment FULLY adaptive across a pre-K through 12 scale?What type of items are used, and how many items support the assessment? By strand?How long is the assessment and does it include enough items to give accurate data?What type of organization is the assessment company? Will they support you?

Review the QuestionsRIT Scale: stability and predictabilityVirtually no shift (change) in the RIT Scale for 20 yearsScale is stable and scores mean the same thing over timeEx: a student with a 200 RIT is at the same instructional level today as 20 years agoQuestion: How stable is the scale an assessment is using?Scale Stability & AccuracyNorming ProcessKingsbury Center at NWEA re-norms MAP test every 3 years (4 years this time)Norms both for growth & status are nationally representativeOur Norm Studies now include 5+ million students and 10s of million of test eventsSome test makers use a quartile or quintile regression model (misrepresents growth at extremes)Question: how representative are the norms? What methodology do they use?Norms for ComparisonsNWEA MAP Assessments are fully adaptive across all gradesSome adaptive assessments are constrained adaptiveWithin grade level (MCA)A couple of grades above or below (STAR)Will still miss outlier studentsAdaptive TestsQuestion: how adaptive is the assessment?New Item Types to assess deeper levels of DOK (depth of coverage)Sample of new items: here.STAR is still primarily MC and assesses lower cognitive skillsNWEA uses 7 to 10 items per strand to determine goal strand scoresSTAR uses fewer items per strandSTAR tests are shorter tests

Item Depth & BreadthItem exposure: STAR limits items to 75 to 90 day exposureMAP limits item exposure to 14 monthsSTAR item bankReading: 4700 itemsMath: 4000 itemsMAP item bankReading: 15,000 ELA itemsMath: 12,000 itemsItem Depth & BreadthQuestion: What type of items and how many items does an assessment use?Why does a MAP test take longer for students to complete? avg is 45 to 50 minutesAssessing the depth of knowledge higher order skills (not just lower cognition)Providing accurate information based on adequate item selection per strandGives you MORE information on broader range and depth of skillsFewer items means higher SEMLESS IS NOT ALWAYS MORE.especially in assessmentTime On TestMAP is untimed: students can show their achievement without time constraintSTAR is timed per item: forced selection after a set period of timeHow will this impact performance?Time on TestQuestion: How does time affect student performance?