PowerPoint Presentation

Assessment and the FAIR-FS: An Overview

1

Welcome to the Clay County School Districts on-line professional development targeting the new FAIR-FS assessment. This training is geared towards the implementation of the FAIR-FS assessment and contains critical information that K-2 teachers and administrators will need prior to beginning the assessment. 12Goals of this on-line training:Explain the purpose of the FAIR-FSExplain how to log into the PMRN to gain access to the FAIR-FS assessment and reportsReview testing timelineUnderstand how to administer the FAIR-FSUnderstand how to administer the WSS (replacement for the ECHOS)Provide a brief overview of the reports generated from the FAIR-FS

Student: Am I accomplishing what my teacher expects? How am I doing compared to other students?Administrator: Are a majority of our students succeeding with our curriculum?

Teacher: Are my students making progress? How do I differentiate instruction?Parent: Is my child making the necessary progress?

3

This first section describes the context for assessment in general and the role that FAIR-FS plays in assessing Floridas students. Schools have a variety of stakeholders with a variety of critical decisions to make in order to impact those life and career outcomes we all know can be impacted by education. Stakeholders increasingly want objective information in order to make their decisions. One source of information (e.g., end of year test) cannot appropriately answer each of these questions.

3Functions of AssessmentIs Lucy responding to the curriculum? How is Lucy doing in reading compared with other students her age? Is Lucy gaining the skills necessary to be a good reader? Is Lucy making appropriate progress to graduate from high school? Is Lucy demonstrating skills that represent the Language Arts Florida Standards?Did Lucy understand the concept presented in class yesterday? Report CardsProgress monitoringEnd of Year testScreeningFormative Assessment4

These are just a few of the critical questions that need to be answered within education and some of the mechanisms by which those questions are answered.

Is Lucy responding to the curriculum? (report card grades, curriculum embedded assessments) How is Lucy doing in reading compared with other students his age? (screening)Is Lucy gaining the skills necessary to be a good reader? (screening)Is Lucy making appropriate progress to graduate from high school? (end of year test)Is Lucy demonstrating skills that represent the standards? (formative assessment)What is the likelihood that Lucy will pass the end of year test? (screening)Did Lucy understand the concept presented in class yesterday? (formative assessment)Does Lucy have a learning disability? (ongoing progress monitoring of intensive evidence based instruction/intervention) Is my child making the necessary progress?(end of year test)

Each assessment tool is developed to answer specific questions and may or may not be used to answer other questions. They must be validated for each specific use. 4Questions Addressed by FAIR-FSWhich important reading skills are strengths and weaknesses for Lucy?What skills should be targeted for instruction in order to improve Lucys reading comprehension?What is the likelihood that Lucy will pass the end of year test?Has Lucy made progress since the beginning of the school year?5

Screening and diagnostic assessments like the FAIR-FS answer different questions than those asked by FCAT or the new LAFS outcome tests or other high stakes assessments. The FAIR-FS is both a screening and diagnostic assessment and measures students strengths and weaknesses in relation to literacy skills.

Present slide.

5The FAIR-FS is a comprehensive assessment designed to:Predict students literacy successDiagnose strengths and weaknesses in the area of literacyResearch-based teachable skillsSet instructional objectivesMonitor literacy growth

6

There are 4 main ways the FAIR-FS can assist classroom teachers in meeting the needs of all students in the classroom: 1. Predict students literacy success. 2. Diagnose students strengths and weaknesses in the area of literacy 3. Assist teachers in setting instructional objectives, and 4. monitoring the literacy growth of all students. As you can see, each of these design features relates to ongoing instruction and literacy growth. FAIR-FS is not about obtaining an outcome score.

6Integrated AssessmentEvidence base for data-based decision making Screening Assessments (FAIR-FS)Diagnostic Assessments (FAIR-FS)Formative Assessments (classroom-based)Outcome Assessments (SAT-10, FSA)

7

Assessing students ability to read and understand requires a systems approach that includes multiple components as part of a carefully planned comprehensive assessment plan. SAT - 10 Stanford Achievement Test, Tenth Edition is considered a gold standard nationally-normed outcome assessment of reading comprehensionFSA Florida Standards Assessment

7Reliability & ValidityFirst 2 questions asked about measurement:Does it actually measure what they say it measures? validity Does it measure the same thing every time? reliability Computer-adaptive format of FAIR-FS increases reliability (and efficiency) by tailoring the task to the studentAverage reliability on FAIR-FS is 0.9 (very high)FAIR-FS demonstrates predictive validity8

In order to make claims of validity, an assessment must first be determined to be reliable. The answers to these 2 questions can be quantified through research studies. In FCRRs studies of the tasks within FAIR-FS, they first established reliability.

The reliability of an assessment is considered to be adequate at 0.7 and good at 0.8. The average reliability for FAIR-FS is exceptional (>0.9). Since the assessment is computer-adaptive we can also calculate the reliability for each individual student. In the field study, researchers found that FAIR-FS was reliable for 99% of students in grades K through 10.

Once we found that tasks were reliable, we then conducted a prospective study to determine validity. There are many different types of validity that can be reported, depending on the purpose of the assessment. Careful scrutiny of technical reports are needed to determine if validity has been established for the purpose the assessment is used.8Reliability & Validity

Not ReliableReliableValidNot Valid9

Presenter: As an example, think of a target. The most valid shots are the ones that hit the bullseye, the center of the target. Reliable shots are the ones that cluster together. The upper left target shows that the shots are scattered, random, and haphazard. In other words, both reliability and validity are low.

The top right target shows that the shots all hit the same general area, so they were reliable. However, they didnt land on the bulls eye so they arent valid.

In the bottom left target, the shots are closer to the bulls eye so they are more valid but not reliable because they arent clustered.

In the bottom right target the shots hit both the bulls eye (valid) and are clustered together (reliable).

The FAIR-FS has been piloted in several districts across the state and has been found to be exceptionally reliable and valid. This means as a classroom teacher, administrator, or parent you have a high level of assurance that the FAIR-FS will accurately predict a students literacy success as set by the golden standard of assessments in Florida the SAT-10.9Efficiency & ReliabilityThe FAIR-FS utilizes Computer Adaptive functionality to:Maximize the amount of information providedMaximize reliabilityMinimize testing timeResults in scores that can be used toTarget instructionMonitor growth in specific skills10

Presenter:In the FAIR 2009 system, users really liked the small amount of time that it took to administer the assessment. However, many expressed frustration that the screening component only included 1 to 2 discrete skills and provided less information about important reading skills than other reading screening assessment tools.

The FAIR-FS has more tasks in the screening to provide quite a bit more information. Adding more tasks increases the amount of time required for assessment. The amount of time for each task has been greatly reduced by using more sophisticated statistics: Item Response Theory. Formulas are calculated by the computer in the background.

10Section SummaryDifferent types of assessments are required to meet different purposes

FAIR-FS purposes

FAIR-FS is reliable and valid for use as a screening toolFAIR-FS is an efficient assessment11

Presenter: Weve come to the end of the first section. The bulleted items summarize what we discussed.

Present slide.

Additional information:Cited from Torgesen & Miller (2009)

For more information about integrating assessments, see Assessments to Guide Adolescent Literacy Instruction at http://centeroninstruction.org/adolescent-literacy-institute

11Progress Monitoring and Reporting Network (PMRN) Overview

12

Next we are going to take a look at accessing the PMRN as this is how you will access the K-2 assessment.12System SpecificationsDesktop, Laptop, Netbook & Thin Client / Virtual Desktop Infrastructure

Input Device RequirementsKeyboard, MouseHeadphone/Earphone RequirementsK-2 requires a Y jack splitter

13

Present slide.

The input device must allow students to select/deselect, drag, and highlight objects and areas. Based on our field testing, we recommend using an external mouse and not a touchpad for the K-2 FAIR-FS AIR. The input device must allow students to enter letters, numbers, and symbols and shift, tab, return, delete, and backspace. To meet security guidelines, each Bluetooth/wireless keyboard and/or mouse must be configured to pair with only a single computer during assessment administration.Other assistive technologies may be needed for students requiring accommodations.

Headphones are required for both the K-2 and 3-12 FAIR-FS. It is recommended that for the K-2 that the audio output have a splitter (Y) and both the teacher and the student have headphones.In the field testing of the K-2 application, assessors found the Y splitter to be necessary since testing in a quiet room with just the speakers playing the audio was a highly unlikely scenario.

There is no requirement for microphones. 13System SpecificationsBrowser Specifications Internet Explorer (IE)Version 9, 10FirefoxVersion 26SafariVersion 5.1.7Flash PlayerVersion 10

NOTE: FAIR-FS CANNOT BE ACCESSED USING CHROME14

This slide shows the various browsers that support the FAIR-FS. Of critical importance is to note that Google Chrome is not to be used as it cant support the assessment.14What is the PMRN?Tool (web-based data management system) for facilitating the use of reading/language arts data to inform instructional decisionsThe PMRN facilitates this process through:Setting the calendar for administering screening assesmentRostering students in grade levels at schools with assigned teachersSummarizing and reporting specific data points for use in instructional decisionsAuthorizing educators to access student data15

In order to facilitate instructional decision-making in the area of reading, teachers, reading coaches, and support staff need a tool to organize data and make that data readily AND securely available to those making instructional decisions for individual students, groups of students, and schools of students. The first version of the Progress Monitoring and Reporting Network (PMRN) was created to organize and report the data from the DIBELS assessments and subsequent versions were enhanced to facilitate use of the FAIR 2009.

The PMRN version 4 serves several vital functions that facilitate the administration of and data organization for the FAIR-FS. This is a very basic view of the complex methods the PMRN v4 uses.15Progress Monitoring and Reporting Networkcalendaringrostering reportingauthorizing3-12 FAIR-FS WAMK-2 FAIR-FS AIRWSSCoachs Log3-12 FAIR-FS ReportsK-2 FAIR-FS ReportsFLKRS ReportsCoachs Log Reports16

The new version of the PMRN (Version 4) performs the calendaring, rostering, authorizing, and reporting functions that enable the use of the FAIR-FS assessments. It provides access to the input applications (shown along the top of this graphic). These are the applications that administer the grades 3-12 assessments and the grades K-2 assessments. In grades K-2, an Adobe application must be downloaded onto a computer and can be administered with or without access to the internet.

The PMRN is also the system used for reporting the Florida Kindergarten Readiness Screener (FLKRS), which is a statutory requirement for screening Kindergarten students who previously attended a Florida VPK program.

The PMRN houses the Professional Development Log for Reading/Literacy Coaches, which is required by State Board Rule 6A-6.053a and is a tool that is used to identify the critical roles of a reading coach that will impact the reading program at a school and to guide the performance of the Reading/Literacy Coach.16New Addresshttps://pmrn.fldoe.org

17

Notice that the PMRN is now completely hosted on FLDOEs web servers and not FCRR.It is important to note that PMRN Users that utilize the SSO can access the PMRN through both the FLDOE SSO Portal and the PMRN Sign In page.17PMRN SecuritySaving Passwords

Signing Out

Protect Your Identity

18

The PMRN was designed to protect student and teacher data. Secure Socket Layer (SSL): The PMRN is a secure socket layer (SSL) encrypted site, using the same 128-bit encryption that a bank website uses.

Saving Passwords: Some computers and Internet browsers allow you to Save Passwords. It is recommended that you do not have your computer remember your PMRN Password. Ask your site technology specialist about disabling the option to Save Passwords.

Signing Out: When you complete a PMRN session, please Sign Out. The PMRN will automatically end your session if you are idle for twenty minutes. If you must leave the computer, please Sign Out so that another person who might use your computer cannot have access to information within the PMRN.Note: To prevent being timed out, click Save, refresh the page, or load a new report within twenty minutes. Typing is not considered an action that prevents being timed out, as the database is not refreshed.

Protect Your Identity: You should keep your SSO User Name and Password in a secure location, and you should not share it with anyone. Details about privacy and security issues can be found on the PMRN site through the Copyright, Privacy Statement, and Disclaimer links found at the bottom of the Sign In page.

18Enhancements to the PMRNNew URL/Web Addresshttps://pmrn.fldoe.org

No student exclusions through PMRNStudents not assessedShould not be rostered into PMRNIf previously rostered, must be withdrawn from school 19

Present slide.

Additional information:Students may not be excluded from testing through the PMRN. Students who will not be assessed using the FAIR-FS are not to be rostered in the PMRN and, if they are rostered, they are to be withdrawn from the school (NOT only removed from the class).

19Enhancements to the PMRNAssessment CalendarThree equal Assessment Periods (AP)AP1 = Instructional Days 1 60AP2 = Instructional Days 61 120AP3 = Instructional Days 121 - 180No Norming PeriodsNo Lockdown PeriodsNOTE: FLKRS must still be administered during the first 30 instructional days

20

Presenter:The year will be divided into three equal Assessment Periods of 60 days each (AP1 = Instructional Days 1-60; AP2 = 61-120; AP3 = 121-180).

There will no longer be an identified Norming period or Lockdown period.

NOTE: FLKRS must still be administered during the first 30 instructional days.

20Enhancements to the PMRNInvalidation and WithdrawalScore invalidation is no longer availableSupport Specialists unable to invalidate scoresStudents not completed with task may be withdrawnStudent may continue with task when enrolled in new school that they left off of in previous school21

The Support Specialists at the Help Desk at the Florida Center for Reading Research or the FLDOE or your district reading contact will not be able to invalidate scores. Students who have not completed a task can be withdrawn without completing the task. The next school that enrolls the student during the same AP will pick up the student where the student left off in the task.

21Enhancements to the PMRNCopied ScoresScores from previous school will transfer automatically when student is enrolledSchool Level Users and teachers do not have the option to copy or not copy scores22

The process of copying scores is being modified.

If a student already has scores from a previous school, the scores will automatically be transferred.

Teachers and school level Users will not have the option to copy or not copy the scores.

22Enhancements to the PMRNPMRN v4 Users Guidev4 Users Guide currently in development by FLDOEv3 Users GuideIt is recommended that Users use the v3 Users Guide until the v4 Users Guide is availableMajor functions of the two applications are similar23

Presenter:At the beginning of the 2014-2015 school year, there will not be a Users Guide for PMRN v4.

Until one is developed, it is recommended that Users use the 2013-2014 v3 Users Guide.

Although the look and feel of PMRN v3 and PMRN v4 are different, the major functions of the two applications are similar.23Service Center InformationFlorida Department of Education (FLDOE) Integrated Education Network Service CenterProvide support via the PMRN v4, K-2 AIR, 3-12 WAM, and FAIR-FSOperational HoursMonday Friday6:00 AM 7:30 PM ETContact via Phone or E-mail855-814-2876ienhelp@fldoe.org

24

Presenter:Questions and requests for assistance should be directed to the Florida Department of Education (FLDOE) Integrated Education Network Service Center. The Service Center will provide Users with assistance on questions and issues regarding the PMRN, K-2 AIR, 3-12 WAM, and FAIR-FS.

Service Center operational hours are Monday through Friday, 6:00 AM 7:30PM ET.24Single Sign-On

25

Another great feature of the new FAIR-FS is that you no longer need to have a separate user name and password to log into the PMRN and to access the FAIR-FS. 25What is Single Sign-On?The Florida Department of Education (FLDOE) Single Sign-On (SSO) provides a simpler way for educators to access several state resources with one username and one password. Website: www.fldoe.org/sso

26

Present slide.

26Who can use Single Sign-On?

Essentially, everyone can use FLDOE SSO, however, the resources you can access depend on your role and the permissions you have been assigned. 27

Present slide.27Teacher Resources on SSOCPALMS - Collaborate, Plan, Align, Learn, Motivate, Share

eIPEP - Electronic Institutional Program Evaluation Plans

ELFAS - English Language Arts Formative Assessment System

IBTP - Items Bank and Test Platform

FSL - Florida School Leaders

PMRN - Progress Monitoring and Reporting Network 28

These resources will be available through the SSO during the upcoming school year and training will be provided for each separately as the training becomes available.

28Signing in on the PMRN

Go to www.fldoe.org/sso. Click the yellow Log In button on the right side.Use the drop down list to select your organization, and then click Continue to Sign In. Enter the username and password used to sign into the CCSD network and proceed to log in.Then, the secure FLDOE SSO page is displayed with your name and authorized resources.

29

This slide shows you how to sign in on the PMRN. It is recommended that you print these power point slides to use as a reference once you go to log into the PMRN.

29

Need Assistance?For additional assistance managing your Single Sign-On account refer to the SSO Support Page on how to contact your district/organizations help desk.

30

Present Slide.

30Single Sign-On & PMRNKey TermsAuthentication is the process of identifying an individualAuthorization is the process of granting or denying access to a network resourceAs it relates to the PMRNSingle Sign-On credentials authenticate users who need to access the PMRN. Users will need authorization granted from within the PMRN by their Principal (SL1), or Reading Coach (SL2) to access PMRN resources. Once a User has been authorized The Users authorization in the PMRN must be linked to the SSO account.The two are linked within the PMRN.

31

Presenter

Weve introduced the SSO, now were going to discuss the interaction of the SSO with the PMRN.31Single Sign-On & PMRNTo access the PMRN through the Single Sign-On Portal use your SSO credentials at http://www.fldoe.org/sso

32

Single Sign-On & PMRNOnce you have signed into the Single Sign-On Portal you should see the PMRN link under the Curriculum & Assessments header.If the link is not available you will need to contact your districts Help Desk for additional authorization.

33

Single Sign-On & PMRNYou can also Log In to the PMRN using your SSO credentials at https://pmrn.fldoe.org

34

The Username and Password fields above the SSO icon is for non-public school access.

34Single Sign-On & PMRNIf you attempt to access the PMRN website and you receive the following error page you will need to contact the CCSD help desk for additional assistance in obtaining your SSO authentication for accessing the PMRN

35

35For More InformationCurriculum questions: Contact your district reading office

Content and policy questions: Contact Just Read, Florida! at 850-245-0503 http://www.justreadflorida.com/

Technical questions: Call Florida Department of Education (FLDOE) Integrated Education Network Service Center 855-814-2876 ienhelp@fldoe.org36

Here is contact information for your reference.36