assessment – an integral part of rti weighing cows wont make em fatter … just as, assessing...

Assessment – An Integral Part of RTI

Weighing cows won’t make ‘em fatter …just as, assessing children will not increase student learning!

Module 1

Model Tier 1 Framework

Review

Purpose of Training Modules

• RTI is not a program; RTI is a process• State provides a framework and process

– Through regulations– Through professional development – Through technical assistance

• District/School develops unique implementation– RTI implementation must take into consideration

unique characteristics of local culture– RTI should build on existing systems/initiatives

• District Leadership – Diversity in contribution/input/skills– Distribution in workload– Everyone is knowledgeable and supportive– Share framework and provide guidance for school

implementation

Tier 1

All Students, All Staff,

All Settings, All Year

??

??

??

?

Why Focus on Tier 1?

• Federal Law and State Regulations must ensure – student has received quality instruction – need for intervention is not due to

poor/inconsistent/ or lack of core instruction

• Research/Previous RTI Implementation– Spend Sufficient time on Tier 1 – Quality

Instruction - before implementation of Interventions

Most

IMPORTANT RTI

Component

RTI Framework• Where have we been?

– Established District Leadership Team– Overview of RTI – Needs Assessment of District and School Level

Implementation– Tier 1 Framework

• Where are we going?– Assessment Framework

• Assessments (December)• Data Management (December)• Data Analysis (February)• Team Problem Solving (February)

– Interventions (April)– SLD Determination (TBD)– Secondary Focus (2008-2009)

Assessment In The 3 Tier Model

RTI Module II

Delaware Department of Education

Purposes of Assessment in RTI

• To inform instruction

• To provide early intervention

• To monitor progress at the student, class, school, and district levels

• To evaluate instructional programs/strategies

Objectives for the Day

• T o identify ways to implement a comprehensive assessment plan

• To develop an understanding of screening, progress monitoring, diagnostic, and outcome assessments

• To identify the critical components of reading, math, early childhood, and behavior as related to assessment

Objectives for the Day (Continued)

• To evaluate current assessment tools• To review available assessment tools• To develop an understanding of a data

management system

Leadership is Key

Questions?

Assessment is the collection of data to make decisions.

(Salvia & Ysseldyke, 1997)

It is useless if we do not use it to guide our actions.

Tier I:Classroom/All

Students -Core Class Instruction

Tier II:Students not

responding to Tier I efforts – group interventions,

Specialized Research-based Interventions

Tier III:Students not responding to Tier I

or II interventions – Sustained Intensive Small Group & Individual

InterventionsPossible Special Education

Identification for non-responders of Tier III interventions

~80% of Students

~15%

~5%

Three Tiered Model

possibleSpecial Services

Incre

asin

g S

up

port

Assessment in a 3 Tier RTI Model

Tier I /Core Instruction

Universal Screening Progress

Monitoring Diagnostic Outcome

Assessment in 3 Tier RTI Model

Tier III

•Progress Monitoring •Diagnostic Assessment

Tier II •Progress Monitoring

•Diagnostic Assessment

Tier I /Core Instruction•Universal Screening •Progress Monitoring

•Diagnostic•Outcome

Comprehensive Assessment Plan

• Universal Screening • Progress Monitoring • Diagnostic Testing• Outcome Testing• Data Management System

Where Should the Assessment Go?

Universal Screening

Progress Monitoring

Diagnostic

Outcome

Which Assessment Term?

The assessment should measure the critical skills/components of the subject area it is assessing.

The assessment should yield similar scores if students were tested on a different day, by a different tester, or on a minimally different set of items.

Which Assessment Term?

The appraisal of student progress by using materials and procedures directly from the curriculum taught.

A simple set of procedures for repeated measurement of student growth toward long range instructional goals.

Which Assessment Term?

A specific criterion level of skills specified as an indication of an acceptableproficiency or mastery

A test used to determine the overall developmental level of a child with respect to other students

Which Assessment Term?

An assessment that provides information about student progress in order to make mid- course corrections or improvements to instruction.

The final assessment, usually quantitative in practice, of the degree to which the goals and objectives of a program have been attained.

Tier I: Universal Screening(NASDSE, 2005)

Universal Screening (of ALL students) occurs at least three times per year.

• Beginning, middle, end

Procedures must identify which students are:• proficient in the target skill, • developing the target skill, and • deficient in the target skill.

Basic question to be answered: • Should student be judged “at risk”?

Tier I Universal Screening

Criteria• Efficient – brief, accurate, inexpensive• Generally administered individually• Multiple probes• Clearly defined procedures for

administering and scoring• Broad Index – measures the Big Ideas• School-wide• Valid and Reliable

Universal Screening

Progress Monitoring

Diagnostic Information Outcome / Summative

Data Management System

Progress MonitoringCriteria

• Measures rate of growth toward an observable, measurable, and targeted goal

• Measures small increments of growth

• Has multiple forms

Progress Monitoring Criteria(Continued)

• Is efficient

• Is individually administered

• Is graphed and viewed regularly

• Is comparable across students

Trend lineor

Aim line?

DIBELS REPORTS

DIBELS REPORTS

Universal Screening

Progress Monitoring

Diagnostic Information Outcome / Summative

Data Management System

Diagnostic Information

– Knowledge about a child’s skills and abilities that is useful in planning instruction

– Can be derived from student work, teacher observations, or other tests, as well as diagnostic tests

Universal Screening

Progress Monitoring

Diagnostic Information Outcome / Summative

Data Management System

Outcome Assessment

• Outcome assessments are important because they give school leaders and teachers feedback about the overall effectiveness of their curriculum.

Universal Screening

Progress Monitoring

Diagnostic Information Outcome / Summative

Data Management System

Tier II

Weekly Progress Monitoring

Diagnostic assessments may need to be considered.

Tier IIDiagnostic Assessments

• Provide in-depth, reliable assessment of component skills

• Are relatively lengthy

• Are given when there is a clear expectation that it will provide new, reliable information about a child’s difficulties to inform more powerful instruction

Tier III

•Progress Monitoring

•Diagnostic Testing

A Comprehensive Plan

Universal Screening

Progress Monitoring

Diagnostic Information and Assessments

Outcome Assessment

Data Management System

Taking Stock

Math

Transforming Our Vision of How to Transforming Our Vision of How to Increase the Mathematics Proficiency Increase the Mathematics Proficiency of All Our Childrenof All Our Children

Effective Math Assessment Tools

• How do the characteristics of mathematical proficiency shape the design of effective screening and progress monitoring tools?

• What are the elements of effective math assessment?

Mathematical Reasoning Proficiency

Adaptive reasoning (Reasoning) –capacity for logical thought, reflection, explanation, and justification.

• Analyze• Compare/Contrast• Make an Inference• Evaluate• Classify

* In order to access reasoning the item must be “novel”

Pilots for Progress Monitoring

• America’s Choice• mCLASS:Math• Keymath3

Effective Mathematics Assessment

Although there is a purpose for assessments that compare the mathematical skills of a student or group of students to others in the district, state, nation, or world, these assessments are not created to facilitate an individual student’s learning.

Working Inside the Black Box:

Assessment for Learning in the Classroom

A follow up study to the original article by William and Black

Effective use of formative assessment has shown a direct correlation to student learning and include the following characteristics:

• The learning targets are shared clearly with students from the beginning of the learning

• Classroom assessments accurately measure achievement of the important learning targets

• Students are given continuous, descriptive feedback that includes evidence about what they currently do understand and what they still need to work on

• Students are involved in the assessment, record keeping, and communication of learning

• Students understand how to close the gap between the goal and where they currently “are” (Black & William, 1988;Stiggins in DuFour et al., 2005)

Can a Universal Screening Tool be formative in design yet still set benchmarks that help to identify students in need of intervention?

Vision for the Future

Statewide Universal Screening Tool with Embedded Formative Assessments

Identifies and Informs

Suggests Interventions

Continuous Fine Tuning of our Collective Knowledge

Identify theFocal Points

for each Grade

Student Work Sampleswith

Suggested InterventionsUnpacking of embeddedAssessments to create

Universal Screening Tool

Selection of embedded

Assessment Tasks

Benchmarks of Student Understanding

AndMathematical Proficiency

Initial Data Collection:Curriculum Based

Formative AssessmentTasks

Teacher Involvement

Identification of students below

Benchmarks

Vision for the Future

Phase OnePhase One• Current Assessment Practice DataCurrent Assessment Practice Data

– Quarterly AssessmentsQuarterly Assessments– Transfer TasksTransfer Tasks

• Lead Teacher TeamsLead Teacher Teams– Grade Level Representation (Initial Focus Grade K-5)Grade Level Representation (Initial Focus Grade K-5)– Nomination ProcessNomination Process– Begin identification of key tasks and student work Begin identification of key tasks and student work

datadata• State/District SupportState/District Support

Early Childhood

Early Childhood Assessmentand Progress Monitoring

Verna Thompson

vthompson@doe.k12.de.us

What happens at Tier 1

ALL young children have access to:• Evidence-based curriculum for ALL areas of

development • Effective teaching strategies & learning

opportunities for ALL children• Universal “probing” of key skills in ALL areas • Assessing acquisition of key skills in ALL areas

Aligned to:

What Happens at Tier 1

Results of progress monitoring of development show:• Most children are making progress

– Curriculum planning is effective– Learning opportunities are meaningful

• Most children are not making sufficient progress– Modification of teaching strategies for whole class

• Some children are not making sufficient progress– “Early intervening” for individual children

Recognition and Response Guidelines for Progress Monitoring and Probing

Where: In a variety of natural settings & routinesWho: By Informed caregivers (teachers, parents, teams) What: Collection of multi-sources of data in ALL areas

• Curriculum based assessment aligned with Early Learning Foundations • Observations• Parent Information• Work Samples• Checklists

When: Ongoing assessment of skills in ALL areas Periodic “Probing” of ALL Key Skills Ideally same tool is used for Monitoring and Probing

Results inform curriculum, teaching strategies, planned learning opportunities for children

Guidelines for Selecting Authentic Assessment Tools

Authentic Assessment measures should have the following characteristics:

• Curriculum-based• Designed to be used multiple times• Easy to score• Sensitive to individual differences• Provide information on both level and rate of growth in

key areas of learning• Related to long term learning goals in curriculum• Aligned to Delaware Early Learning Foundations

Ideally, single measure should be used for authentic assessment and probing for key skills

Authentic Assessment Tools for early Childhood

• Curriculum-based tools – (aligned to Early Learning Foundations)– Creative Curriculum Developmental Assessment– High Scope COR– Carolina Curriculum

• Checklists linked to Early Learning Foundations– (Collected from multi-sources of information)– RORS– Work Sampling– Developmental Checklist Birth to Five

Early Childhood in Delaware

• Head Start and Early Childhood Assistance Programs use authentic assessment

• Early Learning Foundations being revised – Will include format for assessing children

• Child Care Licensing Regulations revised– Centers required to assess children – Training on authentic assessment is in planning

• ABCD Grant – Public Health – Screening initiative

ABCD GrantAssuring Better Child Health and Development

• Collaboration with Medicaid, DPH, DE-AAP, Autism Society of DE

• Developmental Screening for all children at well-visits with a standardized screening tool – Ages & Stages, PEDS– 9 mos, 12 mos, 18 mos, 24 mos, 36 mos

• Being piloted in two community practices in DE • Will be replicated statewide with Medicaid policy• Plans include training early childhood providers

Effort by the National Association of State Heath Policy

Recognition and Response Pilot

Pilot States presently implementing R&R:• Connecticut• Arizona• Maryland• Florida

What Can Delaware School Districts Do Now?

• Advocate for QUALITY Early Childhood programs

• Develop RELATIONSHIPS with families and programs in community

• Develop PARTNERSHIPS with community programs – Communicate with programs in community– Suggest checklists aligned to Early Learning

Foundations– Provide training on using assessment checklists

• Use AUTHENTIC ASSESSMENT to monitor children’s progress

• NOT responsible for screening all children from birth in community

Using Recognition and Response in Kindergarten

Kindergarten Screening:• Consider Pre Kindergarten experiences

– Child was in an environment that provided opportunities

– Kindergarten is the first learning opportunity

• Analyze screening results

Activity

At your table:

1. Select one area of learning from

Delaware Early Learning Foundations

2. Observe children in video clip

3. Document learning opportunities you observed in that area of learning

4. Discuss what you observed with the group

Behavior

RTI and Behavior/Mental Health

Do we have to “do RTI” for

behavior and social-emotional

issues???

No!

Does it make any sense to “do RTI” for behavior and social emotional issues?

Yes!

RTI and Behavior/Mental Health

Are we already “doing RTI” for behavior and social emotional issues?

Quite Possibl

y!

RTI and Behavior/Mental Health

• Tier 1:

– Schoolwide teaching of behavioral expectations and social emotional competencies

– Using schoolwide data to identify and address problems

Universal Screening and Behavior/ Mental Health

• What is already happening?

– Monitoring office discipline referrals

– Teacher referral to school-based problem solving team

Universal Screening and Behavior/ Mental Health

Are we ready to increase our attention to universal screening for behavior and social emotional issues?

Maybe!

Universal Screening and Behavior/ Mental Health

• Purpose: to identify youth who have high risk for developing behavioral or mental health problems

• Conducted on a schoolwide basis

• Typically involves several levels of assessment to avoid over- or under-identification of students

Universal Screening and Behavior/ Mental Health

• Multiple Gating Procedures– Gate 1: teacher nomination procedure– Gate 2: teacher rating scale procedure– Gate 3: observation and/or more detailed

rating scales

Identification of students most at-risk

Intervention Planning

Universal Screening and Behavior/ Mental Health

• Self-report procedures– Gate 1: schoolwide screening– Gate 2: follow up interview– Gate 3: diagnostic interview

Universal Screening and Behavior/ Mental Health

• What does progress monitoring look like?

– The same for academic progress– More individualized around specific

behavioral or social-emotional concerns

Daily Report Cards

Tools for progress monitoring:

http://www.jimwrightonline.com/

php/tbrc/tbrc.php

Universal Screening and Behavior/ Mental Health

• Read more about it….

Some Options…..

Data ManagementSystems

DATA MANAGEMENT FREEWARE

http://www.jimwrightonline.com

RTI: Graph Data for Visual Analysis -Charts and graphs transform progress-monitoring data into visual displays. -Time-series graphs in particular are widely used-A positive trend-line demonstrates when the student is responding well to intervention.

I. Excel Graphs Made Easy -Download pre-formatted Excel spreadsheets -Enter data and create time-series graphs for common academic

measures -Designed by Dr. Jim McDougal and student colleagues Karrie Clark

and Jacklyn Wilson from SUNY College at Oswego.

II. Generate Time-Series Graphs On-Line-Enter your student data into the on-line application ChartDog-Create time-series graphs -Plot trend lines

III. Paper Charts-Chart your data by hand-Collection of blank time-series charts that you can download and print

DATA MANAGEMENTCOMMERCIAL

Existing Grading Program & ExcelSoftwareWeb basedHandheld Qualitative Data

Examples:

http://www.edcheckup.com/ (demos)

http://www.wirelessgeneration.com/ (archived web casts)

Universal Screening

Progress Monitoring

Diagnostic Information Outcome / Summative

Data Management System

A Look at the Real World……

Implementation of a Comprehensive

Data PlanPresented by Capital School District

Pam Hererra Colleen RinkerMichele Waite

A Comprehensive Framework for Reading Instruction

Comprehensive Assessment

System

HEADMaking

informed decisions

based on data

Literacy Support Team(LST)

Instructional Support Team(IST)

Whole Body is Tier 1 instruction

High quality, effective instruction for all students

Grade Level Meetings Grade Level Planning

Clothes are Tiers 2 and 3 interventions and they change based on needs

Comprehensive Assessment Matrix for Reading

• Organization of the assessment system• Purposes for assessments• Timelines for assessments and

schedules• Using an Assessment Team• Importance of professional development

related to administering assessments• Role of district in implementation

School Level Implementation

• Literacy Support Team Meetings and connections to IST

• Use of assessment data• Model Lessons• Follow up with teachers• Data notebook for principal• Importance of principal involvement at all

levels

Implementation at the School Level

• Data notebooks for each teacher• Additional assessments and purposes• Diagnostic assessments (chart)• Infrastructure necessary• Grade level meetings and planning

based on data• Professional Development

Your RoleBefore and After

the RTI Initiative

Topics to Consider

• DE RTI Regulations • Schedules • Professional

Development • Accountability • Materials & Resources• Data Collection &

Review

Taking Stock

Conclusions and Evaluations

Vendors