assessment – an integral part of rti weighing cows wont make em fatter … just as, assessing...
TRANSCRIPT
Assessment – An Integral Part of RTI
Weighing cows won’t make ‘em fatter …just as, assessing children will not increase student learning!
Module 1
Model Tier 1 Framework
Review
Purpose of Training Modules
• RTI is not a program; RTI is a process• State provides a framework and process
– Through regulations– Through professional development – Through technical assistance
• District/School develops unique implementation– RTI implementation must take into consideration
unique characteristics of local culture– RTI should build on existing systems/initiatives
• District Leadership – Diversity in contribution/input/skills– Distribution in workload– Everyone is knowledgeable and supportive– Share framework and provide guidance for school
implementation
Tier 1
All Students, All Staff,
All Settings, All Year
??
??
??
?
Why Focus on Tier 1?
• Federal Law and State Regulations must ensure – student has received quality instruction – need for intervention is not due to
poor/inconsistent/ or lack of core instruction
• Research/Previous RTI Implementation– Spend Sufficient time on Tier 1 – Quality
Instruction - before implementation of Interventions
Most
IMPORTANT RTI
Component
RTI Framework• Where have we been?
– Established District Leadership Team– Overview of RTI – Needs Assessment of District and School Level
Implementation– Tier 1 Framework
• Where are we going?– Assessment Framework
• Assessments (December)• Data Management (December)• Data Analysis (February)• Team Problem Solving (February)
– Interventions (April)– SLD Determination (TBD)– Secondary Focus (2008-2009)
Assessment In The 3 Tier Model
RTI Module II
Delaware Department of Education
Purposes of Assessment in RTI
• To inform instruction
• To provide early intervention
• To monitor progress at the student, class, school, and district levels
• To evaluate instructional programs/strategies
Objectives for the Day
• T o identify ways to implement a comprehensive assessment plan
• To develop an understanding of screening, progress monitoring, diagnostic, and outcome assessments
• To identify the critical components of reading, math, early childhood, and behavior as related to assessment
Objectives for the Day (Continued)
• To evaluate current assessment tools• To review available assessment tools• To develop an understanding of a data
management system
Leadership is Key
Questions?
Assessment is the collection of data to make decisions.
(Salvia & Ysseldyke, 1997)
It is useless if we do not use it to guide our actions.
Tier I:Classroom/All
Students -Core Class Instruction
Tier II:Students not
responding to Tier I efforts – group interventions,
Specialized Research-based Interventions
Tier III:Students not responding to Tier I
or II interventions – Sustained Intensive Small Group & Individual
InterventionsPossible Special Education
Identification for non-responders of Tier III interventions
~80% of Students
~15%
~5%
Three Tiered Model
possibleSpecial Services
Incre
asin
g S
up
port
Assessment in a 3 Tier RTI Model
Tier I /Core Instruction
Universal Screening Progress
Monitoring Diagnostic Outcome
Assessment in 3 Tier RTI Model
Tier III
•Progress Monitoring •Diagnostic Assessment
Tier II •Progress Monitoring
•Diagnostic Assessment
Tier I /Core Instruction•Universal Screening •Progress Monitoring
•Diagnostic•Outcome
Comprehensive Assessment Plan
• Universal Screening • Progress Monitoring • Diagnostic Testing• Outcome Testing• Data Management System
Where Should the Assessment Go?
Universal Screening
Progress Monitoring
Diagnostic
Outcome
Which Assessment Term?
The assessment should measure the critical skills/components of the subject area it is assessing.
The assessment should yield similar scores if students were tested on a different day, by a different tester, or on a minimally different set of items.
Which Assessment Term?
The appraisal of student progress by using materials and procedures directly from the curriculum taught.
A simple set of procedures for repeated measurement of student growth toward long range instructional goals.
Which Assessment Term?
A specific criterion level of skills specified as an indication of an acceptableproficiency or mastery
A test used to determine the overall developmental level of a child with respect to other students
Which Assessment Term?
An assessment that provides information about student progress in order to make mid- course corrections or improvements to instruction.
The final assessment, usually quantitative in practice, of the degree to which the goals and objectives of a program have been attained.
Tier I: Universal Screening(NASDSE, 2005)
Universal Screening (of ALL students) occurs at least three times per year.
• Beginning, middle, end
Procedures must identify which students are:• proficient in the target skill, • developing the target skill, and • deficient in the target skill.
Basic question to be answered: • Should student be judged “at risk”?
Tier I Universal Screening
Criteria• Efficient – brief, accurate, inexpensive• Generally administered individually• Multiple probes• Clearly defined procedures for
administering and scoring• Broad Index – measures the Big Ideas• School-wide• Valid and Reliable
Universal Screening
Progress Monitoring
Diagnostic Information Outcome / Summative
Data Management System
Progress MonitoringCriteria
• Measures rate of growth toward an observable, measurable, and targeted goal
• Measures small increments of growth
• Has multiple forms
Progress Monitoring Criteria(Continued)
• Is efficient
• Is individually administered
• Is graphed and viewed regularly
• Is comparable across students
Trend lineor
Aim line?
DIBELS REPORTS
DIBELS REPORTS
Universal Screening
Progress Monitoring
Diagnostic Information Outcome / Summative
Data Management System
Diagnostic Information
– Knowledge about a child’s skills and abilities that is useful in planning instruction
– Can be derived from student work, teacher observations, or other tests, as well as diagnostic tests
Universal Screening
Progress Monitoring
Diagnostic Information Outcome / Summative
Data Management System
Outcome Assessment
• Outcome assessments are important because they give school leaders and teachers feedback about the overall effectiveness of their curriculum.
Universal Screening
Progress Monitoring
Diagnostic Information Outcome / Summative
Data Management System
Tier II
Weekly Progress Monitoring
Diagnostic assessments may need to be considered.
Tier IIDiagnostic Assessments
• Provide in-depth, reliable assessment of component skills
• Are relatively lengthy
• Are given when there is a clear expectation that it will provide new, reliable information about a child’s difficulties to inform more powerful instruction
Tier III
•Progress Monitoring
•Diagnostic Testing
A Comprehensive Plan
Universal Screening
Progress Monitoring
Diagnostic Information and Assessments
Outcome Assessment
Data Management System
Taking Stock
Math
Transforming Our Vision of How to Transforming Our Vision of How to Increase the Mathematics Proficiency Increase the Mathematics Proficiency of All Our Childrenof All Our Children
Effective Math Assessment Tools
• How do the characteristics of mathematical proficiency shape the design of effective screening and progress monitoring tools?
• What are the elements of effective math assessment?
Mathematical Reasoning Proficiency
Adaptive reasoning (Reasoning) –capacity for logical thought, reflection, explanation, and justification.
• Analyze• Compare/Contrast• Make an Inference• Evaluate• Classify
* In order to access reasoning the item must be “novel”
Pilots for Progress Monitoring
• America’s Choice• mCLASS:Math• Keymath3
Effective Mathematics Assessment
Although there is a purpose for assessments that compare the mathematical skills of a student or group of students to others in the district, state, nation, or world, these assessments are not created to facilitate an individual student’s learning.
Working Inside the Black Box:
Assessment for Learning in the Classroom
A follow up study to the original article by William and Black
Effective use of formative assessment has shown a direct correlation to student learning and include the following characteristics:
• The learning targets are shared clearly with students from the beginning of the learning
• Classroom assessments accurately measure achievement of the important learning targets
• Students are given continuous, descriptive feedback that includes evidence about what they currently do understand and what they still need to work on
• Students are involved in the assessment, record keeping, and communication of learning
• Students understand how to close the gap between the goal and where they currently “are” (Black & William, 1988;Stiggins in DuFour et al., 2005)
Can a Universal Screening Tool be formative in design yet still set benchmarks that help to identify students in need of intervention?
Vision for the Future
Statewide Universal Screening Tool with Embedded Formative Assessments
Identifies and Informs
Suggests Interventions
Continuous Fine Tuning of our Collective Knowledge
Identify theFocal Points
for each Grade
Student Work Sampleswith
Suggested InterventionsUnpacking of embeddedAssessments to create
Universal Screening Tool
Selection of embedded
Assessment Tasks
Benchmarks of Student Understanding
AndMathematical Proficiency
Initial Data Collection:Curriculum Based
Formative AssessmentTasks
Teacher Involvement
Identification of students below
Benchmarks
Vision for the Future
Phase OnePhase One• Current Assessment Practice DataCurrent Assessment Practice Data
– Quarterly AssessmentsQuarterly Assessments– Transfer TasksTransfer Tasks
• Lead Teacher TeamsLead Teacher Teams– Grade Level Representation (Initial Focus Grade K-5)Grade Level Representation (Initial Focus Grade K-5)– Nomination ProcessNomination Process– Begin identification of key tasks and student work Begin identification of key tasks and student work
datadata• State/District SupportState/District Support
Early Childhood
What happens at Tier 1
ALL young children have access to:• Evidence-based curriculum for ALL areas of
development • Effective teaching strategies & learning
opportunities for ALL children• Universal “probing” of key skills in ALL areas • Assessing acquisition of key skills in ALL areas
Aligned to:
What Happens at Tier 1
Results of progress monitoring of development show:• Most children are making progress
– Curriculum planning is effective– Learning opportunities are meaningful
• Most children are not making sufficient progress– Modification of teaching strategies for whole class
• Some children are not making sufficient progress– “Early intervening” for individual children
Recognition and Response Guidelines for Progress Monitoring and Probing
Where: In a variety of natural settings & routinesWho: By Informed caregivers (teachers, parents, teams) What: Collection of multi-sources of data in ALL areas
• Curriculum based assessment aligned with Early Learning Foundations • Observations• Parent Information• Work Samples• Checklists
When: Ongoing assessment of skills in ALL areas Periodic “Probing” of ALL Key Skills Ideally same tool is used for Monitoring and Probing
Results inform curriculum, teaching strategies, planned learning opportunities for children
Guidelines for Selecting Authentic Assessment Tools
Authentic Assessment measures should have the following characteristics:
• Curriculum-based• Designed to be used multiple times• Easy to score• Sensitive to individual differences• Provide information on both level and rate of growth in
key areas of learning• Related to long term learning goals in curriculum• Aligned to Delaware Early Learning Foundations
Ideally, single measure should be used for authentic assessment and probing for key skills
Authentic Assessment Tools for early Childhood
• Curriculum-based tools – (aligned to Early Learning Foundations)– Creative Curriculum Developmental Assessment– High Scope COR– Carolina Curriculum
• Checklists linked to Early Learning Foundations– (Collected from multi-sources of information)– RORS– Work Sampling– Developmental Checklist Birth to Five
Early Childhood in Delaware
• Head Start and Early Childhood Assistance Programs use authentic assessment
• Early Learning Foundations being revised – Will include format for assessing children
• Child Care Licensing Regulations revised– Centers required to assess children – Training on authentic assessment is in planning
• ABCD Grant – Public Health – Screening initiative
ABCD GrantAssuring Better Child Health and Development
• Collaboration with Medicaid, DPH, DE-AAP, Autism Society of DE
• Developmental Screening for all children at well-visits with a standardized screening tool – Ages & Stages, PEDS– 9 mos, 12 mos, 18 mos, 24 mos, 36 mos
• Being piloted in two community practices in DE • Will be replicated statewide with Medicaid policy• Plans include training early childhood providers
Effort by the National Association of State Heath Policy
Recognition and Response Pilot
Pilot States presently implementing R&R:• Connecticut• Arizona• Maryland• Florida
What Can Delaware School Districts Do Now?
• Advocate for QUALITY Early Childhood programs
• Develop RELATIONSHIPS with families and programs in community
• Develop PARTNERSHIPS with community programs – Communicate with programs in community– Suggest checklists aligned to Early Learning
Foundations– Provide training on using assessment checklists
• Use AUTHENTIC ASSESSMENT to monitor children’s progress
• NOT responsible for screening all children from birth in community
Using Recognition and Response in Kindergarten
Kindergarten Screening:• Consider Pre Kindergarten experiences
– Child was in an environment that provided opportunities
– Kindergarten is the first learning opportunity
• Analyze screening results
Activity
At your table:
1. Select one area of learning from
Delaware Early Learning Foundations
2. Observe children in video clip
3. Document learning opportunities you observed in that area of learning
4. Discuss what you observed with the group
Behavior
RTI and Behavior/Mental Health
Do we have to “do RTI” for
behavior and social-emotional
issues???
No!
Does it make any sense to “do RTI” for behavior and social emotional issues?
Yes!
RTI and Behavior/Mental Health
Are we already “doing RTI” for behavior and social emotional issues?
Quite Possibl
y!
RTI and Behavior/Mental Health
• Tier 1:
– Schoolwide teaching of behavioral expectations and social emotional competencies
– Using schoolwide data to identify and address problems
Universal Screening and Behavior/ Mental Health
• What is already happening?
– Monitoring office discipline referrals
– Teacher referral to school-based problem solving team
Universal Screening and Behavior/ Mental Health
Are we ready to increase our attention to universal screening for behavior and social emotional issues?
Maybe!
Universal Screening and Behavior/ Mental Health
• Purpose: to identify youth who have high risk for developing behavioral or mental health problems
• Conducted on a schoolwide basis
• Typically involves several levels of assessment to avoid over- or under-identification of students
Universal Screening and Behavior/ Mental Health
• Multiple Gating Procedures– Gate 1: teacher nomination procedure– Gate 2: teacher rating scale procedure– Gate 3: observation and/or more detailed
rating scales
Identification of students most at-risk
Intervention Planning
Universal Screening and Behavior/ Mental Health
• Self-report procedures– Gate 1: schoolwide screening– Gate 2: follow up interview– Gate 3: diagnostic interview
Universal Screening and Behavior/ Mental Health
• What does progress monitoring look like?
– The same for academic progress– More individualized around specific
behavioral or social-emotional concerns
Daily Report Cards
Tools for progress monitoring:
http://www.jimwrightonline.com/
php/tbrc/tbrc.php
Universal Screening and Behavior/ Mental Health
• Read more about it….
Some Options…..
Data ManagementSystems
DATA MANAGEMENT FREEWARE
http://www.jimwrightonline.com
RTI: Graph Data for Visual Analysis -Charts and graphs transform progress-monitoring data into visual displays. -Time-series graphs in particular are widely used-A positive trend-line demonstrates when the student is responding well to intervention.
I. Excel Graphs Made Easy -Download pre-formatted Excel spreadsheets -Enter data and create time-series graphs for common academic
measures -Designed by Dr. Jim McDougal and student colleagues Karrie Clark
and Jacklyn Wilson from SUNY College at Oswego.
II. Generate Time-Series Graphs On-Line-Enter your student data into the on-line application ChartDog-Create time-series graphs -Plot trend lines
III. Paper Charts-Chart your data by hand-Collection of blank time-series charts that you can download and print
DATA MANAGEMENTCOMMERCIAL
Existing Grading Program & ExcelSoftwareWeb basedHandheld Qualitative Data
Examples:
http://www.edcheckup.com/ (demos)
http://www.wirelessgeneration.com/ (archived web casts)
Universal Screening
Progress Monitoring
Diagnostic Information Outcome / Summative
Data Management System
A Look at the Real World……
Implementation of a Comprehensive
Data PlanPresented by Capital School District
Pam Hererra Colleen RinkerMichele Waite
A Comprehensive Framework for Reading Instruction
Comprehensive Assessment
System
HEADMaking
informed decisions
based on data
Literacy Support Team(LST)
Instructional Support Team(IST)
Whole Body is Tier 1 instruction
High quality, effective instruction for all students
Grade Level Meetings Grade Level Planning
Clothes are Tiers 2 and 3 interventions and they change based on needs
Comprehensive Assessment Matrix for Reading
• Organization of the assessment system• Purposes for assessments• Timelines for assessments and
schedules• Using an Assessment Team• Importance of professional development
related to administering assessments• Role of district in implementation
School Level Implementation
• Literacy Support Team Meetings and connections to IST
• Use of assessment data• Model Lessons• Follow up with teachers• Data notebook for principal• Importance of principal involvement at all
levels
Implementation at the School Level
• Data notebooks for each teacher• Additional assessments and purposes• Diagnostic assessments (chart)• Infrastructure necessary• Grade level meetings and planning
based on data• Professional Development
Your RoleBefore and After
the RTI Initiative
Topics to Consider
• DE RTI Regulations • Schedules • Professional
Development • Accountability • Materials & Resources• Data Collection &
Review
Taking Stock
Conclusions and Evaluations
Vendors