ws:36 determining eligibility for special education in an rti system: advanced applications of...

WS36: Determining Eligibility for Special Education in an RTI System: Advanced Applications of Determining RATE OF IMPROVEMENT

WS:36 Determining Eligibility for Special Education in an RTI System: Advanced Applications of Determining Rate of ImprovementCaitlin S. Flinn, NCSP & Andy E. McCrea, NCSPNASP Annual Convention February 23, 20121Participants willReview the research on interpreting student growth using CBMLearn how to use Excel or Numbers to calculate a rate of improvement (RoI) statisticLearn how student growth fits into the eligibility conversation within an RTI systemLearning Objectives2Assuming participants have at least a basic understanding of:Response to Intervention (RTI) components/frameworkSpecific learning disabilities (SLD)The Individuals with Disabilities Education Act (IDEA)Curriculum-based measurements (CBM)

Todays Perspective3Rate of improvement (RoI) Background and DefinitionsRoI in the Context of an RTI SystemEstablishing a Need for Consistency and for Quantifying RoIGraphing and Calculating RoI for Individual StudentsApplying RoI: Operationalizing Adequate & Inadequate Growth

Workshop Overview4Rate of ImprovementBackground and Definitions5Specific Learning DisabilityInclusionaryExclusionary1. Failure to meet age- or grade-level State standards in one of eight areas: oral expressionlistening comprehensionwritten expressionbasic reading skill reading fluency skillreading comprehension mathematics calculation mathematics problem solving

2. Discrepancy: Pattern of strengths & weaknesses, relative to intellectual ability as defined by a severe discrepancy between intellectual ability and achievement, or relative to age or grade.ORRTI: Lack of progress in response to scientifically based instruction3. Rule out: Vision, hearing, or motor problemsmental retardationemotional disturbancecultural and/or environmental issueslimited English proficiency

4. Rule out lack of instruction by documenting:Appropriate instruction by qualified personnelRepeated assessments

ObservationPA Guidelines, 2008Where RoI Fits into SLD6that is the question!First Define ProgressProgress Monitoring: Continuous progress monitoring of student performance and use of progress monitoring data to determine intervention effectiveness and drive instructional adjustments, and to identify/measure student progress toward instructional and grade-level goals. (PA)

Progress = Rate of Improvement (ROI)

Defining Lack of Progressin response to scientifically based instruction7Growth, progress, learningAlgebraically: slope of a line

Slope: the vertical change over the horizontal change on a Cartesian plane. (x-axis and y-axis graph)

Also called: Rise over runFormula: m = (y2 - y1) / (x2 - x1)Describes the steepness of a line (Gall & Gall, 2007)Rate of Improvement8Finding a students rate of improvement means determining the students learning

What are some ways you are currently using to determine a students learning?Looking at CBM data, are the scores improving?Looking at where the student is performing compared to their aimline (goal) on a graphCreating a line that fits the data points line of best fit, trendlineRate of Improvement9Measures basic skills general outcome measuresTechnically adequate reliable and validRTI4success.org Table of Assessment Tools http://www.rti4success.org/progressMonitoringTools Quick to administerSensitive to growthAlternate forms / repeatableStandardizedRepresented well in educational researchLinked to instruction and interventionWhy use CBM?10Oral Reading FluencyReading ComprehensionEarly Literacy SkillsSpellingWritten Expression Math ComputationMath Concepts and ApplicationsEarly NumeracyBehavior*Skills Measured with RoI1110 data points are a minimum requirement for a reliable trendline (Gall & Gall, 2007)7-8 is minimum for using the Tukey Method (Wright, 1992)8-9 for stable slopes of progress in early writing (McMaster, 2011)Take-away: The more data points the more stable the slope (Christ, 2006; Hintze & Christ, 2004)

How Many Data Points?Is that reasonable and realistic?How does that affect the frequency of administering progress monitoring probes?How does that affect our ability to make instructional decisions for students?

Gall stats textbook for educational researchJim Wright CBM Manual for Teachers http://www.cbmnow.com/documents/cbaManualhand.pdfKristen McMaster, Exceptional Children, http://readperiodicals.com/201101/2251251451.html

12Results Summary

Ted Christs slide see full presentation under NASP 2012 handouts.13Speeches that included visuals, especially in color, improved recall of information (Vogel, Dickson, & Lehman, 1990)Seeing is believing. Useful for communicating large amounts of information quicklyA picture is worth a thousand words.Transcends language barriers (Karwowski, 2006)Responsibility for accurate graphical representations of data

Graphing RoI14To graph data responsibly!

To find the line of best fit with CBM dataSimple linear regressionOrdinary least squares

To quantify RoIUsing a trendline of CBM data, calculate slopeOur proposal for RoI15Rate of ImprovementIn the Context of an RTI System16PA Model www.pattan.net Standards aligned core instructionUniversal screeningInterventions of increasing intensityResearch-based practicesProgress monitoringData analysis teamingParental engagementComponents of RTI17Fuchs & Fuchs (1998)

Hallmark components of Response to InterventionOngoing formative assessmentIdentifying non-responding studentsTreatment fidelity of instruction

Dual discrepancy modelSignificantly below typically performing peers in level and rateDual Discrepancy Model18School Improvement/Comprehensive School ReformResponse to InterventionDual Discrepancy: Level & GrowthRate of Improvement19Classroom Instruction (Content Expectations)Measure Impact (Test)Proficient!Non ProficientContent Need?Basic Skill Need?InterventionProgress Monitor With CBM Rate of ImprovementInterventionProgress MonitorIf CBM is Appropriate MeasureUse Diagnostic Test to Differentiate20RoI for instructional decisions is not a perfect process, but is well-documented and researchedMany sources of error to consider:Standard error of measurement for slope (Christ, 2006)Ben Ditkowsky www.measuredeffects.com Downloads > Monitor with Confidence chartReading passage variability (Ardoin & Christ, 2009)Frequency of progress monitoring (Jenkin, Graff, & Miglioretti, 2009)

Much to be Done!Standard error of slope Ben Ditkowsky at measuredeffects.com21Many sources of error to consider (cont.):Progress monitoring off grade level (Silberglitt & Hintze, 2007)CBM for non-English speaking studentsDifference in growth for benchmarks between fall and spring (Ardoin & Christ, 2008; Christ, Silberglitt, Yeo, & Cormier, 2010; Graney, Missall, & Martinez, 2009; Fien, Park, Smith, & Baker, 2010)Difference in growth depending on initial level of performance (Fien, Park, Smith, & Baker, 2010; Good et. al., 2010, Silberglitt & Hintze, 2007)

Much to be Done!RoI for ELL - Curriculum Based Measurement and Language Proficiency in English Language LearnersElla Farmer, M.Ed.; Laura Swanlund, M.Ed, NCSP; Kathy Pluymert, Ph.D, NCSPCommunity Consolidated School District #15, Palatine ILNASP 201022before adding a trend line, it is important to carefully consider whether the overall pattern in the data is consistent and linear across time, or whether another pattern (nonlinear, curvilinear) better explains the data.Hixson, Christ, & Bradley-Johnson. (2008) Best Practices in the Analysis of Progress Monitoring Data and Decision Making. Best Practices in School Psychology V. 135 (6) 2133-2146.

Typical Growth: Is There Such a Thing?23More growth from fall to winter than winter to spring for benchmarks (3x per year)Christ & Ardoin (2008)Christ, Silberglitt, Yeo, & Cormier (2010)Fien, Park, Smith, & Baker (2010)

More growth from winter to spring than fall to winterGraney, Missall, & Martinez (2009)Typical Growth: Is There Such a Thing?24DIBELS (6th Ed.) ORF NormsFall to WinterWinter to Spring2nd24223rd15184th 13135th 1196th 115

25A snapshot of the last version of DIBELS indicating in bold/underlined the semester with higher growth.Numbers indicate words gained for the semester.AIMSWeb Norms R-CBMBased on 50th PercentileFall to WinterWinter to Spring1st 18312nd25173rd22154th 16135th 17156th 1312

26A snapshot of AIMSWeb R-CBM norms indicating in bold/underlined the semester with higher growth.Numbers indicate words gained for the semester.

Fuchs, Fuchs, Hamlett, Walz, & Germann (1993)Typical weekly growth rates in oral reading fluency and digits correct

Silberglitt & Hintze (2007)Examined weekly growth in R-CBM mediated by level

Shapiro (2008)Described challenging and ambitious goals for rates of improvement

Expected GrowthSilberglitt & Hintze study looked at differences in rates of growth for students who are monitored below grade level. Shapiro Best Practices Vol 527Typical Growth: Example 1

Benchmark ROI=0.88Student SLOPE=2.5Benchmark ROI=1.06Student SLOPE=1.8928Typical Growth: Example 1Whole year example with same data29Looked at Rate of Improvement in small 2nd grade sample

Found differences in RoI when computed for fall and spring:Ave RoI for fall:1.47 WCPMAve RoI for spring:1.21 WCPMTypical Growth: Example 2Unpublished/informally looked at data to see what the trends were.

30Relax instruction after high stakes testing in March/April; a PSSA effect. Depressed initial benchmark scores due to summer break; a rebound effect (Clemens). Instructional variables could explain differences in Graney (2009) and Ardoin (2008) & Christ (in press) results (Silberglitt). Variability within progress monitoring probes (Ardoin & Christ, 2008) (Lent).Typical Growth: Why the Difference Between Semesters?31Fien, Park, Smith, & Baker (2010)Different growth rates depending on beginning levelSilberglitt & Hintze (2007)Differences in growth rates depending on levelLowest and highest deciles had least amount of growth

Typical Growth: Mediated by Level32Good et. al., 2010Growth Rate as Function of Level at BOY (2nd Grade)20th40th60th80thIntensive0 to 50.110.330.560.986 to 150.400.701.051.5316 to 250.951.431.782.20Strategic26 to 341.301.732.062.4335 to 431.501.832.112.5033Good, R. H., Wheeler, C. E., Cummings, K. D., Baker, S. K., Fien, H., & Kameenui, E. J. Rigorous RtI decisions: Normative growth rates for oral reading fluency. NASP Presentation, Chicago March 3, 2010Establishing a Need for ConsistencyAnd for Quantifying RoI34Statistical methods, such as ordinary least square regression can be used to calculate the slope or trend line Visual analysis can also be used to estimate the general pattern of change across time. p 2136Hixson, Christ, & Bradley-Johnson. (2008) Best Practices in the Analysis of Progress Monitoring Data and Decision Making. Best Practices in School Psychology V. 135 (6) 2133-2146.

Multiple Methods for Interpreting Growth35Multiple Methods for Interpreting GrowthQUALITATIVE APPROACHESQUANTITATIVE APPROACHESProfessional Eye Ball ApproachThree Data-Point Decision RuleSplit MiddleStandard Celeration ChartTukey MethodLast Minus FirstTukey Method Plus a statisticSplit Middle Plus a statisticLinear Regression*Next section is in the following format:First slide describes what it is (method/approach)Second slide: what it looks likeDiscuss: Pros/Cons of Method36QualitativeMethods for Interpreting Rate of Improvement37Are the data generally trending in a positive, negative, neutral manner?Where are the data points in relation to the goal or aimline (if available)?Is there variability among the data points?Professional Eye Ball Approach38Professional Eye Ball Approach

39Professional Eye Ball ApproachPROSCONSEasy to use, no calculations involvedMay lead to interesting discussions

Fairly subjectiveMay lead to interesting discussions because there are multiple interpretations of the same data40Requires an aimlineIf three successive data points lie above the aimline, adjust the aimline upwardIf three successive data points lie below the aimline, adjust the instructional interventionIf three successive data points lie around the aimline, make no changes

(Wright, 1992)Three Data-Point Decision RuleJim Wrights CBM Manual for Teachers41Three Data-Point Decision Rule

42Three Data-Point Decision RulePROSCONSEasy to useRequires only an aimline and three data pointsNo calculations or software needed, can complete by handDoes not provide an RoI statisticDoes not indicate a degree of growthNeed to be good at drawing lines and accurately plotting data!43Developed by Ogden Lindsley, precision teachingEnsures a standardization in the display of dataY-axis: set up on a multiply scale to accommodate behavior frequencies ranging from 1 per day to 1,000 per minuteX-axis: set up on an add scale to accommodate 140 successive calendar days, which is about the equivalent of one school semesterMark multiple academic skills/behaviors on same graphLeave blank any days a skill wasnt measured

(White, 1986, p. 524)Standard Celeration Charthttp://psych.athabascau.ca/html/387/OpenModules/Lindsley/introa1.shtml#chart 44Standard Celeration Chart

45Standard Celeration Chart

Annotated with sample data. Errors are decreasing, corrects are increasing.Days absent or not measured are blank.Record Ceiling and Floor are marked.Phase lines included, intervention noted. 46Standard Celeration ChartPROSCONSEasy to useCan measure multiple academic behaviors (errors and corrects)Easy to share with studentsCharts about one semester at a timeNo software or calculations requiredRequires specific graph paper one sheet per studentDoes not provide an RoI statisticDoes not provide a degree of growth47Drawing a line through the two points obtained from the median data values and the median days when the data are divided into two sections. (Shinn, Good, & Stein, 1989)Split the data points into two sections if unequal, draw line on the middle data point.Find the middle/median data point in each section. This gives you the X-value.Figure out the median number of weeks in each section. This gives you the Y-value.Draw a line through those two coordinates.

Split Middle48Split Middle

(6, 63)(15, 83)XX49Split MiddlePROSCONSNo calculations or software neededCan be done fairly easily by handProvides a trendline to compare against an aimline (yes/no for acquisition of skill)Accounts for outliersPossible solution for different RoIs between fall and springDoes not provide an RoI statisticDoes not described degree of growthNeed to have some training in finding the median score and week50Count the number of data points on the graph.Divide the graph into three approximately equal sections.Ignore the middle section and focus on first and third section. Draw an X where the median data point in the first section meets with the median number of weeks in that section. Then do the same for the third section: Draw an X where the median data point meets with the median number of weeks in that section.Draw a line through both Xs, extending to the ends of the graph to see an approximate rate of improvement, or trendline.Tukey Method51Tukey Method

(5, 62)(16, 74)XX52Tukey MethodPROSCONSNo calculations or software needed, can be done fairly easily by handProvides a trendline to compare against and aimline (yes/no for acquisition of skill)Accounts for outliersMay be a solution to account for differences in performance b/t fall and spring RoIIgnores middle 1/3 of dataDoes not provide an RoI statisticDoes not described degree of growthNeed to have some training in finding the median score and week53QuantitativeMethods for Interpreting Rate of Improvement54Iris Centerhttp://iris.peabody.vanderbilt.edu/resources.html

Last data point minus first data point Divided by administration period minus first administration periodRoI = (Y2 Y1) / (X2 X1)RoI = (74 41) / (18 1)33 / 17 = 1.9RoI = 1.9 words gained on average per week

Last Minus First55Last Minus First

Student SLOPE=1.9What if the last data point was the point before (104)?56Last Minus FirstPROSCONSProvides a growth statistic and trendlineCan compare trendline to aimlineEasy to compute, software not necessary, can complete by handDoes not account for all data points, depends only on two data pointsRequires some simple math

57Median point in 2nd section minus median point in 1st sectionDivided by median point in 2nd section minus median point in 1st sectionRoI = (Y2 Y1) / (X2 X1)RoI = (83 63) / (15.5 6.5)20 / 9 = 2.22.2 word correct gained on average per weekSplit Middle Plus58Split Middle Plus

(6, 63)(15, 83)XXStudent SLOPE=2.259Median point in 3rd section minus the median point in 1st sectionDivided by the number of data points minus oneSlope = (74 62) / (16 5)12 / 11 = 1.11.1 words correct gained on average per week

Tukey Method Plus60Tukey Method Plus

(5, 62)(16, 74)XX61Split Middle & Tukey Method Plus a StatisticPROSCONSProvides an RoI statisticProvides a degree of growthCan be compared to aimline or growth of typically performing peers

Tukey plus does not consider all data pointsNo empirical support for adding the statistic to the trendlineRequires some math and knowledge of how to find the median62Used when there is some correlation between two types of data. In this case: words gained (skill) per week (time)Most common type of regression used is least squaresA line of best fit is calculated and drawn through the data pointsThe line of best fit is the line with the minimum amount of error between the data point and the line (vertical deviation)Linear Regressionhttp://www.stat.yale.edu/Courses/1997-98/101/linreg.htm 63Linear Regression

This is what we will teach you to do with your computer.64Linear RegressionPROSCONSConsiders all data pointsProvides an RoI statistic and trendline that can be compared to aimline and RoI of typically performing peersResearchers use it to measure growth of CBM!

Requires software/ computer for calculationsTime consumingNeed several data pointsInfluenced by outlier data points65Need for Consistency

Linear RegressionTukeySplit MiddleLast Minus First66Need for Consistency67MethodRate of ImprovementQualitative Methods?Last Minus First1.9Tukey Method1.1Split Middle2.2Linear Regression2.5Need for Consistency68MethodRoIAfter 18 WeeksQualitative Methods??Last Minus First1.975.2Tukey Method1.160.8Split Middle2.280.6Linear Regression2.586Need for Consistency25 Words69Students daily test scoreswere entered into a computer program. The data analysis program generated slopes of improvement for each level using an Ordinary Least Squares procedure (Hayes, 1973) and the line of best fit. This procedure has been demonstrated to represent CBM achievement data validly within individual treatment phases (Marston, 1988; Shinn, Good, & Stein, in press; Stein, 1987).Shinn, Gleason, & Tindal (1989)Linear Regression70Christ, T. J. (2006). Short-term estimates of growth using curriculum based measurement of oral reading fluency: Estimating standard error of the slope to construct confidence intervals. School Psychology Review, 35, 128-133.Deno, S. L., Fuchs, L. S., Marston, D., & Shin, J. (2001). Using curriculum based measurement to establish growth standards for students with learning disabilities. School Psychology Review, 30, 507-524.Good, R. H. (1990). Forecasting accuracy of slope estimates for reading curriculum based measurement: Empirical evidence. Behavioral Assessment, 12, 179-193.Fuchs, L. S., Fuchs, D., Hamlett, C. L., Walz, L. & Germann, G. (1993). Formative evaluation of academic progress: How much growth can we expect? School Psychology Review, 22, 27-48.Literature Review: RoI and Linear Regression71Jenkins, J. R., Graff, J. J., & Miglioretti, D.L. (2009). Estimating reading growth using intermittent CBM progress monitoring. Exceptional Children, 75, 151-163.Shinn, M. R., Gleason, M. M., & Tindal, G. (1989). Varying the difficulty of testing materials: Implications for curriculum-based measurement. The Journal of Special Education, 23, 223-233.Shinn, M. R., Good, R. H., & Stein, S. (1989). Summarizing trend in student achievement: A comparison of methods. School Psychology Review, 18, 356-370.Literature Review: RoI and Linear Regression72Ease of application Focus on Yes/No to goal acquisition, not degree of growthHow many of us want to calculate OLS Linear Regression formulas (or even remember how)?Why Are There So Many Ways to Demonstrate RoI?73If we are not all using the same model to compute RoI, we continue to have the same problems as past models, where under one approach a student meets SLD criteria, but under a different approach, the student does not. Without a consensus on how to compute RoI, we risk falling short of having technical adequacy within our model.Need for Consistency: Bottom Line74Graphing and Calculating RoIFor Individual Students75Get Out Your Laptops!Open Microsoft ExcelI loveRoI76Graphing RoI for Individual StudentsFall to Winter77In cell A1, type 3rd Grade ORFIn cell A2, type 1st SemesterIn cell A3, type School WeekIn cell A4, type BenchmarkIn cell A5, type Students Name BootsSetting up Your Spreadsheet

78Starting with cell B3, type numbers 1 through 18 going across row 3 (horizontal).Numbers 1 through 18 represent the number of the school week.You will end with week 18 in cell S3.Labeling School Weeks

79Note: You may choose to enter a date for the school week across row 2 for easy identification.We leave out the week of Thanksgiving break and Winter BreakLabeling Dates

80Our example is using DIBELS 6th Ed. 3rd Grade ORF Benchmarks.You would enter the benchmarks for fall and winter of whatever grade level for which you are graphing rate of improvement here.In cell B4, type 77 for the fall benchmark.In cell S4, type 92 for the winter benchmark.Entering Benchmarks

81Enter the following numbers, going across row 5, under the corresponding week numbers.Week 1 41Week 8 62Week 9 63Week 10 75Week 11 64Week 12 80Week 13 83Week 14 83Week 15 56Week 17 104Week 18 74

Entering Student Data

82If a student was not assessed during a certain week, leave that cell blankDo not enter a score of zero (0) if a student wasnt assessed during a certain week. The program will read the 0 as being a score (e.g., zero words correct per minute) and skew your trendline!*CAUTION*

83Highlight cells A4 and A5 through S4 and S5Click Insert from your top rowGraphing the Data

84Find the icon for LineGraphing the Data

85Click the arrow below it to show optionsGraphing the Data

866 graphics appear for 2-D Line graphsChoose Line with MarkersGraphing the Data

87Your graph will appear

Graphing the Data

Here is the default graph.88To change your graph labels, click on your graphYour options appear in the top rowClick on one of the Chart LayoutsGraphing the Data

89Your chosen layout is applied to the graphBy clicking on the labels (Chart Title, etc.) you can edit themY-Axis is words per minuteX-Axis is number of school weeks

Graphing the Data

90Right click (Mac control click) on any of the student data points.From the drop-down menu that appears, click on Add TrendlineGraphing the Trendline

91On that menu, choose LinearTo label your trendline, choose Custom and type in RoI, or Boots ProgressFurther down on that menu, check the box next to Display Equation on ChartGraphing the Trendline

92Click on CloseYour trendline should appear on your graphAn equation will also appear on your graphYou can relocate the trendline by clicking on it and dragging it to a new placeGraphing the Trendline

93You can repeat the same procedure by clicking on one of the benchmark data pointsSuggestion: Label this trendline Typical RoIMove this equation under the firstGraphing Typical RoI

94Y=2.5138x +42.113What does it mean?2.513 is the average words per week the student is gaining based on the given data points42.133 is where the trendline crosses the Y-Axis

Y=0.8824x +76.1180.8824 is the average words gained per week for typically performing peers in 3rd grade for oral reading fluency

Understanding the Equation95Discuss with your neighbor:How is this student progressing?What is the students RoI compared to the typical RoI?Understanding the Graph

96To add additional data points to the graph (e.g., if you are doing ongoing monitoring) once youve already created the graph, simply enter those data in row 5 under the corresponding school week.You dont have to re-create the graph each time you add a data point!Adding More Data Points97The typical RoI can change depending on where (which week) you enter the benchmark scores on your chart.Suggestion: Enter the benchmark scores based on when your school district completes their benchmark administration for the most accurate description of expected student progress.Note98Programming ExcelFirst SemesterCalculating Needed RoICalculating Typical RoICalculating Student RoI99Needed RoIThe rate of improvement needed to close the achievement gapTypical RoIThe rate of improvement of typically performing peers according to the normsStudent RoIThe actual rate of improvement at which the student is achieving based on available data pointsQuick Definitions100In cell T3, type Needed RoIClick on cell T5

In the fx line at the top of the worksheet, type this formula =((S4-B5)/18)Then hit enter/returnCalculating Needed RoI

S4 midyear goal for ORFB5 where the student is starting from, initial benchmark score101Your result in cell T5 should read: 2.833This formula subtracts the students actual beginning of the year (BOY) benchmark from the expected middle of the year (MOY) benchmark, then divides by 18 for the first 18 weeksCalculating Needed RoI

102In cell U3, type Typical RoIClick on cell U4

In the fx line at the top of the sheet, type this formula =SLOPE(B4:S4,B3:S3)

Then hit enterCalculating Typical RoI

On handouts, cross out Benchmark and write Typical103Your result should read: 0.8825This formula considers 18 weeks of growth according to the benchmark data or typical change (growth) expected per week in the target skill. Calculating Typical RoI

104Click on cell U5

In the fx line at the top of your sheet, type this formula =SLOPE(B5:S5,B3:S3)

Then hit enterCalculating Student RoI

105Your result should read: 2.5137This formula considers 18 weeks of student data (as long as you have a few data points) and provides an average growth or change in skill acquisition per week.Calculating Student RoI

106Applying RoIOperationalizing Adequate & Inadequate Growth

107StepsGather the dataGround the data & set goalsInterpret the data108Step 1: Gather DataUniversal Screening Progress Monitoring109Screening Tools Charthttp://www.rti4success.org/screeningTools Progress Monitoring Tools Charthttp://www.rti4success.org/progressMonitoringTools National Center on RtI110Step 2: Ground the Data1) To what will we compare our student growth data?2) How will we set goals?

111Multiple Ways toLook at GrowthFuchs et. al. (1993) Table of Realistic and Ambitious GrowthNeeded Growth Expected Growth & Percent of Expected GrowthGrowth Toward Individual Goal*

*Best Practices in Setting Progress Monitoring Goals for Academic Skill Improvement (Shapiro, 2008)

112Fuchs, Fuchs, Hamlett, Walz, & Germann (1993)Oral Reading Fluency Adequate Response TableRealistic GrowthAmbitious Growth1st 2.03.02nd 1.52.03rd 1.01.54th 0.91.15th 0.50.8113Fuchs, Fuchs, Hamlett, Walz, & Germann (1993)Digit Fluency Adequate Response TableRealistic GrowthAmbitious Growth1st 0.30.52nd 0.30.53rd 0.30.54th 0.751.25th 0.751.2114Needed GrowthDifference between students BOY (or MOY) score and benchmark score at MOY (or EOY).Example: MOY ORF = 10, EOY benchmark is 40, 18 weeks of instruction (40-10/18=1.67). Student must gain 1.67 wcpm per week to make EOY benchmark.115Expected GrowthDifference between two benchmarks.Example: MOY benchmark is 20, EOY benchmark is 40, expected growth (40-20)/18 weeks of instruction = 1.11 wcpm per week.116Tigard-Tualatin School District (www.ttsd.k12.or.us)Looking at Percent of Expected GrowthTier ITier IITier IIIGreater than 150%Between 110% & 150%Possible LDBetween 95% & 110%Likely LDBetween 80% & 95%May Need MoreMay Need MoreLikely LDBelow 80%Needs MoreNeeds MoreLikely LD117From Where Should Benchmarks/Criteria Come?Appears to be a theoretical convergence on use of local criteria (what scores do our students need to have a high probability of proficiency?) when possible. 118If Local Criteria are Not an OptionUse norms that accompany the measure (DIBELS, AIMSweb, etc.).Use national norms.119Making Decisions: Best Practice Research has yet to establish a blue print for grounding student RoI data. At this point, teams should consider multiple comparisons when planning and making decisions.

120Making Decisions: Lessons From the FieldWhen tracking on grade level, consider an RoI that is 100% of expected growth as a minimum requirement, consider an RoI that is at or above the needed as optimal.So, 100% of expected and on par with needed become the limits of the range within a student should be achieving.121Is there an easy way to do all of this?

122

123

124Access to Spreadsheet Templateshttp://rateofimprovement.com/roi/ Click on Downloads Update dates and benchmarks.Enter names and benchmark/progress monitoring data.125What about Students not on Grade Level?

126Determining Instructional LevelIndependent/Instructional/Frustrational Instructional often b/w 40th or 50th percentile and 25th percentile.Frustrational level below the 25th percentile.AIMSweb: Survey Level Assessment (SLA).

127Setting Goals off of Grade Level100% of expected growth not enough.Needed growth only gets to instructional level benchmark, not grade level.Risk of not being ambitious enough.Plenty of ideas, but limited research regarding Best Practice in goal setting off of grade level.128Possible SolutionWeekly probe at instructional level for sensitive indicator of growth.Monthly probes (give 3, not just 1) at grade level to compute RoI.Goal based on grade level growth (more than 100% of expected).129Step 3: Interpreting Growth

130What do we do when we do not get the growth we want?When to make a change in instruction and intervention?When to consider SLD?131When to make a change in instruction and intervention?Enough data points (6 to 10)?Less than 100% of expected growth.Not on track to make benchmark (needed growth).Not on track to reach individual goal.132When to consider SLD?Continued inadequate response despite: Fidelity with Tier I instruction and Tier II/III intervention.Multiple attempts at intervention.Individualized Problem-Solving approach.Evidence of dual discrepancy

133

134Addressing the Much to be DoneMeaningfulness of Curvilinear GrowthNon-CBM dataFuture Directions

135Contact InformationCaitlin S. [email protected]

Andy [email protected]

Web Site: Downloads & Infowww.rateofimprovement.com Thank You!Fall RoI774162637564808383561049274

BenchmarkStudent: SwiperRoI

Fall3rd Grade ORFFirst SemesterSchool Week123456789101112131415161718Needed RoIActual RoIBenchmark77920.8823529412Student: Swiper416263756480838356104742.83333333332.5137651822

Spring RoI92748589698596908410694100110

BenchmarkStudent: SwiperRate of ImprovementExpected RoI

Spring3rd Grade ORFSecond SemesterSchool Week123456789101112131415161718Needed RoIActual RoIBenchmark921101.06Student: Swiper74858969859690841069410021.89

Examples41626375648083835610474

Student: SwiperRoISchool WeekWords Per Minute

Fall RoI774162637564808383561049274



Spring RoI92748589698596908410694100110



Examples41626375648083835610474


Fall RoI774162637564808383561049274



Spring RoI92748589698596908410694100110



Examples41626375648083835610474

Student: SwiperSchool WeekWords Per Minute

Fall RoI774162637564808383561049274



Spring RoI92748589698596908410694100110



Examples41626375648083835610474


Fall RoI774162637564808383561049274



Spring RoI92748589698596908410694100110



Examples41626375648083835610474


Fall RoI774162637564808383561049274



Spring RoI92748589698596908410694100110



Examples41626375648083835610474


Fall RoI774162637564808383561049274



Spring RoI92748589698596908410694100110



Examples41626375648083835610474


Fall RoI774162637564808383561049274



Spring RoI92748589698596908410694100110



Examples41626375648083835610474


Fall RoI774162637564808383561049274



Spring RoI92748589698596908410694100110



Examples41626375648083835610474


Fall RoI774162637564808383561049274



Spring RoI92748589698596908410694100110



Examples41626375648083835610474


Fall RoI774162637564808383561049274



Spring RoI92748589698596908410694100110



Examples41626375648083835610474


2nd Reading Sept to JanOral Reading Fluency09/11/0909/18/0909/25/0910/02/0910/09/0910/16/0910/23/0910/30/0911/06/0911/13/0911/20/0911/27/0912/04/0912/11/0912/18/0901/01/1001/08/1001/15/10Needed RoI*Actual RoI**% of Expected RoI123456789101112131415161718Benchmark44681.413.780.000%3.780.000%3.780.000%3.780.000%3.780.000%3.780.000%3.780.000%3.780.000%3.780.000%3.780.000%3.780.000%3.780.000%3.780.000%3.780.000%3.780.000%3.780.000%3.780.000%3.780.000%3.780.000%3.780.000%* Needed RoI based on difference between week 1 score and Benchmark score for week 18 divided by 18 weeksOral Reading Fluency Adequte Response Table** Actual RoI based on linear regression of all data pointsRealistic GrowthAmbitious Growth1st Grade2.03.0Benchmarks based on DIBELS Goals2nd Grade1.52.03rd Grade1.01.5Expected RoI at Benchmark Level4th Grade0.91.15th Grade0.50.8(Fuchs, Fuchs, Hamlett, Walz, & Germann 1993)

&C&14Second Grade Reading: September to January&RRate of Improvement Spreadsheet Flinn and McCrea, 2009

2nd Reading Jan to MayOral Reading Fluency01/15/0901/22/0901/29/0902/05/0902/12/0902/19/0902/26/0903/05/0903/12/0903/19/0903/26/0904/02/0904/09/0904/16/0904/23/0904/30/0905/07/0905/14/09Needed RoI*Actual RoI**% of Expected RoI123456789101112131415161718Benchmark68901.29Aiden6140526071951.612.17167%Ava49434977575487922.282.76213%Noah494845696154842.282.01156%Olivia6549577079831.391.50116%Liam5553365470831.941.58122%Hannah595464695260821.721.2093%Gavin6440676884791.441.66129%Grace5348466074792.061.76136%Oliver50444668515157782.221.45112%Peyton6350475875771.501.1287%Josh49384955483667772.281.62125%Riley424954696750762.671.76136%Mason5353506460742.061.1791%Zoe343842685551583.111.44111%Ian413145494730462.720.2419%Faith29363536362945443.390.7558%David30234452431963383.330.7961%Alexa18192533332328374.000.9473%Hunter232324483832343.720.7558%Caroline28202840371925303.440.022%* Needed RoI based on difference between week 1 score and Benchmark score for week 18 divided by 18 weeksOral Reading Fluency Adequate Response Table** Actual RoI based on linear regression of all data pointsRealistic GrowthAmbitious Growth1st Grade2.03.0Benchmarks based on DIBELS Goals2nd Grade1.52.03rd Grade1.01.5Expected RoI at Benchmark Level4th Grade0.91.15th Grade0.50.8(Fuchs, Fuchs, Hamlett, Walz, & Germann 1993)

&C&14Second Grade Reading: January to May&RRate of Improvement Spreadsheet Flinn and McCrea, 2009

2nd Math Sept to JanComputation09/11/0909/18/0909/25/0910/02/0910/09/0910/16/0910/23/0910/30/0911/06/0911/13/0911/20/0911/27/0912/04/0912/11/0912/18/0901/01/1001/08/1001/15/10Needed RoI*Actual RoI**% of Expected RoI12345678910111213141516171850th Percentile12240.7125th Percentile8160.471.330.000%1.330.000%1.330.000%1.330.000%1.330.000%1.330.000%1.330.000%1.330.000%1.330.000%1.330.000%1.330.000%1.330.000%1.330.000%1.330.000%1.330.000%1.330.000%1.330.000%1.330.000%1.330.000%1.330.000%* Needed RoI based on difference between week 1 score and Benchmark score for week 18 divided by 18 weeks** Actual RoI based on linear regression of all data pointsPercentiles based on AIMSweb Growth TablesExpected RoI at 50th PercentileExpected RoI at 25th Percentile

&C&14Second Grade Math: September to January&RRate of Improvement Spreadsheet Flinn and McCrea, 2009

2nd Math Jan to MayComputation01/15/1001/22/1001/29/1002/05/1002/12/1002/19/1002/26/1003/05/1003/12/1003/19/1003/26/1004/02/1004/09/1004/16/1004/23/1004/30/1005/07/1005/14/10Needed RoI*Actual RoI**% of Expected RoI12345678910111213141516171850th Percentile24240.0025th Percentile16170.061.3300%1.330.000%1.3300%1.3300%1.3300%1.3300%1.3300%1.3300%1.3300%1.3300%1.3300%1.3300%1.3300%1.3300%1.3300%1.3300%1.3300%1.3300%1.3300%* Needed RoI based on difference between week 1 score and Benchmark score for week 18 divided by 18 weeks** Actual RoI based on linear regression of all data pointsPercentiles based on AIMSweb Growth TablesExpected RoI at 50th PercentileExpected RoI at 25th Percentile

&C&14Second Grade Math: January to May&RRate of Improvement Spreadsheet Flinn and McCrea, 2009

Sheet11/14/111/121/20111/28/115/14/11Needed RoIActual RoI% of Expected RoI12318Benchmark68901.29Student2227563.781.89147%

Sheet2

Sheet3

2nd Reading Sept to JanOral Reading Fluency09/11/0909/18/0909/25/0910/02/0910/09/0910/16/0910/23/0910/30/0911/06/0911/13/0911/20/0911/27/0912/04/0912/11/0912/18/0901/01/1001/08/1001/15/10Needed RoI*Actual RoI**% of Expected RoI123456789101112131415161718Benchmark44681.413.780.000%3.780.000%3.780.000%3.780.000%3.780.000%3.780.000%3.780.000%3.780.000%3.780.000%3.780.000%3.780.000%3.780.000%3.780.000%3.780.000%3.780.000%3.780.000%3.780.000%3.780.000%3.780.000%3.780.000%* Needed RoI based on difference between week 1 score and Benchmark score for week 18 divided by 18 weeksOral Reading Fluency Adequte Response Table** Actual RoI based on linear regression of all data pointsRealistic GrowthAmbitious Growth1st Grade2.03.0Benchmarks based on DIBELS Goals2nd Grade1.52.03rd Grade1.01.5Expected RoI at Benchmark Level4th Grade0.91.15th Grade0.50.8(Fuchs, Fuchs, Hamlett, Walz, & Germann 1993)

&C&14Second Grade Reading: September to January&RRate of Improvement Spreadsheet Flinn and McCrea, 2009

2nd Reading Jan to MayOral Reading Fluency01/15/0901/22/0901/29/0902/05/0902/12/0902/19/0902/26/0903/05/0903/12/0903/19/0903/26/0904/02/0904/09/0904/16/0904/23/0904/30/0905/07/0905/14/09Needed RoI*Actual RoI**% of Expected RoIDual Discrepancy?123456789101112131415161718Benchmark68901.29Aiden6140526071951.612.17167%Keep On TruckinAva49434977575487922.282.76213%Keep On TruckinNoah494845696154842.282.01156%Olivia6549577079831.391.50116%Liam5553365470831.941.58122%Hannah595464695260821.721.2093%Gavin6440676884791.441.66129%Grace5348466074792.061.76136%Oliver50444668515157782.221.45112%Peyton6350475875771.501.1287%Josh49384955483667772.281.62125%Riley424954696750762.671.76136%Mason5353506460742.061.1791%Zoe343842685551583.111.44111%Ian413145494730462.720.2419%BIG PROBLEMSFaith29363536362945443.390.7558%BIG PROBLEMSDavid30234452431963383.330.7961%BIG PROBLEMSAlexa18192533332328374.000.9473%BIG PROBLEMSHunter232324483832343.720.7558%BIG PROBLEMSCaroline28202840371925303.440.022%BIG PROBLEMSMedian1.45* Needed RoI based on difference between week 1 score and Benchmark score for week 18 divided by 18 weeksSD0.65Oral Reading Fluency Adequate Response Table1 SD Below0.80** Actual RoI based on linear regression of all data pointsRealistic GrowthAmbitious GrowthGrowth Criteria1st Grade2.03.0>125%Benchmarks based on DIBELS Goals2nd Grade1.52.085% - 125%3rd Grade1.01.5125%Benchmarks based on DIBELS Goals2nd Grade1.52.085% - 125%3rd Grade1.01.5

ws:36 determining eligibility for special education in an rti system: advanced applications of...

Documents