program effectiveness in garf: where have we been and where do you need to go?
TRANSCRIPT
Program Effectiveness in GARF:
Where Have We Been
and
Where Do You Need to Go?
Literacy Coach’s FocusIn Data Analysis
ProgramEvaluation
Regrouping
Form needs-basedgroups for classroom
instruction
Assign children to interventions
To what extent is my program keepingBenchmark children at benchmark?
Choose instructional emphasis
To what extent is small-group workmoving strategic children to benchmark?
To what extent is my program movingIntensive children to benchmark?
To what extent are classroom effectsapparent?
Literacy Coach’s FocusIn Data Analysis
Regrouping
Form needs-basedgroups for classroom
instruction
Assign children to interventions
Choose instructional emphasis
Which DIBELS reportsshould I use?
Do you have curriculum materials to accomplish this?
Literacy Coach’s FocusIn Data Analysis
ProgramEvaluation
To what extent is my program keepingBenchmark children at benchmark?
To what extent is small-group workmoving strategic children to benchmark?
To what extent is my program movingIntensive children to benchmark?
To what extent are classroom effectsapparent?
Evidence to Consider
DIBELS Summary of Effectiveness Reports, by project and by school
DIBELS Summary of Effectiveness Reports, by Classroom
Information about curriculum and instruction that you gather by school (Implementation Survey) and LCs gather by classroom (observations)
Summary of Effectiveness Reports
General question: How are the groups moving?
• “Instructional Recommendation” is a combined, weighted score used as a starting point
• Specific subtests are targeted at middle and end of year
• Two reports each grade level each year: Beginning to Middle, Middle to End
How do you find the data?
• We asked Wireless to generate state-level reports from last year
• When the bridge is finished to the DIBELS site, you can get these reports from the site
In January, you can start to evaluate changes from last year’s trends
What are they tracking?
Beginning Middle End
K Instructional Recommendation
ISF PSF
1 Instructional Recommendation
NWF ORF
2 Instructional Recommendation
ORF ORF
3 ORF ORF ORF
And who are they tracking?
BenchmarkChildren
Strategic Children
IntensiveChildren
What are your expectations?
• What percentage of benchmark children should be benchmark at the next assessment point?
• What percentage of strategic children should be benchmark at the next assessment point?
• What percentage of intensive children should be benchmark at the next assessment point?
Let’s get some evidenceK N = 8007
1 N = 7823
2 N = 7330
3 N = 7309
Kindergarten
What are the goals?
How realistic are they?
What obstacles do we face?
What strategies are we using?
Kindergarten: Fall to Winter
57
41
3
0
10
20
30
40
50
60
70
80
90
100
Starting Fall at Benchmark
BenchmarkStrategicIntensive
Kindergarten: Winter to Spring
85
13.5
1.50
10
20
30
40
50
60
70
80
90
100
Starting Winter at Benchmark
BenchmarkStrategicIntensive
Kindergarten: Fall to Winter
34
54
12
0
10
20
30
40
50
60
70
80
90
100
Starting Fall at Strategic
BenchmarkStrategicIntensive
Kindergarten: Winter to Spring
70
25
6
0
10
20
30
40
50
60
70
80
90
100
Starting Winter at Strategic
BenchmarkStrategicIntensive
Kindergarten: Fall to Winter
18
52
30
0
10
20
30
40
50
60
70
80
90
100
Starting Fall at Intensive
BenchmarkStrategicIntensive
Kindergarten: Winter to Spring
3732 31
0
10
20
30
40
50
60
70
80
90
100
Starting Winter at Intensive
BenchmarkStrategicIntensive
What does that mean?
First Grade
What are the goals?
How realistic are they?
What obstacles do we face?
What strategies are we using?
First Grade: Fall to Winter
58
36
6
0
10
20
30
40
50
60
70
80
90
100
Starting Fall at Benchmark
BenchmarkStrategicIntensive
First Grade: Winter to Spring
87
11
20
10
20
30
40
50
60
70
80
90
100
Starting Winter at Benchmark
BenchmarkStrategicIntensive
First Grade: Fall to Winter
28
54
18
0
10
20
30
40
50
60
70
80
90
100
Starting Fall at Strategic
BenchmarkStrategicIntensive
First Grade: Winter to Spring
55
32
13
0
10
20
30
40
50
60
70
80
90
100
Starting Winter at Strategic
BenchmarkStrategicIntensive
First Grade: Fall to Winter
15
4144
0
10
20
30
40
50
60
70
80
90
100
Starting Fall at Intensive
BenchmarkStrategicIntensive
First Grade: Winter to Spring
2730
43
0
10
20
30
40
50
60
70
80
90
100
Starting Winter at Intensive
BenchmarkStrategicIntensive
What does that mean?
Second Grade
What are the goals?
How realistic are they?
What obstacles do we face?
What strategies are we using?
Second Grade: Fall to Winter
92
71
0
10
20
30
40
50
60
70
80
90
100
Starting Fall at Benchmark
BenchmarkStrategicIntensive
Second Grade: Winter to Spring
81
18
10
10
20
30
40
50
60
70
80
90
100
Starting Winter at Benchmark
BenchmarkStrategicIntensive
Second Grade: Fall to Winter
29
41
30
0
10
20
30
40
50
60
70
80
90
100
Starting Fall at Strategic
BenchmarkStrategicIntensive
Second Grade: Winter to Spring
16
53
31
0
10
20
30
40
50
60
70
80
90
100
Starting Winter at Strategic
BenchmarkStrategicIntensive
Second Grade: Fall to Winter
2
10
88
0
10
20
30
40
50
60
70
80
90
100
Starting Fall at Intensive
BenchmarkStrategicIntensive
Second Grade: Winter to Spring
2
12
86
0
10
20
30
40
50
60
70
80
90
100
Starting Winter at Intensive
BenchmarkStrategicIntensive
What does that mean?
Third Grade
What are the goals?
How realistic are they?
What obstacles do we face?
What strategies are we using?
Third Grade: Fall to Winter
83
16
10
10
20
30
40
50
60
70
80
90
100
Starting Fall at Benchmark
BenchmarkStrategicIntensive
Third Grade: Winter to Spring
84
16
00
10
20
30
40
50
60
70
80
90
100
Starting Winter at Benchmark
BenchmarkStrategicIntensive
Third Grade: Fall to Winter
22
58
20
0
10
20
30
40
50
60
70
80
90
100
Starting Fall at Strategic
BenchmarkStrategicIntensive
Third Grade: Winter to Spring
28
64
8
0
10
20
30
40
50
60
70
80
90
100
Starting Winter at Strategic
BenchmarkStrategicIntensive
Third Grade: Fall to Winter
1
18
81
0
10
20
30
40
50
60
70
80
90
100
Starting Fall at Intensive
BenchmarkStrategicIntensive
Third Grade: Winter to Spring
2
30
68
0
10
20
30
40
50
60
70
80
90
100
Starting Winter at Intensive
BenchmarkStrategicIntensive
Curriculum Design
If you are not yet getting the results you want . . .
Your curriculum design is not yet sufficient OR
Your professional development system is not yet sufficient
How do you find out which? What do you do about it?
Whole-Class Instruction
• Why are the benchmark groups not entirely stable?
• Is the whole-class instruction explicit? Balance? Systematic? Interactive?
Needs-based Instruction
• Is this time directed by a “guided reading” model or is it truly needs-based?– Do intensive children get support at the word
level? Is it organized and repetitive?– Do we differentiate between fluency problems
and word-level problems?• Do fluency groups get true fluency work?
– Do teachers manage instruction so that needs-based time is truly interactive and intensive?
Intervention
• Are intervention programs matched to children’s needs in the cognitive model?
• Should we help coaches to select informal assessments or to use their diagnostic assessments?
Next Steps: School Level
• Find positive outliers in each region . . . schools where there is significant positive difference between their achievement and the state mean
• Move to the implementation interviews to see what might explain these differences
• Steal great ideas for other, less successful schools
Next Steps: Classroom Level
• Find positive outliers in each school . . . Classrooms where there is significant positive difference between their achievement and the school mean
• Move to the LC observations to see what might explain these differences
• Steal great ideas for other, less successful classrooms