program effectiveness in garf: where have we been and where do you need to go?

48
Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

Upload: shanon-robbins

Post on 24-Dec-2015

217 views

Category:

Documents


3 download

TRANSCRIPT

Page 1: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

Program Effectiveness in GARF:

Where Have We Been

and

Where Do You Need to Go?

Page 2: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

Literacy Coach’s FocusIn Data Analysis

ProgramEvaluation

Regrouping

Form needs-basedgroups for classroom

instruction

Assign children to interventions

To what extent is my program keepingBenchmark children at benchmark?

Choose instructional emphasis

To what extent is small-group workmoving strategic children to benchmark?

To what extent is my program movingIntensive children to benchmark?

To what extent are classroom effectsapparent?

Page 3: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

Literacy Coach’s FocusIn Data Analysis

Regrouping

Form needs-basedgroups for classroom

instruction

Assign children to interventions

Choose instructional emphasis

Which DIBELS reportsshould I use?

Do you have curriculum materials to accomplish this?

Page 4: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

Literacy Coach’s FocusIn Data Analysis

ProgramEvaluation

To what extent is my program keepingBenchmark children at benchmark?

To what extent is small-group workmoving strategic children to benchmark?

To what extent is my program movingIntensive children to benchmark?

To what extent are classroom effectsapparent?

Page 5: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

Evidence to Consider

DIBELS Summary of Effectiveness Reports, by project and by school

DIBELS Summary of Effectiveness Reports, by Classroom

Information about curriculum and instruction that you gather by school (Implementation Survey) and LCs gather by classroom (observations)

Page 6: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

Summary of Effectiveness Reports

General question: How are the groups moving?

• “Instructional Recommendation” is a combined, weighted score used as a starting point

• Specific subtests are targeted at middle and end of year

• Two reports each grade level each year: Beginning to Middle, Middle to End

Page 7: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

How do you find the data?

• We asked Wireless to generate state-level reports from last year

• When the bridge is finished to the DIBELS site, you can get these reports from the site

In January, you can start to evaluate changes from last year’s trends

Page 8: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

What are they tracking?

Beginning Middle End

K Instructional Recommendation

ISF PSF

1 Instructional Recommendation

NWF ORF

2 Instructional Recommendation

ORF ORF

3 ORF ORF ORF

Page 9: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

And who are they tracking?

BenchmarkChildren

Strategic Children

IntensiveChildren

Page 10: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

What are your expectations?

• What percentage of benchmark children should be benchmark at the next assessment point?

• What percentage of strategic children should be benchmark at the next assessment point?

• What percentage of intensive children should be benchmark at the next assessment point?

Page 11: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

Let’s get some evidenceK N = 8007

1 N = 7823

2 N = 7330

3 N = 7309

Page 12: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

Kindergarten

What are the goals?

How realistic are they?

What obstacles do we face?

What strategies are we using?

Page 13: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

Kindergarten: Fall to Winter

57

41

3

0

10

20

30

40

50

60

70

80

90

100

Starting Fall at Benchmark

BenchmarkStrategicIntensive

Page 14: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

Kindergarten: Winter to Spring

85

13.5

1.50

10

20

30

40

50

60

70

80

90

100

Starting Winter at Benchmark

BenchmarkStrategicIntensive

Page 15: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

Kindergarten: Fall to Winter

34

54

12

0

10

20

30

40

50

60

70

80

90

100

Starting Fall at Strategic

BenchmarkStrategicIntensive

Page 16: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

Kindergarten: Winter to Spring

70

25

6

0

10

20

30

40

50

60

70

80

90

100

Starting Winter at Strategic

BenchmarkStrategicIntensive

Page 17: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

Kindergarten: Fall to Winter

18

52

30

0

10

20

30

40

50

60

70

80

90

100

Starting Fall at Intensive

BenchmarkStrategicIntensive

Page 18: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

Kindergarten: Winter to Spring

3732 31

0

10

20

30

40

50

60

70

80

90

100

Starting Winter at Intensive

BenchmarkStrategicIntensive

Page 19: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

What does that mean?

Page 20: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

First Grade

What are the goals?

How realistic are they?

What obstacles do we face?

What strategies are we using?

Page 21: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

First Grade: Fall to Winter

58

36

6

0

10

20

30

40

50

60

70

80

90

100

Starting Fall at Benchmark

BenchmarkStrategicIntensive

Page 22: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

First Grade: Winter to Spring

87

11

20

10

20

30

40

50

60

70

80

90

100

Starting Winter at Benchmark

BenchmarkStrategicIntensive

Page 23: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

First Grade: Fall to Winter

28

54

18

0

10

20

30

40

50

60

70

80

90

100

Starting Fall at Strategic

BenchmarkStrategicIntensive

Page 24: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

First Grade: Winter to Spring

55

32

13

0

10

20

30

40

50

60

70

80

90

100

Starting Winter at Strategic

BenchmarkStrategicIntensive

Page 25: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

First Grade: Fall to Winter

15

4144

0

10

20

30

40

50

60

70

80

90

100

Starting Fall at Intensive

BenchmarkStrategicIntensive

Page 26: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

First Grade: Winter to Spring

2730

43

0

10

20

30

40

50

60

70

80

90

100

Starting Winter at Intensive

BenchmarkStrategicIntensive

Page 27: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

What does that mean?

Page 28: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

Second Grade

What are the goals?

How realistic are they?

What obstacles do we face?

What strategies are we using?

Page 29: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

Second Grade: Fall to Winter

92

71

0

10

20

30

40

50

60

70

80

90

100

Starting Fall at Benchmark

BenchmarkStrategicIntensive

Page 30: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

Second Grade: Winter to Spring

81

18

10

10

20

30

40

50

60

70

80

90

100

Starting Winter at Benchmark

BenchmarkStrategicIntensive

Page 31: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

Second Grade: Fall to Winter

29

41

30

0

10

20

30

40

50

60

70

80

90

100

Starting Fall at Strategic

BenchmarkStrategicIntensive

Page 32: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

Second Grade: Winter to Spring

16

53

31

0

10

20

30

40

50

60

70

80

90

100

Starting Winter at Strategic

BenchmarkStrategicIntensive

Page 33: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

Second Grade: Fall to Winter

2

10

88

0

10

20

30

40

50

60

70

80

90

100

Starting Fall at Intensive

BenchmarkStrategicIntensive

Page 34: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

Second Grade: Winter to Spring

2

12

86

0

10

20

30

40

50

60

70

80

90

100

Starting Winter at Intensive

BenchmarkStrategicIntensive

Page 35: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

What does that mean?

Page 36: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

Third Grade

What are the goals?

How realistic are they?

What obstacles do we face?

What strategies are we using?

Page 37: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

Third Grade: Fall to Winter

83

16

10

10

20

30

40

50

60

70

80

90

100

Starting Fall at Benchmark

BenchmarkStrategicIntensive

Page 38: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

Third Grade: Winter to Spring

84

16

00

10

20

30

40

50

60

70

80

90

100

Starting Winter at Benchmark

BenchmarkStrategicIntensive

Page 39: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

Third Grade: Fall to Winter

22

58

20

0

10

20

30

40

50

60

70

80

90

100

Starting Fall at Strategic

BenchmarkStrategicIntensive

Page 40: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

Third Grade: Winter to Spring

28

64

8

0

10

20

30

40

50

60

70

80

90

100

Starting Winter at Strategic

BenchmarkStrategicIntensive

Page 41: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

Third Grade: Fall to Winter

1

18

81

0

10

20

30

40

50

60

70

80

90

100

Starting Fall at Intensive

BenchmarkStrategicIntensive

Page 42: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

Third Grade: Winter to Spring

2

30

68

0

10

20

30

40

50

60

70

80

90

100

Starting Winter at Intensive

BenchmarkStrategicIntensive

Page 43: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

Curriculum Design

If you are not yet getting the results you want . . .

Your curriculum design is not yet sufficient OR

Your professional development system is not yet sufficient

How do you find out which? What do you do about it?

Page 44: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

Whole-Class Instruction

• Why are the benchmark groups not entirely stable?

• Is the whole-class instruction explicit? Balance? Systematic? Interactive?

Page 45: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

Needs-based Instruction

• Is this time directed by a “guided reading” model or is it truly needs-based?– Do intensive children get support at the word

level? Is it organized and repetitive?– Do we differentiate between fluency problems

and word-level problems?• Do fluency groups get true fluency work?

– Do teachers manage instruction so that needs-based time is truly interactive and intensive?

Page 46: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

Intervention

• Are intervention programs matched to children’s needs in the cognitive model?

• Should we help coaches to select informal assessments or to use their diagnostic assessments?

Page 47: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

Next Steps: School Level

• Find positive outliers in each region . . . schools where there is significant positive difference between their achievement and the state mean

• Move to the implementation interviews to see what might explain these differences

• Steal great ideas for other, less successful schools

Page 48: Program Effectiveness in GARF: Where Have We Been and Where Do You Need to Go?

Next Steps: Classroom Level

• Find positive outliers in each school . . . Classrooms where there is significant positive difference between their achievement and the school mean

• Move to the LC observations to see what might explain these differences

• Steal great ideas for other, less successful classrooms