pd 360 impact assessment: impact of pd 360 on student proficiency rates prepared by steven h. shaha,...
TRANSCRIPT
PD 360 Impact Assessment:Impact of PD 360 on
Student Proficiency Rates
Prepared by Steven H. Shaha, PhD, DBASummer 2009
1
PD 360 Impact AssessmentPD 360 Impact AssessmentExecutive SummaryExecutive Summary
• Statistically significant* advantages were verified favoring schools with PD 360 versus District Benchmarks.
• Math (p<.001)
• Reading (p<.001)
2* Statistical significance establishes genuine differences between groups and verifies that impacts were “real” and not merely due to chance and, in this case, due to any pre-existing biases in group differences. The appropriate p-values are included with all differences explained herein.
Reading
3
Note to Reader: To better dramatize the magnitude of the consistently favorable impact of PD 360, graphics included hereafter represent a variety of perspectives and a sampling of different interpretive insights, and not an exhaustive nor uniformly arrayed set of results.
Improvements vs.. District for All School Levels
4
74.0
75.0
76.0
77.0
78.0
79.0
80.0
81.0
06-07 07-08
Mea
n Sc
ores
PD 360 Impact:Closing the Reading Gap
Schools
Districts
06-07 07-08 ChangePercent Change
PD 360 Advantage
Schools 75.2 79.8 4.6 6.95% 71.98% Greater ImprovementDistricts 77.9 80.6 2.7 4.04%
Students in high utilizing schools experienced 72% greater experienced 72% greater improvement improvement than the district benchmarks (p<.001)
Students in high utilizing schools experienced 72% greater experienced 72% greater improvement improvement than the district benchmarks (p<.001)
The performance gap closed The performance gap closed to no significant difference despite beginning at a significantly lower performance level (p<.01)
The performance gap closed The performance gap closed to no significant difference despite beginning at a significantly lower performance level (p<.01)
ReadingReading
74.0
75.0
76.0
77.0
78.0
79.0
80.0
81.0
82.0
06-07 07-08
Mea
n Sc
ore
PD 360 Impact on Elementary Schools:Reading
Schools
Districts
Improvements vs. District for Elementary Schools
5
Elementary Schools
06-07 07-08 ChangePercent Change
PD 360 Advantage
Schools 75.3 79.9 4.6 7.23% 83.96% Greater ImprovementDistricts 78.7 81.3 2.6 3.93%
Elementary school students in high utilizing schools experienced 84% greater experienced 84% greater improvement improvement than the district benchmarks (p<.001)
Elementary school students in high utilizing schools experienced 84% greater experienced 84% greater improvement improvement than the district benchmarks (p<.001)
The performance gap closed The performance gap closed to no significant difference despite beginning at a significantly lower performance level (p<.01)
The performance gap closed The performance gap closed to no significant difference despite beginning at a significantly lower performance level (p<.01)
ReadingReading
74.075.076.077.078.079.080.081.082.083.0
06-07 07-08
Mea
n Sc
ore
PD 360 Impact on Middle Schools:Reading
Schools
Districts
Improvements vs. District for Middle Schools
6
Middle Schools
06-07 07-08 ChangePercent Change
PD 360 Advantage
Schools 76.8 82.2 5.4 8.13% 71.70% Greater ImprovementDistricts 78.3 81.6 3.3 4.73%
Middle school students in high utilizing schools experienced experienced 72% greater improvement 72% greater improvement than the district benchmarks (p<.001)
Middle school students in high utilizing schools experienced experienced 72% greater improvement 72% greater improvement than the district benchmarks (p<.001)
Performance for high utilizing schools flip-flopped with district schools flip-flopped with district performance (p=not significant)
Performance for high utilizing schools flip-flopped with district schools flip-flopped with district performance (p=not significant)
ReadingReading
Improvements vs. District for High Schools
7
72.0
73.0
74.0
75.0
76.0
77.0
78.0
79.0
06-07 07-08
Mea
n Sc
ore
PD 360 Impact on High Schools:Reading
Schools
Districts
High Schools
06-07 07-08 ChangePercent Change
PD 360 Advantage
Schools 73.4 77.1 3.6 5.31% 56.49% Greater ImprovementDistricts 76.2 78.5 2.3 3.40%
High school students in high utilizing schools experienced experienced 56% greater improvement 56% greater improvement than the district benchmarks (p<.001)
High school students in high utilizing schools experienced experienced 56% greater improvement 56% greater improvement than the district benchmarks (p<.001)
The performance gap closed The performance gap closed to no significant difference despite beginning at a significantly lower performance level (p<.01)
The performance gap closed The performance gap closed to no significant difference despite beginning at a significantly lower performance level (p<.01)
ReadingReading
Improvements vs. District for All School Levels
8
Higher performing schools versus lower performing:Percent of Teachers
Registered as Users
Segments Viewed
Higher Performing Schools
62.0 406.0
Lower Performing Schools
53.6 398.0
Advantage 8.4 8.015.6% 2.0%
High performing schools were characterized by 16% more teachers as 16% more teachers as registered users registered users than lower performing schools (p<.001)
High performing schools were characterized by 16% more teachers as 16% more teachers as registered users registered users than lower performing schools (p<.001)
Teachers in high performing schools viewed 2% more segments viewed 2% more segments than in lower performing schools (p<.01)
Teachers in high performing schools viewed 2% more segments viewed 2% more segments than in lower performing schools (p<.01)
ReadingReading
Improvements vs. District for Elementary Schools
9
Higher performing schools versus lower performing:Percent of Teachers
Registered as Users
Avg Minutes Viewed
Higher Performing Schools
57.5 109.8
Lower Performing Schools
56.1 98.8
Advantage 1.4 11.02.4% 11.1%
High performing elementary schools were characterized by 2.4% more 2.4% more teachers as registered users teachers as registered users than lower performing schools (p<.01)
High performing elementary schools were characterized by 2.4% more 2.4% more teachers as registered users teachers as registered users than lower performing schools (p<.01)
Teachers in high performing elementary schools viewed PD 360 11% more viewed PD 360 11% more minutes minutes than in lower performing schools (p<.01)
Teachers in high performing elementary schools viewed PD 360 11% more viewed PD 360 11% more minutes minutes than in lower performing schools (p<.01)
ReadingReading
Improvements vs. District for Middle Schools
10
Higher performing schools versus lower performing:Percent of Teachers
Registered as Users
Segments Viewed
Higher Performing Schools
76.3 505.1
Lower Performing Schools
50.8 346.2
Advantage 25.5 158.950.2% 45.9%
High performing middle schools were characterized by 50% more teachers as 50% more teachers as registered users registered users than lower performing schools (p<.001)
High performing middle schools were characterized by 50% more teachers as 50% more teachers as registered users registered users than lower performing schools (p<.001)
Teachers in high performing middle schools viewed 46% more segments viewed 46% more segments than in lower performing schools (p<.001)
Teachers in high performing middle schools viewed 46% more segments viewed 46% more segments than in lower performing schools (p<.001)
ReadingReading
Improvements vs. District for High Schools
11
Higher performing schools versus lower performing:Percent of Teachers
Registered as Users
Higher Performing Schools 55.4Lower Performing Schools 50.9Advantage 4.5
8.8%
High performing high schools were characterized by 9% more teachers as 9% more teachers as registered users registered users than lower performing schools (p<.01)
High performing high schools were characterized by 9% more teachers as 9% more teachers as registered users registered users than lower performing schools (p<.01)
ReadingReading
Math
12
Note to Reader: To better dramatize the magnitude of the consistently favorable impact of PD 360, graphics included hereafter represent a variety of perspectives and a sampling of different interpretive insights, and not an exhaustive nor uniformly arrayed set of results.
Improvements vs. District for All School Levels
13
MathMath
56.0
58.0
60.0
62.0
64.0
66.0
68.0
70.0
72.0
06-07 07-08
Mea
n Sc
ore
PD 360 Impact:Closing the Math Gap
Schools
Districts
06-07 07-08 ChangePercent Change
PD 360 Advantage
Schools 60.9 67.0 6.1 17.97% 399.29% Greater ImprovementDistricts 68.1 69.8 1.7 3.60%
The performance gap closed The performance gap closed to no significant difference despite beginning at a significantly lower performance level (p<.01)
The performance gap closed The performance gap closed to no significant difference despite beginning at a significantly lower performance level (p<.01)
Students in high utilizing schools experienced 5 times greater experienced 5 times greater improvement - 399% - improvement - 399% - than the district benchmarks (p<.001)
Students in high utilizing schools experienced 5 times greater experienced 5 times greater improvement - 399% - improvement - 399% - than the district benchmarks (p<.001)
Improvements vs. District for Elementary Schools
14
MathMath
58.0
60.0
62.0
64.0
66.0
68.0
70.0
72.0
74.0
06-07 07-08
Mea
n Sc
ore
PD 360 Impact on Elementary Schools:Math
Schools
Districts
Elementary Schools
06-07 07-08 ChangePercent Change
PD 360 Advantage
Schools 62.7 69.4 6.8 13.84% 235.61% Greater ImprovementDistricts 69.1 71.4 2.3 4.13%
The performance gap closed The performance gap closed to no significant difference despite beginning at a significantly lower performance level (p<.01)
The performance gap closed The performance gap closed to no significant difference despite beginning at a significantly lower performance level (p<.01)
Students in high utilizing schools experienced nearly 3 ½ times experienced nearly 3 ½ times greater improvement - 235% - greater improvement - 235% - than the district benchmarks (p<.001)
Students in high utilizing schools experienced nearly 3 ½ times experienced nearly 3 ½ times greater improvement - 235% - greater improvement - 235% - than the district benchmarks (p<.001)
Improvements vs. District for Middle Schools
15
MathMath
58.0
60.0
62.0
64.0
66.0
68.0
70.0
72.0
06-07 07-08
Mea
n Sc
ore
PD 360 Impact on Middle Schools:Math
Schools
Districts
Middle Schools
06-07 07-08 ChangePercent Change
PD 360 Advantage
Schools 63.5 67.9 4.4 9.36% 298.62% Greater ImprovementDistricts 70.4 70.9 0.4 2.35%
The performance gap closed The performance gap closed substantially despite beginning at a significantly lower performance level (p=not significant)
The performance gap closed The performance gap closed substantially despite beginning at a significantly lower performance level (p=not significant)
Students in high utilizing schools experienced 4 times greater experienced 4 times greater improvement - 298% - improvement - 298% - than the district benchmarks (p<.001)
Students in high utilizing schools experienced 4 times greater experienced 4 times greater improvement - 298% - improvement - 298% - than the district benchmarks (p<.001)
54.0
56.0
58.0
60.0
62.0
64.0
66.0
68.0
06-07 07-08
Mea
n Sc
ore
PD 360 Impact on High School:Math
Schools
Districts
Improvements vs. District for High Schools
16
MathMath
High Schools
06-07 07-08 ChangePercent Change
PD 360 Advantage
Schools 54.4 61.6 7.1 36.77% 749.41% Greater ImprovementDistricts 63.2 65.5 2.3 4.33%
The performance gap closed The performance gap closed substantially despite beginning at a significantly lower performance level (p=not significant)
The performance gap closed The performance gap closed substantially despite beginning at a significantly lower performance level (p=not significant)
Students in high utilizing schools experienced nearly 8 ½ times experienced nearly 8 ½ times greater improvement - 749% - greater improvement - 749% - than the district benchmarks (p<.001)
Students in high utilizing schools experienced nearly 8 ½ times experienced nearly 8 ½ times greater improvement - 749% - greater improvement - 749% - than the district benchmarks (p<.001)
Improvements vs. District for All School Levels
17
High performing schools were characterized by 11.7% more teachers as 11.7% more teachers as registered users registered users than lower performing schools (p<.001)
High performing schools were characterized by 11.7% more teachers as 11.7% more teachers as registered users registered users than lower performing schools (p<.001)
Teachers in high performing schools experienced 2.4% more users viewing experienced 2.4% more users viewing segments segments than in lower performing schools (p<.01)
Teachers in high performing schools experienced 2.4% more users viewing experienced 2.4% more users viewing segments segments than in lower performing schools (p<.01)
MathMath
Higher performing schools versus lower performing:Percent of Teachers
Registered as Users
Percent of Users
ViewingHigher Performing Schools
62.1 95.3
Lower Performing Schools
55.6 93.0
Advantage 6.5 2.211.7% 2.4%
Improvements vs. District for Elementary Schools
18
High performing elementary schools were characterized by nearly 20% more nearly 20% more teachers as registered users teachers as registered users than lower performing schools (p<.001)
High performing elementary schools were characterized by nearly 20% more nearly 20% more teachers as registered users teachers as registered users than lower performing schools (p<.001)
Teachers in high performing elementary schools viewed PD 360 47% more viewed PD 360 47% more minutes minutes than in lower performing schools (p<.001)
Teachers in high performing elementary schools viewed PD 360 47% more viewed PD 360 47% more minutes minutes than in lower performing schools (p<.001)
MathMath
Higher performing schools versus lower performing:Percent of Teachers
Registered as Users
Avg Minutes Viewed
Higher Performing Schools
65.0 135.2
Lower Performing Schools
54.3 91.9
Advantage 10.7 43.319.8% 47.1%
Improvements vs. District for Middle Schools
19
High performing middle schools were characterized by 35% more teachers as 35% more teachers as registered users registered users than lower performing schools (p<.001)
High performing middle schools were characterized by 35% more teachers as 35% more teachers as registered users registered users than lower performing schools (p<.001)
Teachers in high performing middle schools viewed 5% more segments viewed 5% more segments than in lower performing schools (p<.01)
Teachers in high performing middle schools viewed 5% more segments viewed 5% more segments than in lower performing schools (p<.01)
MathMath
Higher performing schools versus lower performing:Percent of Teachers
Registered as Users
Segments Viewed
Higher Performing Schools
74.7 437.6
Lower Performing Schools
55.2 416.7
Advantage 19.5 20.935.3% 5.0%
Improvements vs. District for High Schools
20
High performing high schools were characterized by 8% more teacher 8% more teacher viewing viewing than lower performing schools (p<.01)
High performing high schools were characterized by 8% more teacher 8% more teacher viewing viewing than lower performing schools (p<.01)
MathMath
Higher performing schools versus lower performing:Percent of
Users Viewing
Higher Performing Schools 98.1Lower Performing Schools 90.8Advantage 7.3
8.1%