new ways to think about framing accountability to your community
TRANSCRIPT
![Page 1: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/1.jpg)
Welcome!
http://www.slideshare.net/JFCronin/data-presentation-for-indiana-superintendents
This presentation can be accessed at:
![Page 2: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/2.jpg)
New ways to think about framing accountability to your community.
John Cronin, Ph.D.Senior Director of Education ResearchNWEA
![Page 3: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/3.jpg)
What you will learn in the next 45 minutes
• A proposed meaning for accountability• The value of transparency.• A different approach to metrics.• Observations based on my review of
available community accountability reports.
![Page 4: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/4.jpg)
Accountability is NOT simply about meeting targets!!
![Page 5: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/5.jpg)
What is accountability?
Accountability is a dialogue between the stakeholders and the leaders of their schools. Part of that dialogue is understanding and aligning the goals and objectives of the parents, the schools, and the larger community. Another part of that dialogue is discussing how your schools are doing in reaching those goals.
![Page 6: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/6.jpg)
What is accountability?To be “answerable” means that accountability is a dialogue between the stakeholders and the leaders of their schools. One part of that dialogue is understanding the goals and objectives of the community. Another part of that dialogue is discussing how your schools are doing in reaching those goals.
The most important part of the dialogue is what you’re doing to improve performance based on this information, which is leading.
![Page 7: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/7.jpg)
The lesson from the Atlanta cheating scandal
![Page 8: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/8.jpg)
Source: Aviv, R (2014, July 21). Wrong Answer. The New Yorker. Retrieved on June 16, 2016 from http://www.newyorker.com/magazine/2014/07/21/wrong-answer
“After more than two thousand interviews, the investigators concluded that forty-four schools had cheated and that a “culture of fear, intimidation and retaliation has infested the district, allowing cheating—at all levels—to go unchecked for years.” They wrote that data had been “used as an abusive and cruel weapon to embarrass and punish.” Several teachers had been told that they had a choice: either make targets or be placed on a Performance Development Plan, which was often a precursor to termination. At one elementary school, during a faculty meeting, a principal forced a teacher whose students had tested poorly to crawl under the table.”
![Page 9: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/9.jpg)
Differences in fall-spring test durations
An illustration of gaming
15%
25%
61%
Mathematics
Spring < Fall Spring = Fall Spring > FallSpring < Fall Spring = Fall Spring > Fall
0.0
1.0
2.0
3.0
4.0
5.0
6.0
Mathematics
Grow
th In
dex
Differences in growth index score based on fall-spring test
durations
![Page 10: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/10.jpg)
The four principles that guide your communications to your audiences.
• Clearly identify who your audience is.
• Understand what that audience wants to know about your schools and their performance.
• Be transparent about how you are performing on these objectives.
• Communicate your strategy to improve performance, and corrective action when strategy fails.
![Page 11: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/11.jpg)
The community-based accountability experiment
![Page 12: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/12.jpg)
A simple process
• Establish processes to involve the community in setting accountability goals as an alternative or complement to ESSA objectives
• Use these objectives to drive local performance.
• Report performance and progress on those goals annually.
![Page 13: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/13.jpg)
The annual report contains two parts, a management letter, and indicators.
![Page 14: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/14.jpg)
The management letter is…Your narrative. It is the centerpiece of the accountability report.
The report on indicators is…Like financials in an annual report. They are a common set of facts (mastered by all leaders) that are the basis for discussing student achievement. They demonstrate transparency and, if well presented, allow readers to make their own judgments
![Page 15: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/15.jpg)
Part 1 - The management letter…
Is an opportunity to show leadership.
It is a conversation with your stakeholders about the performance of the school system. In it you explain your business, your successes and failures, and discuss your intended actions for improving the educational experience of students.
![Page 16: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/16.jpg)
The tone of the management letter
• Should be optimistic• Should be plain yet thoughtful• Should be conversational• Should be realistic in addressing successes and
transparent about problems• Should be effusive in praising of subordinates,
critical of yourself.
![Page 17: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/17.jpg)
The management letter…
Is not simply for the community…
It requires the district’s leader(s) to think seriously about your work and communicate coherently about the state of the district and your strategy. It forces you to reflect and clarify your thinking.
![Page 18: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/18.jpg)
• Be transparent about failures• Have a strategy in place to address it• Accept responsibility and temper expectations• Demonstrate progress
How to be transparent without getting killed
![Page 19: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/19.jpg)
Our bad news from 2014 comes from (Burlington-Northern Santa Fe) and is unrelated to earnings. During the year, BNSF disappointed many of its customers. These shippers depend on us, and service failures can badly hurt their businesses. (transparency about failure)
BNSF is, by far, Berkshire’s most important non-insurance subsidiary and, to improve its performance, we will spend $6 billion on plant and equipment in 2015. That sum is nearly 50% more than any other railroad has spent in a single year and is a truly extraordinary amount, whether compared to revenues, earnings or depreciation charges. (our strategy to address failure)
An illustration of good tone from the Berkshire Hathaway Management Letter
![Page 20: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/20.jpg)
Though weather, which was particularly severe last year, will always cause railroads a variety of operating problems, our responsibility is to do whatever it takes to restore our service to industry-leading levels. That can’t be done overnight: The extensive work required to increase system capacity sometimes disrupts operations while it is underway. (recognizing responsibility and tempering expectations)
Recently, however, our outsized expenditures are beginning to show results. During the last three months, BNSF’s performance metrics have materially improved from last year’s figures. (reporting progress)
From the Berkshire Hathaway Management Letter
![Page 21: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/21.jpg)
One of our commitments is to help more students graduate prepared for college. Advance Placement is a key piece of this strategy and our goals were to increase participation in the program, increase the number of students sitting for AP tests, and improving average scores on AP exams.
Our data show that we are doing well on two of the three. In the past three years we’ve increased the number of students participating in the program by 20% across our high schools, and the average scores on the exams have generally improved. In English for example, the average score improved from 3.5 to 3.7 and In Calculus AB the average score improved from 3.1 to 3.5. That’s impressive progress and all credit goes to Marsha White, who led this initiative and the teachers at Wilson and Jefferson high schools who made this happen. (crediting others for success)
Unfortunately, the storyline isn’t perfect. While more students are taking the courses, the improvement in the number of students sitting for the tests has not kept pace. (Admit failure)
How this would read in a community-based accountability management letter
![Page 22: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/22.jpg)
You know from news reports that the costs of poor performance on the state pension fund have passed to schools. This has stressed our budget and one consequence was that we froze hiring at the high schools. Because of retirements and transfers at some of the high schools, we did not replace AP teachers who retired or transferred and those courses were dropped. I should’ve seen that problem coming (accepting responsibility and tempering expectations).
This year our principals reviewed teacher assignments and are reinstituting courses in areas where we have a teacher with appropriate credentials. Because those teachers will be dealing with new content and larger class sizes, we’re funding summer training for them in both the AP content and in teaching strategies that better enable adaption of instruction with larger student groups (demonstrating progress).
We can’t promise that this will entirely solve the problem, and our pass rate on AP exams may take a hit while our new AP teachers master their assignment. I’ll keep you apprised in my monthly newsletter as to how this is going.
How this would read in a community-based accountability management letter
![Page 23: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/23.jpg)
Observations from the review of data in community-based accountability reports
![Page 24: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/24.jpg)
Everyone is entitled to their opinion, but not to their own facts. Daniel Patrick Moynihan
![Page 25: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/25.jpg)
Your annual report should be your “almanac” when it comes to student achievement facts.
Any member of your leadership team who represents educational data should be thoroughly familiar with it.
In any data discussion, the discussion should begin with agreement on what the facts actually are.
![Page 26: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/26.jpg)
Learnings from reviews of annual reports
![Page 27: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/27.jpg)
Accomplishments! • The reports are data rich!• They are reflect a broader picture of schools than
standardized tests and school report cards.• They report on aspects of education that the
community values.• They provide measurements of performance.
![Page 28: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/28.jpg)
The volume of data in the reports is overwhelming
Suggestion – The management letter should bring focus. Make sure every indicator reflects an outcome that the community values.
The scorecard section of the report can be an appendix.
![Page 29: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/29.jpg)
The reports focus more on “scoring” schools than “informing” stakeholders.
Suggestion – Establish goals rather than categories and report whether schools have “achieved the goal”, and whether they are “improving”, and “accelerating”.
![Page 30: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/30.jpg)
Exemplary (3 points)
Recognized (2 points)
Acceptable (1 point)
Unacceptable(0 points)
Percentage of students reading at or above grade level in third grade
100-90% 80-89% 70-79%
X
Less than 70%
From a community-based accountability report
![Page 31: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/31.jpg)
Our first grade reading rate continues to be below goal and we were alarmed by the declines in reading that we were seeing in 2009 through 2013. Two years ago, we brought in a part-time specialist to help at-risk readers and teachers added more guided reading practice to the schedule. The last two years show we’ve improved, and that improvement is accelerating thanks to the hard work kindergarten and first grade teachers. To get closer to goal this year, we’re paying for teachers to take additional staff development and offering release time for teachers to observe and coach each other as they implement new practices.
How this could be communicated in a management letter?
![Page 32: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/32.jpg)
Four types of metrics:
Achievement – How are students learningGrowth – How much progress do students make within a yearImprovement – Are we improving achievement and/or rate of growth of students over timeAcceleration – Is the rate of change accelerating, decelerating, going negative
![Page 33: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/33.jpg)
The reports are heavy on reporting status and light on reporting trends
Suggestion – Report your key indicators longitudinally and focus on improvement and acceleration/deceleration in those indicators.
![Page 34: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/34.jpg)
![Page 35: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/35.jpg)
![Page 36: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/36.jpg)
2015 2014 2013 2012 2011 2010 200940%
50%
60%
70%
80%
90%
100%
75% 72% 68%60%
51% 48% 50%
HS Graduation Rate – Unacceptable, improving, but improvement is decelerating
![Page 37: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/37.jpg)
Grade Students Athletes % Fine Arts %
9 3276 1487 50% 2256 76%
10 3190 1215 41% 1656 56%
11 2967 932 31% 1233 42%
12 2795 703 24% 1028 35%
Percent of students participating in athletics and fine arts programs
![Page 38: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/38.jpg)
The reports generally do not address issues related to equity.
Suggestion – Identify and include metrics in your report that show how you are addressing the particular needs of at-risk and minority populations and how you are addressing achievement gaps.
![Page 39: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/39.jpg)
Calculus AB 2014 2015
Enrolled % Taking
% 3+ Enrolled % Taking % 3+
All students 347 257 74% 389 300 77%
Enrollment and student performance in AP Calculus AB
![Page 40: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/40.jpg)
Calculus AB 2012 2013
Enrolled
% Taking % 3+ Enrolled % Taking % 3+
All students 347 257 74% 389 300 77%
Minority students
100 40 70% 130 50 70%
Comment – Our goal has to improve both enrollment and minority participation in our AP program and our schools have accomplished that. We’re concerned that while minority participation increased, the proportion of minority students actually testing declined this past year. Our principals reported that cost of the exams has been a barrier for some students, so we’ve committed to paying the cost of the exam for all students identified as eligible for free and reduced lunch.
Enrollment and student performance in AP Calculus AB
![Page 41: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/41.jpg)
The reports report satisfaction but show little interest in the dissatisfied.
Suggestion – Include follow-up questions to illuminate who and what may contribute to dissatisfaction and discuss how you are addressing those issues.
![Page 42: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/42.jpg)
Very Satisfied
Somewhat Satisfied
Dissatisfied Unsure
2014 30% 55% 9% 3%
Survey Question – How satisfied are you with the education that the school system provides your child?
![Page 43: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/43.jpg)
Very Satisfied
Somewhat Satisfied
Dissatisfied Unsure
2014 30% 55% 9% 3%
Under 30k 12% 60% 22% 6%
30-70k 20% 62% 15% 3%
Above 70k 40% 50% 6% 4%
Survey Question – How satisfied are you with the education that the school system provides your child?
![Page 44: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/44.jpg)
Top five sources of dissatisfaction – high schools parents who responded somewhat or dissatisfied
1. Fees for athletic and extracurricular participation (15%)
2. Lack of communication/responsiveness by some teachers (12%)
3. Cutbacks in AP course availability (8%)4. Poor teaching (6%)5. Elimination of jazz choir program (4%)
![Page 45: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/45.jpg)
The reports lack leading indicators that are predictive of success at the next level.
Example – The report will include dropout rates, but lacks reporting on elementary and middle school indicators that are empirically tied to this metric and would be key to prevention.
![Page 46: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/46.jpg)
The reports show little evidence that a poor result leads to some decision to change it.
![Page 47: New ways to think about framing accountability to your community](https://reader035.vdocuments.us/reader035/viewer/2022070512/589d9fd11a28abfb3d8b6549/html5/thumbnails/47.jpg)
Florida District
Highly Effective
Effective Needs Improvement
Developing Unsatisfactory VA Score Florida Ranking
1 44.4% 55.6% 0.0% 0.0% 0.0% 0.39 1092 25.0% 75.0% 0.0% 0.0% 0.0% 0.37 1213 90.9% 9.1% 0.0% 0.0% 0.0% -0.14 28024 60.7% 39.3% 0.0% 0.0% 0.0% -0.14 27975 81.2% 18.8% 0.0% 0.0% 0.0% -0.16 28316 37.3% 54.2% 1.7% 0.0% 6.8% 0.12 8807 81.3% 18.8% 0.0% 0.0% 0.0% 0.22 4028 41.7% 55.6% 1.4% 1.4% 0.0% -0.34 32749 52.2% 47.8% 0.0% 0.0% 0.0% 0.16 66410 27.0% 66.2% 1.4% 0.0% 5.4% 0 176411 7.1% 72.6% 9.5% 10.7% 0.0% -0.08 2445
Teacher Evaluation Ratings in Eleven Florida Schools 2013