making self review work for you julie foley & dave shepherd education review office october 2010...
TRANSCRIPT
Making self review work Making self review work for youfor you
Julie Foley & Dave ShepherdJulie Foley & Dave ShepherdEducation Review OfficeEducation Review Office
October 2010October 2010
[email protected]@ero.govt.nz
ERO self review presentationERO self review presentation 22
Talk outlineTalk outline
• Recapping concepts and activities from the 2009 presentation to the CPPA
• Schools’ self review and ERO’s 2010/11 reviews - how they can complement each other.
• Continuing to build evaluation capacity in the sector – ERO’s strategic theme
Evaluation Capacity Evaluation Capacity BuildingBuilding
ERO’s current thinking about ERO’s current thinking about internal and external evaluationinternal and external evaluation
Ko te Tamaiti te Putake o te KaupapaKo te Tamaiti te Putake o te Kaupapa
ERO’s conceptual frameworkERO’s conceptual framework
• Complementary evaluation takes key aspects of internal and external evaluation, and accountability and improvement, and adjusts these to the relevant context. It is informed by other evaluation theories and aims to resolve the conundrum raised by Nevo (2000, p.28) “Everybody seems to hate external evaluation
while nobody trusts internal evaluation”.
44
ERO’s Reviews 2010ERO’s Reviews 2010An evaluation approach that balances external ERO review with school self review according to each school’s circumstances.
55
Schools experiencing difficulties with very limited self review
Schools sustaining high quality
performance and continuous
improvement through effective self review
Schools operating well with established processes of self review
ERO self review presentationERO self review presentation 66
Self review is…Self review is…• Self review involves investigating evidence
about student outcomes and current ways of doing things to find out where improvement is needed. Planning for school improvement requires schools to set goals and targets for better student outcomes and make changes that are necessary to bring about those improvements.
(MoE Planning and Reporting, 2003)
ERO self review presentationERO self review presentation 77
Or…Or…• …the deliberate and on-going process of
finding out how well our practice enhances children’s learning and development. Review allows us to see which aspects of our practice are working well and what we could do better. As a result we can make decisions about what to do to improve. Through review our practice is transformed, and ultimately, our children’s learning benefits. (MoE, ECE, 2006)
ERO self review presentationERO self review presentation 88
Or…Or…
• Kura are most successful when there is an robust process of on-going internal review that leads to further improvement. Effective internal review focuses on the key goals for students and the evidence of the extent to which they are actually being achieved…[It is] rigorous, informed by excellence in evaluation practice, with findings based on evidence…
(ERO, Te Aho Matua, 2008)
ERO self review presentationERO self review presentation 99
Or…Or…• Self evaluation will collect and analyse a combination of
qualitative and quantitative data that helps leaders and teachers develop themselves, their practices, promote high performance, help support student progress and develop intervention programmes from a proactive perspective. (Paphitis, n.d.)
• Self evaluation is a dialogic process, a meeting of hearts and minds, a forging of new ways of doing, vital and continuing because it is the core of a school’s educational life. It is the essence of the learning community, the intelligent school, schools that learn.
(MacBeath, 1994)
ERO self review presentationERO self review presentation 1010
Summary. Self review is:Summary. Self review is:
• Centred around giving all students the most successful educational experiences possible
• Set in a collaborative culture of continuous self reflection and constructive critique
• Self initiated by a desire to seek constant improvement – mostly planned but sometimes spontaneous
• Celebrating and sustaining what is working well while striving to improve what is not
• A willingness to embrace challenge and change• Setting long term goals and mid-term targets and
gathering rigorous and wide-ranging evidence to examine the extent to which these have been achieved
•
ERO self review presentationERO self review presentation 1111
Different types of reviewsDifferent types of reviewsbut all linked to the big picturebut all linked to the big picture
ERO self review presentationERO self review presentation 1212
Balancing accountability and Balancing accountability and improvementimprovement
Complementary evaluation
Eg Assurance statements
Eg External audit
Eg Self review
Eg ERO education review
INTERNAL
EXTERNAL
ACOUNTABILITY IMPROVEMENT
ERO self review presentationERO self review presentation 1313
Turning data into evidence…Turning data into evidence…
• Schools gather lots of data – ‘hard’ (quantitative/ numerical) and ‘soft’; (qualitative/narrative). The raw or aggregated data is only the first level of analysis. It describes ‘what is’ or ‘what is happening’.
• The second level of analysis takes data and turns it into information. It explains ‘what this means’. It allows schools to make key statements or comparisons.
• The third level uses this information as evidence to support judgments (how well/to what extent), to make decisions (if this so then we need to…) and to prioritise (the most compelling need is…).
ERO self review presentationERO self review presentation 1414
On-going review cycles…On-going review cycles…
Considering
Informing
MonitoringImplementing
Organising
ERO self review presentationERO self review presentation 1515
What are we learning from our What are we learning from our workshops?workshops?
Schools do it differently:
ERO self review presentationERO self review presentation 1616
What are we learning from ERO’s good What are we learning from ERO’s good practice in self review case studies?practice in self review case studies?
• Importance of leadership to drive the vision• Importance of ‘buy-in” from all involved• Belief that students can achieve• Belief that teachers can make a difference• Importance of the big picture• Commitment to the long term• Importance of prioritising• Remembering to celebrate and build on successes• Importance of gathering useful data and analysing it
effectively• Using strengths from in and outside the school• There is no one way- each school and community is
unique but we can learn from each other
DiscussionDiscussion
Developing your own definition – Developing your own definition – purpose statement purpose statement •Mapping your review activities – Mapping your review activities – quandrant diagramquandrant diagram•Finding an overall structure that Finding an overall structure that integrates your review activitiesintegrates your review activities•What process works for your school?What process works for your school?
Recapping Recapping
• Thinking about self review
• Tools and strategies suggested
• Feedback – • What did you use?• How well did it work for your school?• What changes have you made as a
result? • Share ideas
ERO self review presentationERO self review presentation 1818
What ERO is finding in schoolsWhat ERO is finding in schools
• Pivotal Questions for internal review
• What is so?
• Why is it so?
• So what?
• Now what?
ERO self review presentationERO self review presentation 1919
Planning reviewsPlanning reviews
• What do we need to review?
• Why?
• How?
ERO self review presentationERO self review presentation 2020
Focus of reviewsFocus of reviews
• Balance needed
• What is on paper / documented
• School practices
• Outcomes for students
ERO self review presentationERO self review presentation 2121
ERO’s reviewsERO’s reviews
What is the place of What is the place of
school’s self reviewschool’s self review
in 2010 and ahead?in 2010 and ahead?
Principles underpinning ERO’s Principles underpinning ERO’s reviewsreviews
2323
Enquiry focused
Responsive
Transparent
Collaborative & constructive
Building evaluation capacity
ERO self review presentationERO self review presentation 2424
What do we know about effective What do we know about effective schools?schools?
• National and international research, eg, BES• ERO’s Evaluation at a Glance: Effective
Schools:– A focus on the learner– Strong leadership within an inclusive culture– Effective teaching– Quality engagement with parents, whānau and
communities– Coherent and consistent policies and practices within
a cycle of continuous self review
ERO self review presentationERO self review presentation 2525
ERO’s differentiated review cycleERO’s differentiated review cycle• Rather than being based on a “one-size-fits-all”
three-yearly cycle, the differentiated review cycle is based on ERO’s confidence that the school is providing a quality education for all students and maintaining an on-going process of continuous self improvement
• Self review is the tool through which schools monitor and sustain this continuous improvement and provide evidence of what they are doing and what is working for their students
Deciding on Future ActionDeciding on Future Action• ERO intends to carry out another review over a
period of one-to-two years:
Building capacity• ERO is likely to carry out the next review in three
years: Developing capacity • ERO is likely to carry out the next review in four-
to-five years: Sustaining capacity – Criteria– Process Trends– Moderation
ERO self review presentationERO self review presentation 2626
ERO’s Reviews 2010ERO’s Reviews 2010An evaluation approach that balances external ERO review with school self review according to each school’s circumstances.
2727
Schools experiencing difficulties with very limited self review
Schools sustaining high quality
performance and continuous
improvement through effective self review
Schools operating well with established processes of self review
Schools operating well with Schools operating well with established self review processesestablished self review processes
2929
Scoping & review design phase•using self review to set the scene•plan the evaluation, balancing the need for first hand evidence with evidence from self review
On site external review phase•Gathering evidence•triangulate data•ongoing & collaborative synthesis
Reporting phase•internal reporting•external reporting
use self review as the basis for external review
use external review to test, affirm, strengthen & broaden self review
Building & using school capacity to:
account for progress & achievement of goals, & generate further priorities for development & reviewSchool self reviewing & reporting to school
community
School self reviewing & reporting to school community
Schools performing very well with Schools performing very well with high quality self review processeshigh quality self review processes
3030
School builds & presents “a case” comprised of valid & significant examples of using self review to:•maintain high levels of student progress & achievement•make sound decisions & build continuous school improvement
External review phase•scope & design review using the school’s “case”•meta-evaluation of school’s self review•external reporting
challenge their own assumptions about the quality & efficacy of self review processes & findings
Using & building school capacity to:
engage with & use external meta- evaluation to enhance self review
School self reviewing & continuously improving & reporting to school community
School self reviewing & continuously improving & reporting to school community
Schools experiencing difficultiesSchools experiencing difficulties
3131
Development & review planning phase•planning the action, evaluation & support to improve student achievement
Implementing phase•implementing development•adapting & modifying (praxis)
Concluding phase•internal reporting•external reviewing•external reporting
identify purposes & goals, & plan what & how to evaluate
systematically implement planning & use data to self review, adapt & improve
Building school capacity to:
account for progress & achievement of goals & generate further priorities for development & review
ERO self review presentationERO self review presentation 3434
ERO’s updated evaluation ERO’s updated evaluation indicatorsindicators
• Six dimensions:– Student learning: engagement, achievement
and progress– Effective teaching– Leading and managing the school– Governing the school– Safe and inclusive school culture– Engaging parents, whānau and communities
Focus on success for MāoriFocus on success for Māori
• Govt – MoE and ERO priority
• Based on clear evidence of disparity
• Expectations outlined in NAGs
• How well is your school promoting / achieving successful outcomes for Māori?
• What does your self review tell you about this evaluative question?
ERO self review presentationERO self review presentation 3535
ERO self review presentationERO self review presentation 3636
Support availableSupport available• From ERO
• Framework for reviews• Evaluation Indicators• Good practice case studies (not yet available)• Evaluation at a Glance: Effective Schools (not yet available)• Board/Whānau Assurance Statement Checklist• National Evaluations• Getting the most out of your ERO review booklets• Workshops and cluster meetings
• From other sources:• MoE self review tools for National Standards• MoE Kiwi Leadership material• MoE governance resource• Commercial tools• Local advisors/consultants• Each other!
ERO self review presentationERO self review presentation 3737
To concludeTo conclude
• Be assured that self review is not new and much of what you already do fits under that umbrella
• Make on-going self review, reflection and action a “state of mind” in your school
• Ensure smaller review cycles feed data into strategic self review in a coherent way
• ERO are not looking for particular ways of doing things but at the quality of the evidence and the robustness of the processes that you use for your decision making
• Ask yourself : “How do you know that you make a difference for every student in your school?”