oecd reviews of evaluation and assessment in education: sweden - stockholm – 7 december 2011
DESCRIPTION
Deborah Nusche Policy Analyst Education and Training Policy Division OECD Directorate for Education The OECD Review on Evaluation and Assessment Frameworks for Improving School Outcomes, launched in late 2009, is designed to respond to the strong interest in evaluation and assessment issues evident at national and international levels. It will provide a description of design, implementation and use of assessment and evaluation procedures in countries; analyse strengths and weaknesses of different approaches; and provide recommendations for improvement. The Review looks at the various components of assessment and evaluation frameworks that countries use with the objective of improving student outcomes. These include student assessment, teacher appraisal, school evaluation and system evaluation. The analysis focuses on primary and secondary levels of education.TRANSCRIPT
Image: dan / FreeDigitalPhotos.net
Deborah Nusche
Policy AnalystEducation and Training Policy Division
OECD Directorate for Education
Stockholm – 7 December 2011
OECD Reviews of Evaluation and Assessment in Education: SWEDEN
OECD Review on Evaluation and Assessment Frameworks for Improving School Outcomes
●Purpose: To explore how systems of evaluation and assessment can be used to improve the quality, equity and efficiency of school education.
●The Review will:
Synthesise research-based evidence on the impact of evaluation and assessment strategies and disseminate this knowledge among countries.
Identify innovative and successful policy initiatives and practices.
Facilitate exchanges of lessons and dialogue.
Identify policy options for governments to consider.
OECD Review on Evaluation and Assessment Frameworks for Improving School Outcomes (2009-2012)
OECD Review on Evaluation and Assessment Frameworks for Improving School Outcomes: SCOPE
• The Review looks at different elements of evaluation and assessment.
• Student assessment;
• Teacher appraisal;
• School evaluation;
• System evaluation;
• Comprehensive approach: Investigation of each component individually, as well as the coherence of the framework as a whole (including the links between the different components).
Overarching policy question: “How can assessment and evaluation policies work together more effectively to improve student outcomes in primary and secondary schools?”
OECD Review on Evaluation and Assessment Frameworks for Improving School Outcomes: SCOPE
24 countries: Australia, Austria, Belgium (Flemish Community), Belgium (Fr.), Canada, Czech Rep., Denmark, Finland, France, Hungary, Iceland, Ireland, Israel, Korea, Luxembourg, Mexico, Netherlands,
New Zealand, Norway, Poland, Portugal, Slovak Republic, Slovenia and Sweden.,
12 countries: Australia, Belgium (Flemish Community), Chile, Czech Republic, Denmark, Luxembourg, Mexico, New Zealand, Norway, Portugal, Slovak Republic
and Sweden.
●Outputs:●24 Country Background Reports: 9
are completed, more to come soon.●12 Country Reviews: AUS, DNK, NOR,
SWE completed, more to come soon. ●12 Literature Reviews /conceptual
papers on issues including: formative and summative assessment; teacher evaluation; school evaluation; education standards; use of results.
●Final Comparative Report: to be completed in 2012
OECD Review on Evaluation and Assessment Frameworks for Improving School Outcomes: OUTPUTS
All outputs are available at www.oecd.org/edu/evaluationpolicy
OECD Review on Evaluation and Assessment Frameworks for Improving School Outcomes
The Country Review visit to Sweden (May 2010)
• Sweden: first Country Review of the OECD project
• Preparatory work: data analysis, bringing existing OECD work on Sweden together
• Country background report prepared by Sweden
• Formation of an expert team
• 6 day visit to Sweden
– Meetings with the national authorities and all key stakeholder groups
– Visits to 4 municipalities and 6 schools
– In each school: group meetings with the leadership team, teachers and students
OECD Review on Evaluation and Assessment Frameworks for Improving School Outcomes
System
School
Classroom
Student assessment
Teacher appraisal
School evaluation
System evaluation
Evaluation and assessment framework
Structure of the OECD Review report for Sweden
1. The Evaluation and Assessment Framework
2. Student assessment3. Teacher appraisal4. School evaluation5. System evaluation
The Evaluation and assessment framework
Sweden’s approach to evaluation and assessment is built on high trust system fostering ownership, participation and commitment: Students at the centre: students are part of their own assessment; student feedback is valued
and contributes to teacher appraisal and school evaluation Teacher professionalism: teachers are trusted professionals; strong focus on teacher-based
assessment in the classroom; teachers score national tests ; A strong focus on school self evaluation: conceived as a collaborative process; availability of
self-evaluation tools
…complemented with national goals and feedback mechanisms A focus on providing clear expectations from the centre Working towards a data-rich environment: rich feedback for schools Transparency in monitoring and reporting
Evaluation and assessment framework: Strengths & Challenges
School evaluation
System evaluation
Student assessment
Evaluation and assessment framework: Key elements are well established…
ResponsibilityResponsibilityInstrumentsInstruments
Student assessment
Teachers
With support from NAE
School evaluation
School
Municipality
National Inspectorate
Student assessment
Curriculum, syllabi, criteria
National tests in Years 3,5,9 and upper secondary
National test bank
School evaluation
School self-evaluation tools
Municipal quality reports
Inspection visits & reports
System evaluation
International student assessments
National assessments
Thematic reviews
System evaluation
Ministry
NAE
Inspectorate …BUT more work is necessary to integrate them in a coherent framework
Teacher appraisalTeacher appraisal
School leader
Evaluation and assessment framework
Student assessment
Sweden has a balanced approach to student assessment…
– Broad approach to assessment with a mix of classroom-based assessments and performance-based national assessments
….but lacks an externally validated measure of student performance
– But all types of assessment are corrected and graded by students’ own teachers
– Variability in teachers’ assessment skills– Issues of fairness in student assessment and
grading – Limits to the use of aggregate test scores as
measures of school, municipality and system performance
System
School
Classroom
Student assessment
Teacher appraisal
School evaluation
System evaluation
Individual components: Strengths & Challenges
Teacher appraisal
Swedish teachers are trusted professionals…– Teachers have a high degree of autonomy– Appraisal and salary progression is up to
individual school leaders
…but do not have sufficient opportunities for professional feedback
– Teacher appraisal is not regulated by law– There are no professional standards for
teachers– Little guidance and support to enhance
teacher appraisal – Limited career progression opportunities
System
School
Classroom
Student assessment
Teacher appraisal
School evaluation
System evaluation
Individual components: Strengths & Challenges
School evaluation
There has been strong investment in school evaluation…
– School evaluation builds on a mix of school self-evaluation, municipal school evaluation and national inspections
– A variety of quantitative and qualitative feedback
…but there are large variations in local capacity – Great variability in municipal evaluation and
school self-evaluation approaches – Traditions of professional autonomy– Shift towards greater accountability might
jeopardise the improvement function
System
School
Classroom
Student assessment
Teacher appraisal
School evaluation
System evaluation
Individual components: Strengths & Challenges
System evaluation
Education system evaluation is well established…
– Education system performance is monitored through a broad range of tools
…but data collection and presentation could be further improved
– National data on learning outcomes relies on teacher grading
– Since teacher grading is uneven, the use of grades as measures of municipality and system performance is not appropriate
– Data presentation could be optimised for use by stakeholders
System
School
Classroom
Student assessment
Teacher appraisal
School evaluation
System evaluation
Individual components: Strengths & Challenges
Summary of key issues
– Developing a coherent framework for evaluation and assessment– Building teachers’ assessment capacity and increasing the reliability of
national assessments – Establishing teacher appraisal as an integral part of the evaluation
and assessment framework – Articulating school self-evaluation and external school evaluation– Strengthening education system monitoring
Where to go from here?
5 suggestions
School evaluation
System evaluation
Student assessment
Suggested next steps (1): Establish a strategic plan or framework for E&A
Take stock and map all the existing
elements of E&A and clarify purposes
Take stock and map all the existing
elements of E&A and clarify purposes
Clarify responsibilities and
encourage cooperation
Clarify responsibilities and
encourage cooperation
Follow up with competency
descriptions and training
Follow up with competency
descriptions and training
Identify underdeveloped
elements and missing links (e.g.
how teacher appraisal and
school evaluation can reinforce each
other)
Identify underdeveloped
elements and missing links (e.g.
how teacher appraisal and
school evaluation can reinforce each
other)
Map existing tools and support to
optimise practices
Map existing tools and support to
optimise practices
Teacher appraisal
The improvement of teaching and learning should be at the centre
School evaluation
System evaluation
Schoolprincipal appraisal
Student assessment
Develop a framework of teaching standards
Can help to align key elements of the teaching profession such as initial
training, registration, professional development, regular appraisal and career
advancement
Develop a framework of teaching standards
Can help to align key elements of the teaching profession such as initial
training, registration, professional development, regular appraisal and career
advancement
Articulate teacher appraisal and school evaluation
Articulate teacher appraisal and school evaluation
Strengthen developmental
appraisal-Internal school process
-Linked with PD and school development
Strengthen developmental
appraisal-Internal school process
-Linked with PD and school development
Suggested next steps (2) : Develop teacher appraisal as part of the E&A framework
Introduce a system of teacher
registration to determine career-
progression
-Registration at key stages in the career-More summative-Stronger external
component
Introduce a system of teacher
registration to determine career-
progression
-Registration at key stages in the career-More summative-Stronger external
component
Teacher appraisal
Hold school principals accountable for teacher appraisal
Hold school principals accountable for teacher appraisal
Teacher appraisalCountry Examples?
Chile:The Good Teaching Framework provides
clear guidance to all aspects of the
teaching profession
Australia:Appraisal gives access to Advanced Skills Teaching
Positions
New Zealand:professional body of
teachers holds the lead role in defining teacher
standards
School evaluation
System evaluation
Student assessment
Suggested next steps (3): Increase the reliability of national assessments
Make the existing tests more reliable
• Initial teacher training
• Professional development• Systematic moderation
Make the existing tests more reliable
• Initial teacher training
• Professional development• Systematic moderation
Consider adaptations in test design
“complex assessments”
Consider adaptations in test design
“complex assessments”
Consider developing a national
monitoring sample survey
Consider developing a national
monitoring sample survey
Teacher appraisal
Increase reliability of national assessmentsCountry Examples?
New Zealand: The National Education
Monitoring Project
Denmark:Computer-based
assessments provide rapid feedback
Queensland, Australia:Moderated
internal assessments
School evaluation
Self evaluationMunicipal evaluation
Inspections
System evaluation
Student assessment
Suggested next steps (4): Further develop internal and external school evaluation
Maintain and reinforce the
improvement focus
Maintain and reinforce the
improvement focus
Learn from successful schools
across SwedenTo identify these, need to develop reliable national
assessment system
Learn from successful schools
across SwedenTo identify these, need to develop reliable national
assessment system
Continue to refine quality indicators for
school evaluation
Continue to refine quality indicators for
school evaluationTeacher appraisal
Further strengthen schools’ self evaluation capacities
Provide additional training resources Continue to focus
inspections on this aspect
Further strengthen schools’ self evaluation capacities
Provide additional training resources Continue to focus
inspections on this aspect
School evaluationCountry Examples?
Hong KongLearning from innovating
schools
New Zealand:collaborative
school evaluation model
Scotland:Web-based resources for
school self-evaluation, basedon the Inspectorate’s Evaluation framework
School evaluation
System evaluation
MunicipalCentral
Student assessment
Suggested next steps (5): Strengthen education system monitoring and mobilise existing data
Further build evaluation capacity
at central and municipal levels
- Knowledge management nationally
- Capacity building at the municipal level
Further build evaluation capacity
at central and municipal levels
- Knowledge management nationally
- Capacity building at the municipal level
Improve mobilisation of existing
information
Improve mobilisation of existing
information
Explore ways to more reliably
monitor education outcomes at the
system level
Explore ways to more reliably
monitor education outcomes at the
system level
Teacher appraisal
Pointers for policy development– Map out overall framework – Include missing components (esp. teacher appraisal)– Ensure articulation between components (e.g. teacher appraisal and
school evaluation; school self-evaluation and external evaluation)– Increase reliability of national assessment system – Further develop internal and external school evaluation– Strengthen national system monitoring
To conclude: completing the evaluation and assessment framework
