evaluation practice in the nordic countries: different national traditions or a common approach?...
TRANSCRIPT
Evaluation practice in the Nordic countries: Different national traditions
or a common approach?
Hanne Foss HansenDepartment of Political Science
University of Copenhagen
Structure
• Educational evaluation: Concepts and approaches
• Case 1: Higher education -Brief reviews country by country -Similarities and differences
• Case 2: Primary and secondary education (P/S)
• The effects of all this evaluation? The future?
The concept of evaluation
• Everyday language: Measurement, assessment, judgement
• Evaluation language: ”A careful assessment of the merit and worth of processes, structures, output and outcome of interventions and organizations, intended to play a role in future, practical actions situations”
The concept of educational evaluation
• Testing, student assessment, programme evaluation, personel evaluation, auditing, accreditation, benchmarking, curriculum evaluation and probably even more.
Educational evaluation: Focus on many levels
-Individuals (pupils, students, teachers)-Classrooms/courses-Curriculum/programmes-Organizations (schools, universities)-Fields (all schools in a municipality, all
programmes in a discipline)-The national level (national quality development
and quality assurance systems)-The international level (PISA, EQUIS in the
business school area)
Educational evaluation: Many purposes, many uses
-Documenting
-Controlling
-Learning/improving
-Reforming
-Legitimating
-Symbolizing
Focus today primarely on
• The new forms of evaluation (programme evaluation, auditing, accreditation etc.) not on the classical questions of testing and student assessment
• Meso-evaluation defined as evaluation coupled not only to professional practice but also to educational policy
Higher education I
• Adoption of evaluation in the late 1980´s
• 1992-1999: The Danish Center for Evaluation of Higher Education
• 1999: The center is reorganized into the Danish Evaluation Institute (EVA)
Higher education II
• 1990’s: Programme evaluation • 2002: Accreditation is introduced • 2003: A new university law stresses the
responsibility of the universities themselves to conduct evaluations (EVA unclear role)
• 2004: Auditing is introduced
----• 2005: EVA is made responsible for accreditation
of professional education
Higher education I
• Adoption of evaluation in the late 1960’s and early 1970’s, gaining renewed priority in the mid 1990’s
• 1995 The National Agency for Higher Education (Högskoleverket) is established
Higher education II
• 1999-2002: Auditing is the main task
• 2001-2006: Programme evaluation becomes the main task
• Accreditation is also part of the picture
Higher education I
• Adoption of evaluation in the mid 1990´s
• 1996: The Finnish Higher Education Evaluation Council (Finheec) is etablished
Higher education II
• As law places responsibility for evaluation with the higher educational institutions an important purpose of the council is to help institutions to develop quality assurance and development systems
• The council also initiates evaluations of different types
• Accreditation is important in relation to polytechnics and professional courses
• 2004: Auditing
Higher education I
• Adoption of evaluation in the late 1990´s
• 1998: ”Norgesnettrådet” is established
• 2003: The Norwegian Agency for Quality Assurance in Education is established
Higher education II
• Auditing of all higher educational institutions
• Accreditation of programmes and institutions applying for new programmes and institutional status
• The Ministry of Education initiates evaluations of higher educational reforms (Høgskolereformen, Kvalitetsreformen)
Higher education
• Adoption of evaluation in the mid 1990’s
• 1999: It becomes mandatory for higher educational institutions to develop quality assurance systems
• The Ministry of Education initiates programme evaluation ad hoc
• No formalised accreditation system
Similarities across countries in talk, organisation and focus
• Adoption of meso-evaluation in all countries
• Anchoring evaluation in semi-autonomous organizations specialized in evaluation (not Iceland)
• Educational evaluation is decoupled from evaluation of research
• A turn towards auditing (N, DK, FIN)?
From national imitation to international regulative pressures?
National pressures
International pressures
weak
strong
weak strong
Bologna
1990
2005
Factors explaining convergence
• Public-sector reforms: New Public Management, focus on results and effectiveness
• Internationalization: The Bologna proces and the aim of establishing a European Higher Education Area in 2010
• Networking across agencies at Nordic as well as European level
Differences in institutional processes
• Time span in adoption (from Sweden in the late 1960’s, to Denmark in the late 1980’s and Norway in the late 1990’s)
• Time span in institutionalization (e.g. routinization in Denmark from 1992, in Norway from 2003)
• Norway as the late adopter has constructed the most radical system
Differences in balances between quality development (QD) and
control (C) purposes
• DK: QD more than C (except professional education)
• S: From C more than QD to QD more than C
• FIN: QD more than C (except professional education)
• N: C but also QD• IS: QD more than C
Differences in decision contexts
• From Denmark where there is no direct coupling to sanctioning and rewarding (except in professional education) to Norway where there is a direct coupling to sanctioning and rewarding with Sweden somewhere in between
Differences in evaluation models
- Self-evaluation is an important element in DK, S and FIN but not in N
Differences in composition of evaluation panelsDK N S
Peers + + + + + +
Educational research
- + + +
Educational leadership
- + + -
Students - + +
Other users + + - -
Differences in coordination across individual evaluations
Coordination by:
Denmark Norway
Procedures Strong Strong
Specified criteria
Only used in some evaluations
Strong
Panel members Weak Strong
Board decisions - Strong
Factors explaining divergence
Differences in:
• political-administrative cultures
• strategies in public-sector reforms
• structures and traditions in educational systems
• timing and content of higher educational reforms
P/S education
• Late 1990’s the Ministry of Education introduces a program ”Quality development in public Schooling” (attention and tools )
• 1999: EVA gets responsibility for evalution in P/S
• 2002: A law about transparency and openness makes it compulsory to educational institutions to publish evaluations of the quality of teaching
• 2005: Government proposes to establish a council and an agency for quality development
P/S education
1997: Municipalities have each year to work out written quality reports
2003: The agency for education is split up in the Swedish Agency for Education and an agency for school development
2004-2009: Inspection programme. Inspection reports serve as starting points for improving the quality of municpal schooling.
P/S education
• 2003: A council for educational evaluation is established. The council has to plan and implement external evaluations as well as develop methods and coordinate local evaluation
P/S education
• 2004: The Directorate for Primary and Secondary Education is established. The directorate is responsible for an internet-based quality assessment system ensuring transperency in quality information.
P/S education
• Schools have to do and publish self-evaluations
• Every 5th year The Ministry of Education assesses the evaluation methods used by schools (site-visits)
P/S education: Similarities
• Evaluation adopted in all countries • International studies have put educational
quality and evaluation on the agenda (PISA & TIMMS)
• All countries build national institutional capacity to deal with quality and evaluation (increasing state control)
• Transparency in monitoring is important (strenghtening market forces)
P/S education: Differences
• Balances between quality development and control purposes (S: C control but also QD; DK, N, FIN and IS: more soft approaches)
• ? – Too early to really conclude on the practice of the new agencies
Comparing the two cases
Higher education
-Time span in adoption (from late 1960’s to late 1990’s)
-Policy-driven development
P/S
-Later adoption but no time span
-Problem-driven development (DK, N)
Evaluation practice in the Nordic countries: Different national traditions
or a common approach?
Conclusions• Similarities in talk
• Similarities and important differences in actions
• Evaluation is an elastic concept giving room for national and local constructions
Effects of growth in meso-evaluation I
Two very different ways of thinking:
1) Optimism related to the development of learning organizations and a knowledge society
2) Pessimism related to the development of an audit society based om distrust
Effects of growth in meso-evaluation II
• Are educational institutions transformed into
learning organizations or into ”auditable commodities”?
• Is professional practice part of or de-coupled from evolving evaluation cultures?
Limited empirical knowledge in the Nordic countries
The future
A turn towards:
-Auditing and accreditation?
-Evidence-based professional practice?
-Evidence-based educational policy?