reproductions supplied by edrs are the best that can be made · institution united nations...

DOCUMENT RESUME

ED 475 532 HE 035 769

AUTHOR Campbell, Carolyn; Rozsnyai, Christina

TITLE Quality Assurance and the Development of Course Programmes.Papers on Higher Education.

INSTITUTION United Nations Educational, Scientific, and CulturalOrganization, Bucharest (Romania). European Centre for HigherEducation.

ISBN ISBN-192-9069-171-3

PUB DIXTE 2002-00-00NOTE 'f 218p.; For the four-volume series, see HE 035 767-770.AVAILABLE FROM Carfax Publishing, Taylor & Francis Ltd., Customer Services

Department, 325 Chesnut Street, 8th Floor, Philadelphia, PA19106 ($15). Tel: 800-354-1420 (Toll Free); Fax: 215-625-2940; Web site: http://www.cepes.ro.

PUB TYPE Books (010) -- Reports Descriptive (141)EDRS PRICE EDRS Price MF01/PC09 Plus Postage.DESCRIPTORS Foreign Countries; *Higher Education; *Institutional

Evaluation; Program Evaluation; Quality ControlIDENTIFIERS *Europe; *Quality Assurance

ABSTRACT

This volume is the fourth in a series related to a project tocreate a Regional University Network on Governance and Management of HigherEducation in South East Europe. It is both an essay on and source manual forquality assurance in higher education. Part 1 describes the origins andconcepts of quality assurance systems that have been emerging in highereducation throughout Europe, North America, and elsewhere. Examples arechosen from Western, Central, and Eastern Europe. The question of programevaluation versus institutional evaluation is explored, as is the question ofhaving one or several evaluation and assessment agencies for any one country.Part 2 contains five "annexes" on topical issues, mostly from officialsources, that illustrate how different quality assurance organizationsoperate. Part 3 contains a set of specialized bibliographies on the subjectof quality assurance in higher education. (SLD)

I

Reproductions supplied by EDRS are the best that can be madefrom the original document.

CEPES

I! I

PERMISSION TO REPRODUCE ANDDISSEMINATE THIS MATERIAL HAS

BEEN GRANTED BYN,cr)

kr)tr)N

a) TO TO THE EDUCATIONAL RESOURCESW INFORMATION CENTER (ERIC)1

1

$ 0

A

Ii i I I

a0 .

I. r' '

1 1

, I , II' , I i I

U S DEPARTMENT OF EDUCATIONOffice of Educational Research and Improvement

EDUCATIONAL RESOURCES INFORMATIONCENTER (ERIC)

his document has been reproduced asieceived from the person or organizationoriginating it

Minor changes have been made toImprove reproduction quality

8 Points of view or opinions statea in thisdocument do not necessady representofficial OERI position or policy

UNESCO - CEPES

Authors are responsible for the choice and the presentation of thefacts contained in the Papers on Higher Education and for theopinions expressed therein, which are not necessarily those ofUNESCO and do not commit the Organization.

The designations employed and the presentation of the materialdo not imply the expression of any opinion whatsoever on the partof UNESCO concerning the legal status of any country, territory,city, or of its authorities, or concerning the delimitation of itsfrontiers or boundaries.

BEMCEPES

Papers on Higher Education

Regional University network

on Governance and Management

of Higher Education in South East Europe

Quality Assurance and theDevelopment of Course Programmes

Carolyn Campbell and Christina Rozsnyai

Bucharest2002

Papers on Higher Education

Editor cf the Series:

Leland Conley Barrows

Assistants to the Editor:

Maria-Ana DumitrescuValentina Pislaru

Viorica Popa

ISBN 92-9069-171-3© UNESCO 2002

Contents

Preface 9

PART I. UNDERSTANDING THE PHENOMENON OFQUALITY ASSURANCE 13

Chapter 1. Background The Changing Context ofHigher Education 15

1.1. The Changing Environment for Higher EducationA Driver for the Emergence and Evolution of NationalQuality Assurance Systems 15

Chapter 2. Concepts of and Approaches to Quality 19

2.1. Concepts of Quality 192.2. Approaches to Quality and Standards 222.3. Establishing a Regulatory Framework for Quality:

A Role for Government 242.4. The Purpose of Quality Assurance 252.5. The Scope of quality assurance 262.6. The Cost of Evaluation 272.7. What Makes a "Good" External Quality Assurance

system? 28Chapter 3. Key Terms 31

Chapter 4. Quality Assurance and Programme Designin Europe 35

4.1. Introduction 354.2. Quality Assurance and Programme Design in

Western Europe: The Evolution and Current Stateof Affairs Description and Comparative Analysis 364.2.1. Increased Institutional Autonomy in Return

for Greater Accountability 37

4.2.2. Diversification and Expansion 374.2.3. Need for New Methods of Teaching, Learning,

and Assessment 384.2.4. Internationalization 384.2.5. Concepts of and Approaches to Quality 384.2.6. Purposes of Evaluation 404.2.7. Scope of Evaluation: Institutional or

Programmatic Universities or Colleges 424.2.8. Institutional Evaluation 444.2.9. Thematic Reviews and Reports 464.2.10. One Agency or Several 464.2.11. Evaluation Criteria and Standards 474.2.12. Evaluation Methods and Instruments 484.2.13. The Self-Evaluation Study 484.2.14. External Experts: Selection, Training, and

Briefing 504.2.15. Reports and Judgments 514.2.16. Aspects of Programme Design 524.2.17. Two Cycles of Education 534.2.18. Emerging Trends in Quality Assurance

and the Design of Programmes in WesternEurope 54

4.3. Quality Assurance and Programme Design inthe Central and Eastern European Countries:The Evolution and Current State of AffairsDescription and Comparative Analysis 594.3.1. Methodology 664.3.2. Programme Design 674.3.3. Levels of Study 684.3.4. The Legal Setting for Programme Design 744.3.5. Quality Assurance 774.3.6. Quality Assurance Co-operation between

Central and Eastern European Countries 804.3.7. Expected Trends 82

PART II. ANNEXES 85

Annex 1 Resources Chart from the December 2001INQAAHE Survey 87

Annex 2 Descriptions of Higher Education Institutions andQuality Assurance in Central and Eastern EuropeanCountries 89

2.1. Albania 892.2. Bulgaria 902.3. Croatia 922.4. The Czech Republic 932.5. Estonia 952.6. Hungary 972.7. Latvia 992.8. Lithuania 1012.9. Macedonia 1032.10. Poland 1052.11. Romania 1082.12. The Russian Federation 1092.13. Slovakia 1122.14. Slovenia 1142.15. Ukraine 1162.16. Country Overview of Accreditation and Quality

Assessment Agencies 1182.16.1. Clarification of Terminology 1182.16.2. Schematic Presentation of Accreditation

and Quality Assessment Agencies by Centraland Eastern European country 119

2.16.3. Internet Links for Quality Assurance in theCentral and Eastern European Countries 127

2.16.4. Questionnaire sent to Quality AssuranceAgencies in Fifteen Central and EasternEuropean Countries 128

Annex 3. Glossary 131

Annex 4. Quality Assurance: Aspects of Processes,Approaches, and Networking 137

4.1. Quality Assurance Agencies: Principles, Values,and Process Standards 1384.1.1. FH Conference/FH Council Austria: External

Evaluation of Fachhochschulen in Austria 1384.1.2. Denmark: Danmarks Evalueringsinsitut

(EVA) 1394.1.3. Germany: Accreditation, Certification, and

Quality Assurance Institute (ACQUIN) 1404.1.4. The United Kingdom: The Quality Assurance

Agency for Higher Education (QAA) 1404.1.5. United States of America: CHEA 142

4.2. Quality Assurance: Self-Evaluation Documentsand Reports 1434.2.1. United Kingdom: Guidance to Institutions on

Producing a Self-Evaluation An Example fromthe Draft Handbook on Institutional Audit inEngland 143

4.2.2. Germany: FH Conference/FH Council:Quality System 146

4.3. The Selection, Appointment, and Training of Peers:Some Examples of Current Practice 1504.3.1. Austria: FH Conference/FH Council 1504.3.2. Denmark: Evaluation Group (EVA) 1524.3.3. The United Kingdom: The Quality Assurance 152

4.4. International Benchmarking and Evaluation:The Danish Evaluation Agency (EVA) AnInternational Comparative Evaluation ofProgrammes in Agricultural Science 153

4.5. Approach to Quality Assurance: The Exampleof an Agency in a Central and Eastern EuropeanCountry 157

4.6. Quality Assurance from the Perspective ofa University Network Association: A PolicyStatement of the European University Association(EUA) 161

4.7. Accreditation and Recognition: Some InternationalInitiatives by the Engineering Profession 1674.7.1. The European Federation of National

Engineering Associations (FEANI) 1674.7.2. The Washington Accord 168

4.8. What Do Quality Assurance Networks Do?The European Network for Quality Assurancein Higher Education (ENQHA) 169

Annex 5. The Design of Study Programmes:Establishing External Reference Points for Qualityand Standards 173

5.1. The Development of External Reference Pointsfor Quality and Standards of Study Programmes 1745.1.1. Towards Shared Descriptors for Bachelor's

and Master's Degrees: An International ApproachA Report from the Joint Quality Initiative

Group 1745.2. The Design of Study Programmes 178

5.2.1. The Central Evaluation and AccreditationAgency, Hannover, Germany (ZevA) 179

5.3. Towards Subject Specific Descriptors forQualifications: A National Approach TheQuality Assurance Agency for Higher Educationof the United Kingdom 187

5.4. Tuning Structures in Higher Education: LessonsLearned in how to Achieve Results 189

PART III. REFERENCES, BIBLIOGRAPHIES, ANDWEB RESOURCES 193

1. List of References 193

2. Bibliographies 197

2.1. Central and Eastern Europe: Useful Publications on aCountry by Country Basis 197

2.2. General Readings or Readings Pertinent to all theCentral and Eastern European Countries 205

2.3. Thematic Bibliographies 2072.3.1. The Changing Context of Higher Education:

Trends, Challenges, and Responses 2072.3.2 Governance and Management in Higher

Education 2092.3.3. Income Generation and Entrepreneurship 2102.3.4. The Evaluation of Institutional Performance 2112.3.5. Human Resource Management 2122.3.6. Quality Assurance and Management 2122.3.7. Open and Distance Learning 216

3. Some Useful Websites 217

3.1. Intergovernmental Organizations 2173.2. Non-Governmental Organizations 2183.3. Subject Accreditation/Recognition Agencies

Operating on an International Basis 2183.3.1. Business and Management 2183.3.2. Engineering 219

3.4. Some Other Resources on Aspects of Qualityand/or Education 219

The Authors 221

UNESCO-CEPES Publications 227

Preface

This publication is the fourth realization in a series of fourpublications on aspects of governance in higher education thatare being produced as a partial outcome of the joint UNESCO-CEPES European Commission project to create a RegionalUniversity Network on Governance and Management of HigherEducation in South East Europe. The Programme wasoriginally presented through Table One "Democracy and GoodGovernance" of the Stability Pact for South East Europe aspart of its "quick-start package". It has been developed throughthe Task Force on Education and Youth, Enhanced GrazProcess, a coordinating mechanism for educational co-operation with South East Europe.

The basic assumption of the Programme is that, whenconsidering the overall situation in the countries of the region,education in general higher education in particular shouldplay a key role in supporting the search for sustainable peace,reconciliation, and development of civil society.

Its wider objectives include the following:

integration of the universities and higher educationauthorities of Southeastern Europe into existingEuropean networks;development of higher education policies that are basedon European standards and international best practicein the areas of strategic management, financialmanagement, relations with civil society, and qualityassurance;development of national and institutional capacities andskills in higher education strategic management andpolicy making;stimulation of the establishment and/or consolidation ofnew structures and mechanisms of financialmanagement, based on the principles of universityautonomy and accountability, while encouraging theestablishment of links with civil society and localeconomies.

9

12

10 C. CAMPBELL and C. ROZSNYAI

The anticipated outcomes of the programme are expected toinclude the following: (i) integration of the countries of SouthEastern Europe into the European Higher Education Area asdefined in the Bologna Declaration; (ii) the creation of anetwork of the authorities and institutions involved in highereducation through which good practice in academic governance,policy making, strategic and financial management, andquality assurance in higher education can be exchanged; (iii)strengthened national institutional capacities and skills inregard to strategic management and policy making in highereducation; (iv) the creation of new structures and mechanismsfor financial management, based on the principles of universityautonomy and accountability, while encouraging links withcivil society and local economies.

This volume, that brings the series to a close, is both anextended essay and a source manual on the necessity for goodquality assurance mechanisms in higher education. Itsauthors, Carolyn Campbell of the University of Surrey in theUnited Kingdom, and Christina Rozsnyai of the HungarianAccreditation Committee, are experts in the field.

Part I of the volume describes the origins and the conceptsof the quality assurance systems that have been emerging inhigher education throughout Europe, North America, andelsewhere. Examples are chosen from Western Europe andfrom Central and Eastern Europe. The question of programmeassessment and evaluation versus institutional evaluation isdiscussed as is the question of one or several evaluation andassessment agencies for any one country.

Part II of the volume consists of a number of readings ontopical issues drawn from a variety of mostly official sourcesillustrating how different quality assurance organizations indifferent countries operate. In addition, a glossary of frequentlyused terms is included as well as a set of short descriptions ofthe quality assurance institutions in fifteen Central andEastern European countries.

In addition to listing the references cited in Parts I and II ofthe volume, Part III includes a set of specialized bibliographies

PREFACE 11

on the subject of quality assurance in higher education. Someof the publications in one or more of the bibliographies arefollowed by short summaries. The Websites of the majorquality assurance agencies_ in Europe and America are alsolisted.

In short, this publication will give the reader a verycomprehensive overview of the phenomenon of qualityassurance in higher education. It, along with the othervolumes: No. 1, a survey of governance structures of highereducation in several countries of South East Europe (withsuggestions for improvement), No. 2, an extended essay onstrategic planning and management of higher education, andNo. 3, a study of financial management in higher education,particularly formula funding, provide a comprehensive set ofsuggestions as to how best to approach the global reform ofgovernance in higher education.

The four volumes have been published in the UNESCO-CEPES series, Papers on Higher Education. Our hope is thatthey will contribute strongly to the anticipated goal of creatinga successful Regional University Network of Governance andManagement of Higher Education in South East Europe.

Jan SadlakDirector of UNESCO-CEPES

Part I

Understanding the Phenomenon

of Quality Assurance

Chapter 1

Background - The Changing Contextof Higher Education

Quality assurance in higher education was until relativelyrecently an implicit activity. Arguments that quality could notbe measured but could be recognized by academics when andwhere it existed were prevalent. However, over the last twodecades, a number of factors have combined to challengetraditional views about quality in higher education and how itis assured. These factors have been elaborated by manycommentators individuals and organizations such asUNESCO and the World Bank and have led to the making ofquality assurance in higher education "a central objective ofgovernmental policies and an important steering mechanism inhigher education systems worldwide" (Van Damme, 2002, p. 6).

In the rapidly changing environment of higher education,the maintenance of high quality and standards in educationhas become a major concern for higher education institutionsand governments; thus, the demand for explicit qualityevaluation and assurance processes has increased. The resulthas been the introduction of national quality assurancesystems into many countries and the planned introduction ofsuch systems into other countries.

The challenges facing higher education worldwide includethe following:

the need to assure quality and standards against abackground of substantially increased participation aprocess often referred to as the massification of highereducation. This process accelerated throughout the latterpart of the Twentieth Century as many countries beganto consider that their economic and social future wasdependent, in part, on the availability of quality highereducation for the majority of the population rather than

15

16

16 C. CAMPBELL and C. ROZSIVYAI

for a small elite. However, expansion has not alwaysbeen well planned or controlled;the expansion in student numbers with either constantor declining (public) funding resulting in a lower unit ofresource per student. This position has beencompounded by the inefficient use of available resources.Examples of inefficiencies include overly high staff-student ratios, programme duplication in many smallinstitutions/units with high unit costs, and under-utilized facilities. Such inefficiencies divert resourcesfrom such objectives as quality and access (World Bank,2002);increased demand for accountability in higher educationinstitutions as a result of deregulation and the grantingof increased autonomy in regard to such matters ascurriculum design, the selection of students, and theappointment of staff. However, increased autonomy hasnot always been accompanied either by financialauthority or by improved institutional management andstrategic planning capabilities;the meeting of new expectations in terms of the"employability" of graduates in the knowledge society;the addressing of demands from a variety of stakeholdersfor increased and improved information aboutprogrammes and institutions and about the skills,competencies, and aptitudes which graduates possess;the contribution to the achievement of social andpolitical agendas such as access, inclusion, and equity.

In addition to these factors, recent developments includethe appearance of new providers of tertiary education,sometimes in competition with traditional public highereducation, and new modes of provision, such as on-linelearning, resulting from the information and communicationtechnology revolution. An example of this type of competition isreflected in the "new technologies" and the rise of the"Academies" of Microsoft, Cisco, SAP, etc., that have created a

CONCEPTS OF AND APPROACHES TO QUALITY 17

parallel universe of IT qualifications and standards with globalcoverage (Adelman, 2000).

Quality in higher education is not only a national concernbut has become an international issue through academic,political, and commercial developments associated withglobalization, such as the rise of market forces in tertiaryeducation and the emergence of a global market for skilledprofessionals and graduates. In some countries, the traditionalproviders of higher education are facing competition fromtransnational education providers as well as from theemergence of local commercial providers. Through theinternationalization of higher education national systems,qualifications and individual higher education institutionshave become exposed to the wider world. This exposure hasstimulated a demand for better information and transparencyabout quality and standards in order to attract and retainstudents and staff, both national and international students,and to secure the recognition of qualifications.

Quality assurance is a central thrust in the process ofchange in European higher education following the signing ofthe Bologna Declaration and the Prague Communiqué(<http : / /www. msmt. cz/ summit/ prague>) , and has beenhighlighted as a policy implication in the discussions beingsponsored by the Global Agreement on Trade in Services(GATS) on the further liberalization of the trade in educationservices (<http: / /www.wto.org>).

Chapter 2

Concepts of and Approaches to Quality

2.1. CONCEPTS OF QUALITY

There are many different understandings of the term, quality,often reflecting the interests of different constituencies orstakeholders in higher education. Thus, quality is amultidimensional and often a subjective concept (PHAREMulticountry Handbook, 1998).

Conceptions of quality were categorized by Harvey andGreen (1993) and were elaborated in the PHARE Manual ofQuality Assurance: Procedures and Practices (1998). Theyinclude the following:

Quality as excellence. This definition is considered to bethe traditional academic view that holds as its goal to bethe best.Quality as "zero errors". The idea of "zero errors" isdefined most easily in mass industry in which productspecifications can be established in detail, andstandardized measurements of uniform products canshow conformity to them. As the "products" of highereducation, the graduates, are not expected to beidentical, this view is not always considered to beapplicable to higher education.Quality as :fitness for purpose". This view requires thatthe product or service meet a customer's needs,requirements, or desires. Learners (students) andprospective learners, those who fund higher education,the academic community, government, and society atlarge are to a greater or lesser extent all clients or usersof higher education but may have very different views ofboth "purpose" and "fitness".

A major weakness of the "fitness for purpose" conceptis that it may seem to imply that "anything goes" in

19


higher education so long as a purpose can be formulatedfor it. This weakness is more likely to be exacerbated inlarge and diverse higher education systems in which awide range of "purposes" at institutional level may beidentified by individual institutions, generally throughtheir mission statements, and at more precise academiclevels through the learning outcomes of particularprogrammes. This diversity is often further complicated intransnational and distance education (situations in whicheducational provision crosses borders) as there isfrequently a divergence of national views between"sending" and "receiving" countries as to both "fitness"and "purpose".

By complementing "fitness for purpose" with a notion of"fitness of purpose", an evaluation can consider andchallenge the comprehensiveness and relevance of purposesin order to ensure improvements.

Quality as transformation. This concept focuses firmly onthe learners: the better the higher education institution,the more it achieves the goal of empowering studentswith specific skills, knowledge, and attitudes whichenable them to live and work in the knowledge society.This notion of quality may be particularly appropriatewhen there have been significant changes in the profileof learners, for example, when changes in society orpolitics have enhanced access to higher education forlarge numbers of disadvantaged learners. It is arguedthat the delivery of a transformational quality approachinvolves five key elements (Harvey and Knight, 1996, p.117):

envisioning quality as a transformational processdesigned to enhance the experience of students;a bottom-up approach to continuous improvement;responsiveness and openness as the means of gaininggreater trust;an emphasis on effective action;

PROGRAMME DESIGN 21

external monitoring which is sensitive to internalprocedures (and values).While this notion is popular, it may be difficult to

measure quality as transformation in terms ofintellectual capital (Lomas, 2002).

Quality as threshold. Defining a threshold for qualitymeans setting certain norms and criteria. Anyprogramme, department, or institution, which reachesthese norms and criteria, is deemed to be of quality. Theadvantage of setting a threshold is that it is objective andcertifiable. However, there are arguments that setting athreshold creates uniformity across the higher educationsystem. This argument might well apply if institutionsadopt a "compliance" mentality and only do what issufficient to satisfy the minimum. There are significantdisadvantages to this concept, especially when thecriteria and standards are based on quantitative "input"factors enshrined in law. It cannot readily be adapted tochanging circumstances or to stimulate change andinnovation. In this respect, the "threshold" can mitigateagainst improvement. Neither does it take account of"output" standards, the actual level of achievement bygraduates, the criteria used to assess theseachievements, and how that assessment is verified.Nevertheless, in many European higher educationsystems, a "minimum standards" variant has been usedif only as a starting point in the quest for quality.Quality as value for money. The notion of accountabilityis central to this definition of quality with accountabilitybeing based on the need for restraint in publicexpenditure (Lomas, 2000).Quality as enhancement or improvement This conceptemphasizes the pursuit of continuous improvement andis predicated on the notion that achieving quality iscentral to the academic ethos and that it is academicsthemselves who know best what quality is at any point


in time. Disadvantages of this concept are that it isdifficult to "measure" improvement and that the evidenceof improvement may not be easily discernible to theoutside world.

Some of these concepts of quality still hold true especiallywhen explicit quality assurance and accreditation proceduresare being developed and introduced for the first time either atsystem or at institutional level. But, notions of quality areevolving or merging, either as the result of the changingcontext in which higher education institutions are operating insome countries, or as a result of growing expertise withinhigher education systems and institutions in devising theirown concepts of quality and models of evaluation and qualitymanagement. Mismatches between the requirements of theexternal quality assurance agency and institutionalapproaches to quality can be a cause of tension in relations.

2.2. APPROACHES TO QUALITY AND STANDARDS

Some interest has been shown in the application of qualityprocesses and standards adapted from other sectors, forexample, ISO 9001:2000 standards, to higher educationinstitutions. The International Organization for Standardization(ISO) <http://www.iso.ch/en> is a worldwide federation ofnational standards bodies from 140 countries. Established in1947, it is a non-governmental organization with the missionto promote the development of standardization and relatedactivities in the world with a view to facilitating theinternational exchange of goods and services and to developingco-operation in the spheres of intellectual, scientific,technological, and economic activity. ISO has developedstandards and a process for certifying that organizations haveachieved them.

ISO has been described as requiring organizations "to saywhat they do, do what they say, then prove it to a third party"(Seddon, 2000). Universities have tended to seek thiscertification for aspects of their non-academic services such asconference facilities, such student services as accommodation

22

PROGRAMME DESIGN 23

and careers advice and individual academic departments,rather than for their internal quality monitoring or academicprocesses as a whole. One European Quality AssuranceAgency (Agencia per la Qua litat del Sistema UniversitariaCataluyna (<http://wwvv.agenqua.org>) has sought and gainedISO 9002 Certification for its own processes. A criticism of theISO approach has been of its perceived focus on process ratherthan on outcomes. This criticism is apparently beingaddressed in the new process standards 9001:2000 whichfeature an increased focus on customer satisfaction.

The continuous quality improvement model of theEuropean Foundation for Quality Management (EFQM), the"Excellence" model, is another example of a standard forbusiness and industry which has developed a variant to beapplied to public sector services (<http://www.efqm.org/new_website/>). The model is being used by severaluniversities in the United Kingdom, (<http:/ /www.excellence.shu.ac.uk>), Germany, and Turkey, and the second round ofinstitutional accreditation in Hungary will be based on thismodel adapted to local needs. The impetus to establish EFQMcame from a desire to develop a European framework forquality improvement, based on Total Quality Management(TQM) principles, along the lines of the Malcolm BaldridgeModel in the USA and the Deming Prize in Japan, (see<http: / /www.qpronline.com/baldridge /index. html> for furtherinformation).

The EFQM also supports benchmarking exercises that itviews as a powerful tool in the quest for continuousimprovement. The European Benchmarking Code of Conduct,a guide intended to advance the professionalism andeffectiveness of benchmarking in Europe, is published byEFQM. Higher education institutions in some countries areincreasingly interested in benchmarking. This interest isreflected in the purposes and aims of some of the nationalevaluation agencies (see below) and has been the topic ofrecent workshops organized by the European Network for


Quality Assurance (ENQA see <http: / /wvvw.enqan.net> fordetails).

Developments in new forms of provision in higher educationsuch as on-line or "e"-learning and cross border ortransnational education are prompting the articulation of newstandards and criteria at both national and institutional levelin higher education. Conflicting views exist as to whether ornot the same standards and evaluation approaches can beused for different segments of the tertiary education systemand whether they can be applied to traditional and new formsof higher education provision alike. For those for whom theanswer is yes, the way forward is viewed as a focus on definingacademic standards and qualifications in terms of learningoutcomes and competencies rather than as a focus ontraditional input and process factors. Arguments for embracingthe same standards and qualifications include the desire toaccommodate diversity and to facilitate lifelong learningwithout creating a confusing jungle of separate qualifications.Whatever view is taken, a consensus is emerging about theneed for clearer standards and criteria as well as for betterinformation and disclosure about institutional and studentperformance.

Whatever concept of quality is adopted by a nationalsystem, the evaluation procedures introduced by the externalevaluation agency must match it. The same principle applies atinstitutional level as well, but all institutions within a systemneed not adopt the same approach to quality in any onesystem.

It has recently been observed that, "twenty years ofoperational expertise in quality assurance in higher educationhas not led to a consensus on how the concept of qualityshould be defined, rather the opposite" (Van Damme, 2002).

2.3. ESTABLISHING A REGULATORY FRAMEWORK FORQUALITY: A ROLE FOR GOVERNMENT

The external evaluation of the quality of higher educationprovision began more than a hundred years ago, with the

PROGRAMME DESIGN 25

introduction of accreditation in the United States of America.However, accreditation in the United States was and still isorganized by non-governmental organizations which eitheraccredit institutions (the "regional" accreditors and somenational accreditors) or programmes (the specialized accreditorsand some national accreditors). While accreditation is voluntaryand non-governmental, it is, nevertheless, a requirement foraccess to Federal government funding such as student loans(through recognition of the accreditors by the United StatesDepartment of Education), and may be a requirement for thelicensing of professionals by authorities in individual States.Accreditation has facilitated but not necessarily guaranteedcredit transfer between institutions and also student admissionto graduate programmes. The Council for Higher EducationAccreditation (CHEA <http: / /www.chea.org>) is a non-governmental body which carries out the recognition ofaccreditation agencies in the United States and fosters researchon aspects of accreditation and related quality assurancematters.

The American approach to quality in higher educationcontrasted with early approaches to quality in Europe which,in the main, were based on legislation and government controlover higher education institutions and the design ofprogrammes. However, many European governments havedetermined that traditional academic controls and legislationare not effective in dealing with the range of challenges facedby higher education and have either introduced or facilitatedthe establishment of explicit quality assurance systems. Moredetailed commentary on and analysis of the development ofexternal quality assurance in Western and Central andEastern Europe follows below.

2.4. THE PURPOSE OF QUALITY ASSURANCE

At system level, the purpose of quality assurance has to bedetermined and communicated to higher educationinstitutions. There is continuous debate as to whether theemphasis of external quality assurance should be on

25


accountability or on improvement and how an appropriatebalance between these two purposes might be struck. It mightbe appropriate in certain circumstances, for instance, ifaddressing the rapid growth of unregulated private educationor the introduction of new types of institutions orqualifications, to put an emphasis on accountability andcompliance. However, as institutions develop more effectiveand sophisticated internal quality assurance mechanisms,pressure will grow to move the balance from compliance toimprovement. Moreover, if external requirements, for example,in respect of programme approval and design are very rigid,there may be limited scope to demonstrate improvement andinstitutions may not have the flexibility to respond eitherrapidly or in an innovative manner to new demands.

2.5. THE SCOPE OF QUALITY ASSURANCE

The scope of quality assurance is often determined by theshape and size of the higher education system itself. Thetendency is for a single national agency to evaluate all highereducation provision (both university and non-universityprovision), but there are some notable exceptions of countriesin which there are separate agencies that are responsible fordifferent types of programmes, institutions, regions, orpurposes. These exceptions reflect not only the different stagesof development within higher education systems but alsopolitical and cultural preferences (World Bank, 2002).Definitions of the scope of quality assurance can be narrow orbroad. An example of the latter, that is, a broad scope, mightinclude the following dimensions (Middlehurst, 2001; ENQA):

regulation: including legal frameworks, governance,responsibilities, and accountabilities;educational process: admission/selection of students,registration or enrollment, curriculum design anddelivery, support for learning, assessment of students;curriculum design and content: approval and/oraccreditation levels and standards;

26

PROGRAMME DESIGN 27

learning experience: user protection, student experience,complaints, and appeals;outcomes: qualifications, certificates, transcripts, theDiploma Supplement, transferability, recognition, andvalue.

The respective roles of the national system and institutionsneed to be addressed. While both may have a role in all ofthese dimensions of quality assurance, it may not be of equalweight.

2.6. THE COST OF EVALUATION

Cost may be an important consideration in determining factorssuch as the focus, scope, and purposes of an externalevaluation system. Costs include not only those of the settingup of the agency and of the operation of the external processbut also the hidden costs to institutions. The latter includestaff time in preparing for external evaluation and thecollection and collation of information for the self-evaluationand have to-be taken into account in determining the type andamount of information to be provided by institutions toexternal agencies. The more such information can be drawnfrom that which already exists within institutions or whichwould be of use to institutions for their own internal qualitysystems, the better. Other factors, which might be taken intoaccount in deciding on the focus and scope of evaluation,include:

the number and types of institutions in the nationalsystem;whether or not institutions, individual programmes, orbroader groupings of subject/disciplines are to be thefocus. The costs of either evaluating or accrediting on aprogramme-by-programme basis may be substantial. Itis worth noting that countries such as the Netherlandsand Germany are, or will be, using this basis for theaccreditation of their new Bachelor's and Master'sDegrees. This situation contrasts with that prevailing in


the United States, in which accreditation of individualprogrammes only applies in areas such as business andmanagement, health and medicine, technology andengineering, or for certain types of provision, e.g.,distance learning;the frequency of evaluations a national cycle atprogramme level or on an ad hoc/on demand basis? Thelength of cycles of evaluation tends to vary but is neverless than three years (unless a serious problem has beenidentified) and can extend up to ten years;the remuneration of experts. In many systems, expertsare recruited on a "volunteer" basis and receive onlyreimbursement for expenses for travel and subsistence;however, in others, there may be some kind ofhonorarium or indeed some compensation paid to theexpert's own institution for the time spent in activities asan external expert for the external agency.

These factors all haiie cost implications, but an advantageof smaller systems is that universal visiting atprogramme/discipline level is potentially more affordable thanin large higher education systems.

The means by which the activities of evaluation agencies inEurope are funded vary from receipt of a direct subventionfrom the Ministry of Education to a mixture of subscriptionincomes from higher education institutions and from contractsor evaluation or accreditation fees. However, a common featureis that agencies have autonomy over the use of their funds.

2.7. WHAT MAKES A "GOOD" EXTERNAL QUALITYASSURANCE SYSTEM?

A consensus is emerging as to what constitutes an appropriatesystem of external evaluation, the core process elements ofwhich include:

reliance on autonomous or semi-autonomous agencies.This reliance gives legitimacy to internal review processes(EUA policy statement 2001, in Annex 4.6);

PROGRAMME DESIGN 29

agreement on explicit standards and expectations;a self-evaluation study by the academic department,faculty, or institution;an external review conducted by visiting experts;a written report or recommendations;public reporting of the results;recognition that the evaluation process in itself is at leastas important as the results (El Khawas, 1998).

Other success factors include consultation and dialoguewith stakeholders and higher education institutions by theexternal agency when developing processes and criteria, thatis, before setting a system in action. This type of dialogue isconsidered essential to achieving meaningful outcomes to theevaluation process (Franke, 2002). Programme accreditation/evaluation benefits from clarity and a focus on teaching andlearning processes, practices, and performance (Genis, 2002).

A good system for quality assurance should meet its desiredobjectives without unnecessary bureaucracy or intrusion inthe primary activities of an institution (Middlehurst, 2001).

Chapter 3

Key Terms

The vocabulary of "quality" is often jargon-ridden and arcanewith the same words being used to describe differentprocesses. A glossary of terms is included in Annex 3; however,an abridged version, expanding on certain key terms, follows:Accreditation is defined in many ways. Three examples are:

"Accreditation is a formal, published statement regardingthe quality of an institution or a programme, following acyclical evaluation based on agreed standards" (CRE,now EUA, 2001)."Accreditation is a process of external quality reviewused by higher education to scrutinize colleges,universities, and higher education programmes forquality assurance and quality improvement" (CHEA,2000)."Accreditation is the award of a status. Accreditation as aprocess is generally based on the application ofpredefined standards. It is primarily an outcome ofevaluation" (The European Training Foundation, 1998).

While accreditation has different definitions, forms, andfunctions, it generally has the following as characteristics:

It provides [proof) (or not) that a certain standard isbeing met in a higher education course, programme, orinstitution. The standard met can either be a minimumstandard or a standard of excellence.It involves a benchmarking assessment.Judgments are based solely on quality criteria, never onpolitical characteristics, and are always yes/no.The emphasis is on accountability.

31

30


A benchmark is a reference point or criterion by which tomeasure something.

Benchmarking is a process of identifying and learning fromgood practices in other organizations.

(Subject) benchmark statements represent general expectationsabout standards (levels of student attainment) at a givenlevel in a particular subject area. They are referencepoints in a quality assurance framework rather thanprescriptive statements about curricula. (Examples ofnational subject benchmark statements are available at<http://www.qaa.ac.ulc>, and international developmentsare available at <http: / /www.relint.deusto.es/Tuning>;also, see Annex 5.4.

A binary system is a dual system of higher education with atraditional university sector and a separate and distinctnon-university sector.

Evaluation is a general term denoting any process leading tojudgments and/or recommendations regarding the orquality of a unit. (A unit is an institution, programme,discipline.) Evaluation can be an internal process self-evaluation or an external one conducted by externalexperts, peers, or inspectors.

An institutional audit is concerned with the processes wherebyan institution assures quality assurance and qualityenhancement. An underlying theme in quality audits isthe question: how does an institution know that thestandards and objectives it has set for itself are beingmet? On what evidence is the assessment of the qualityof its work based, and are there procedures in place toensure that the significant processes are followed up andcontinuously improved?

Institutional review addresses the ultimate responsibility forthe management of quality and standards that rests withan institution as a whole.

Licensing is the awarding of permission to operate either a newhigher education institution or a new course programmebased on ex ante evaluation of plans.

KEY TERMS 33

Quality assessment, quality measurement, and review ofquality are all taken here to be synonymous withevaluation, especially when there is an external elementto the procedure.

Quality assurance is an all-embracing term covering all thepolicies, processes, and actions through which thequality of higher education is maintained and developed.

Chapter 4

Quality Assurance and Programme Designin Europe

4.1. INTRODUCTION

The changing context of higher education worldwide has beenoutlined above in general terms. Over the last century, many ofthe drivers, which prompted the development of externalevaluation of higher education in Western Europe, had theircounterparts in the Central and Eastern countries, even if themotives were, for the most part, dissimilar. The followingsections describe aspects of existing quality assurancemechanisms in higher education in Europe. They deal,respectively, with Western Europe, mainly describingdevelopments in the European Union, and with Central andEastern Europe.

This division was not made to create a new barrier betweenEast and West, but rather to allow space to describe andreflect on the different starting points, pace, and modes bywhich higher education systems and institutions are travellingtowards the same destination the European HigherEducation Area. Although Sections 4.2 and 4.3 cover similartopics, the emphases are different. Section 4.3 ondevelopments in Central and Eastern Europe is complementedby a detailed annex (Annex 2) describing the higher educationsystems of the region and a first, comprehensive, updating ofinformation on the quality assurance agencies in CentralEurope, based on a questionnaire (Annex 2.16.4) sent tomembers of the new regional network for quality assurance.Section 4.2 on Western Europe focuses more on developmentalwork being done at national and international level to establishexternal reference points for quality including generic andsubject specific descriptors for qualifications.

35

33


Many of the details in Section 4.2. concerning qualityassessment and accreditation in the European Union weretaken from published material, including questionnairescompleted by members of the International Network for QualityAssurance Agencies for Higher Education (<http: / /www.ingahe.nb),descriptions of their work, in English, provided by agencies ontheir Websites and in publications, from contacts at agenciesand other academic associations, and from recent seminarsand workshops on quality and qualifications that have beenheld as part of the Bologna process. A separate questionnairewas not sent out to ENQA members as a survey mapping thesimilarities and differences in European quality assurance aspracticed in the various countries is nearing completion. It isbeing managed by the Danish Evaluation Agency on behalf ofthe European Network for Quality Assurance (ENQA) andcomprises two phases:

A survey of quality assurance in thirty-two countries,which will focus on the evaluation methods used inthem, and country specific reports will be prepared.The second phase of the project is an analytical report onthe state-of-the-art of evaluation procedures, which willbe based on a number of identified themes and aspectsof the evaluation methodologies and proceduresidentified in the first phase.

Both reports will be made available on the ENQA web-site<http: / /www. enqa. net> .

4.2. QUALITY ASSURANCE AND PROGRAMME DESIGN INWESTERN EUROPE: THE EVOLUTION AND CURRENT STATEOF AFFAIRS DESCRIPTION AND COMPARATIVE ANALYSIS

The drivers, that prompted the introduction of external qualityassurance systems in the European Union, mirror many of thepoints made above concerning the changing context in whichhigher education was operating:

PROGRAMME DESIGN 37

4.2.1. Increased Institutional Autonomy in Return for GreaterAccountabilityThe grant of increased autonomy to higher education wasparalleled by a rise in the demand for accountability to re-assure authorities that resources were being well-spent, avalue-for-money argument in certain Western countries, andto ensure that the expectations of interested constituenciesstudents, would-be-students, their parents, employers, andsociety at large would be met. Increased institutionalautonomy, however, pointed up the need for enhancement andimprovements in institutional leadership, informationgathering and dissemination, and effective decision-makingwithin institutions.

4.2.2. Diversification and ExpansionIn addition to increased numbers of students resulting inincreasingly large classes, the diversification of the studentintake including more adult learners raised issues ofteaching and learning methods and put strains on access tolearning resources and support. In many countries, greaterlevels of participation in higher education were not alwaysmatched by a corresponding rate of increase in the number ofgraduates. Significant percentages of European students didnot complete their studies raising issues such as the waste ofhuman talent and potential and of public money. Reasons forhigh drop-out rates included the growing mismatch betweenthe official length of studies and the actual time spent gainingqualifications perhaps reflecting problems of overloadedcurricula, out-dated teaching and learning methods, and thelack of intermediate or shorter qualifications in the highereducation systems in question. The retention and progressionof students remains a concern in many countries and hasemerged as a performance indicator in several systems eitherfor quality assurance or for funding purposes, e.g., inDenmark, in Finland, in the United Kingdom, and in someLander in Germany.


Diversity has meant not only the development of new typesof higher education institutions such as polytechnics (e.g., inPortugal and Finland) and other tertiary level institutions, butalso the introduction of new disciplines and programmes,including shorter degree programmes in certain systems. Thewider variety of types of higher education institutions and ofcourse programmes has presented challenges for students,who have needed more and better information to help themmake informed choices about the best study options for them,and to external evaluation agencies, the criteria of which werepreviously been predicated on traditional university provision.Diversity has also highlighted the number of barriers thatremain in terms of mobility and progression as links andpathways between different types of higher education are notalways defined.

4.2.3. Need for New Methods of Teaching, Learning, andAssessmentThe development and widespread use of ICT applications foreducation has created the need for new skills for teachers andhas implications for curriculum design and the definition ofnew criteria for the quality assurance of delivery.

4.2.4. InternationalizationThe mobility of students either in full programmes or inexchange schemes has increased demands for the recognitionof qualifications, credits, and study periods abroad. Thisdemand has prompted the need for greater transparency ofcurricula and assessment.

Not all of these factors have applied at one and the sametime to all European countries, but they all have a commonconsequence, that is, the need for both institutions andgovernments to manage "change".

4.2.5. Concepts of and Approaches to QualityAt the time of the production of the PHARE Quality AssuranceManual (1998), it was observed that all Western European

PROGRAMME DESIGN 39

evaluation procedures were based on quality as enhancementrather than on quality standards. This landscape, however, isa changing one, and work in individual national systems andwithin professional and discipline-related areas at national andinternational level is underway to refine standards, criteria,qualification descriptors, and other external reference pointsfor quality. While this work still includes identification ofcertain quantitative input standards, it is now moving towardsemphasizing the definition of learning outcomes and outputstandards and competencies in order to create academicqualification frameworks that accommodate diversity whilesecuring a clear understanding of what academic qualificationsmean and the associated knowledge and skills students shouldhave. The focus is shifting to outcomes rather than process(see Annexes 5.1.-5.4. on the Joint Quality Initiative Project,the Tuning Project, QAA Subject Benchmark. Statements, andZEvA Standards for BAMA Programmes) and on providing clearexternal reference points for quality and standards.. Theorganization of quality assurance at system level may beperceived as moving (slowly) towards defming "fitness ofpurpose" but with a strong accent on improvement. This latteremphasis is borne out by, for example, the articulation of moreformal "follow-up" processes (Sweden, the United Kingdom,and the European University Association) to establish whatimprovements had been made following evaluation judgmentsand recommendations.

The origins of the majority of the external evaluationagencies in Western Europe were at the initiative ofgovernment (see the summary of the INQAAHE questionnaireresponses in Annex 1). Exceptions include the Netherlandsand Belgium (Flanders), in which the initiative was taken bythe associations of higher education institutions, and Germanyand the United Kingdom, in which initiatives have been takenjointly by higher education organizations and government. Notwo national agencies are identical either in aims, focus ofoperations, coverage of the education sector, criteria, andstandards used. This diversity reflects the different national


and regional contexts in which national agencies operate andthe constituencies which they serve. Even when countries havesimilar social, economic, and political backgrounds, forexample, the Nordic countries, there is little apparentconvergence (Smeby and Stensaker, 1999).

Although established mainly at government initiative,evaluation agencies in Western Europe operate as independent,autonomous organizations, free of control by government oruniversities. This principle was set out in the EuropeanCouncil Recommendation on Quality (1998)). In somecountries, this independence is enshrined in law, e.g., inFrance, where the Comite Nationale d'Evaluation reports to thePresident of the Republic; in others, it is a reflection of thenature of the evaluation agency, for instance, in the UnitedKingdom where the QAA is a charitable trust managed by aBoard of Directors. What autonomy means in practice is thatthe agencies are mandated to carry out approved processesand to make their judgments and recommendations withoutinterference or influence from government and or highereducation institutions. This mandate does not mean thatagencies are not accountable to governments, ministries, orother stakeholders or that their processes or indeed their veryexistence may not be subject to change in the light ofmodifications in government higher education policy.

4.2.6. Purposes of EvaluationA summary of the responses to the INQAAHE questionnaire bymembers can be found in Annex 1. It appears to demonstratethat the purposes of existing agencies have evolved beyond thetraditional quality improvement/enhancement/accountabilitycontinuum. Of the fourteen Western European evaluationagencies that responded in detail, nine (Denmark, Finland,Germany (AKK), Norway, the Netherlands (VSNU and HBO-Raad), Spain (Catalonia), Sweden, and The United Kingdom)indicated three or more purposes as important to veryimportant:

Quality enhancement was cited by all.

PROGRAMME DESIGN 41

Accountability was cited by eight; however, only mostimportant by four (Austria (Akkreditierungsrat privateuniversities <http://www.bmbwk.gv.at>), Belgium (VLIR

university QA), the Netherlands (HBO-Raad), and TheUnited Kingdom (QAA).Information provision was cited as important by sixagencies (Denmark, Germany (AKK), the Netherlands(HBO-Raad), Norway, Spain (Catalonia), and the UnitedKingdom).Benchmarking was rated as important by six agencies(Austria Akk), Denmark, Finland, Netherlands (HBO-Raad) and (VSNU), and Spain (Catalonia). This criterionmay be interesting to note, as all are smaller countriesand/or a regional agency.

Accreditation as a purpose was mentioned, not unexpectedly,by the new, national accreditation agency in Germany(Aklcreditierungsrat) but also as the sole purpose of thePortuguese evaluation agency for teacher education (INAFOP<http: / /www.inafop.pt>) and as one of the purposes of the twoAustrian agencies. In contrast with the Central and EasternEuropean countries, it appears that accreditation as a purposeof evaluation is not yet a feature of most European Unionsystems. However, this conclusion belies the fact that in somecountries, accreditation or homologation of degrees and/orinstitutions is undertaken by the Ministry of Education, e.g., inFrance and Spain (ENQA, 2000, Institutional Evaluations).Accreditation-like practices occur in Nordic countries, mostlyin relation either to professional programmes or to privateinstitutions (ENQA, 2001). Accreditation activities of statutoryand professional bodies as well as the granting of degree-awarding powers occur in the United Kingdom as an ex antetype of accreditation for institutions (for criteria see<http: / /www.dfes.gov.uk>). However, the majority approach toexternal quality assurance in Western Europe remains eitherprogramme and/or institutional evaluation.

This extension to the purposes of agencies is reflected intheir aims, tasks, or mission statements and is perhaps an


indication that when new agencies are being established theyshould have the flexibility to adapt to changes without thenecessity to have cumbersome legislation enacted.

In the case of agencies in the Central and EasternEuropean countries, the purpose of quality assessment isaccreditation. The responses of agencies to the INQAAHEquestionnaire were aimed at showing what they perceived astheir most important purposes beyond the formal judgment ofcompliance with set criteria. The variety of replies very likelyimplies that the question allows for different levels ofinterpretation. Half of the eight Central and Eastern Europeanagencies that responded to this question named accountabilityas well as improvement and enhancement as the first orsecond most important purpose of their activity (Albania,Hungary, Latvia, and Slovakia). But Bulgaria setbenchmarking and provision of information first and markedaccountability least important. For Slovenia, improvement andthe provision of information were most important, while theCzechs focus on benchmarking and accountability. For thelatter, improvement/enhancement and providing informationare tied second. Estonia ranked accountability last,improvement orientation second, and benchmarking asmoderately important, with nothing for "important" or "veryimportant" (i.e., 4 or 5), and it is possible that the reverseordering was intended.

4.2.7. Scope of Evaluation: Institutional or ProgrammaticUniversities or CollegesAll the agencies in the Central and Eastern Europeancountries conduct both institutional and programmeassessment. Usually, the assessments are undertaken inseparate processes, but Hungary, for example, in its first cycle,evaluated programmes so as to arrive at an accreditationdecision regarding the given institution.. The HungarianAccreditation Committee is planning on separating the twoprocesses, to some degree, in its second round. Croatia focuseson institutional assessment but also looks at the quality of

PROGRAMME DESIGN 43

programmes at the evaluated institution. Slovakia seems tohave introduced the combined approach in its new highereducation law that went into effect in April 2002. Russia andUkraine also evaluate programmes within their institutionalaccreditation procedure.

When evaluation agencies were first established in theEuropean Union, they tended to focus either on programme oron institutional evaluation. The United Kingdom at thebeginning of the 1990s was unique for a larger state in Europebecause it covered both institutional and subject evaluation avery expensive operation for a large and diverse highereducation system. In Ireland, the former NCEA carried outvalidations and approvals at both programme and institutionallevels. However, agencies in smaller countries or those whichare regionally based generally began with a focus onprogramme or discipline evaluation on a national/regionalbasis (e.g., The Netherlands VSNU and HBO-Raad, Belgium(VLIR), Denmark, Spain (regional Catalonia).

The evaluation of programmes as a focus has had potentialbenefits both for quality assurance at system and institutionallevel in:

- providing an opportunity for small-scale pilot exercises totest new processes before their wider implementation;providing an overall picture of the state of health of adiscipline in a country at a particular time; highlightingoverall matters for improvement at national level as wellas in individual institutions;facilitating the planning of a cycle of reviews anddeveloping and managing a programme of continuousengagement with institutions which may be necessaryto build up institutional capacity and familiarity withevaluation;potentially creating a multiplier effect by involving awider range of actors as evaluators and creating a"critical" mass of actors (academics, administrators, andstudents) in higher education institutions withexperience of evaluation which may stimulate the


development of internal, institutional quality assurancemechanisms at "grass-roots level";where the focus is on discipline or subject, rather thanprogramme evaluation, offering the potential of internalbenchmarking within higher education institutions, inwhich the same subject is offered in several faculties;potentially providing information which is of more directvalue to students (especially international students) andemployers;providing comparative information to institutionalmanagers and leaders on issues such as teaching andacademic standards across the institution;providing evidence of the effectiveness of institutionalquality assurance practices.

Criticisms have been voiced that focusing on programmeevaluation may place a considerable burden on institutions asthey may have to manage multiple evaluation events in anyone year. These criticisms were aired most recently in theUnited Kingdom at the conclusion of the first round of reviewsof all subjects in England. By the conclusion of the eight-yearperiod of review, less than 1 percent of provision had beenjudged to be failing. However, this result was evidence of theeffectiveness of internal review processes and contributed tothe development of a new external review process which willutilize information from internal evaluation processes (forfurther details, see Report 02/15, 2002 of the Higher EducationFunding Council for England <http://www.hefce.ac.uk>) andfocus, in the main, on institutional audits.

4.2.8. Institutional EvaluationEvaluation of higher education institutions in Europe becamecommon as of the mid-1980s. France was the first country toinitiate comprehensive university evaluations in 1985. TheComite National d'Evaluation (<http: / /www.cne_evaluation.fr>)was established by the Law on Higher Education of 1984. TheLaw also granted universities administrative, pedagogical,research, and financial autonomy. In addition to the evaluation

PROGRAMME DESIGN 45

of individual institutions, the possibility of organizinginstitutional reviews/evaluations based on a regional groupingof institutions exists in Ffance. This possibility provides anopportunity to determine the extent to which a region is wellserved by its higher education institutions. The second seriesof institutional evaluations in France has a lighter process:

Because the "legitimacy" issue was no longer critical, theacademic community did not need a comprehensivereport to be assured of institutional unity.Because the CNE wanted its recommendations tobecome more efficient for quality improvement anddecision-making, only the main issues identified in theself-assessment and external evaluation were to beincluded in the recommendations.

Institutional evaluation is also a focus for quality evaluationor audit in Sweden, Finland, Norway, Ireland, and the UnitedKingdom. However, the new Swedish system is evolving fromquality audit at institutional level to quality assessment of allhigher educational provision for general and professionaldegrees. A full round of institutional audit was completed in1998. The intention of the audit had been to assess thestrategies, goals, plans, systems, methods, and organizationused to secure and develop overall quality in each institutionrather than on evaluating quality against set criteria.Criticisms of the process included the fact that institutionalaudit did not reach out within institutions to the core ofactivities at departmental level and that some of the auditswere considered to be "toothless" (Franke, 2002).

An alternative to national institutional review is theinstitutional review process offered by the European UniversityAssociation. This opportunity has often been taken up incountries that do not have a national evaluation system atuniversity level, e.g., Portugal and Greece, or when a giveninstitution is interested in adding an international dimensionto the review (information on EUA quality policy andinstitutional review practices can be found in Annex 4.6 and at<http: / /www.unige.ch/eua>) .


4.2.9. Thematic Reviews and ReportsSeveral agencies undertake thematic reports or evaluationseither at the request of government or other authorities or ontheir own initiative. An example of the latter is the currentprogramme of the Danish Evaluation Institute which includesevaluations on classroom observation and on methods of self-evaluation in the Danish education sector.

4.2.10. One Agency or SeveralWithin binary systems of higher education or when there is amixture of private and state provision, decisions have to betaken as to whether there are separate focuses for differentprovision or one process with appropriate standards andcriteria for different providers. Systems in smaller Europeancountries have tended to take an inclusive approach.Denmark, Sweden, Finland, and Norway each have a singleevaluation agency to cover all higher education provision;indeed, the Danish agency covers all types of education fromKindergarten to research evaluation. Exceptions are Austria,the Netherlands, and Belgium (Flanders). The policy ofinclusion has also been in effect in the United Kingdom andFrance in which all nationally recognized higher educationinstitutions are subject to institutional evaluation by thenational agency. In the Netherlands, Belgium (Flanders), andIreland, there are separate agencies for university and non-university institutions; however, the proposed DutchAccreditation Agency for Higher Education will cover bothsectors of higher education. In Austria and Portugal, there areno evaluation agencies for public universities, but separateagencies exist in Austria for private universities and forFachhochschulen, and in Portugal there is an accreditationagency for teacher training education (INAFOP).

The German accreditation system may appear to beevolving along the lines of the American system with arecognizing body (Akk in Germany, CHEA in the US) whichaccredits agencies operating at regional or professional/disciplinary area.

PROGRAMME DESIGN 47

If a trend can be discerned in this aspect of evaluation, it isnot only one favouring multiple purposes for agencies but alsomultiple focuses of evaluation for single agencies.

A further development is the existence of multiple purposeagencies which have responsibility not only for quality assuranceor accreditation but also as the authority for the recognition offoreign academic and/or professional qualifications. Examplesinclude Network Norway Council (<http://www.nnr.no>), theHogskoleverket, Sweden (<http : / /www. hsv. se>) , and theLithuanian Center for Quality Assessment in Higher Education(<http:/ /www.vu.lt>) . To some extent, these national initiativeshave anticipated the growth in networking and contactbetween the emergent QA networks and the academicrecognition information networks NARICs (in the EuropeanUnion) and ENICs (for Council of Europe and UNESCO-CEPES).

4.2.11. Evaluation Criteria and StandardsWhat criteria and standards are set for external evaluations?Again, there is no one rule for Western European countries,and practice varies with approaches which include "anenabling legal framework with- broad general guidance, re.purpose", to a strict legal framework or one which assignsresponsibility for determination of standards to the Agency inco-operation with the constituencies (or stakeholders) to refinereference points. In Finland, the criteria are formulated by theagency in consultation with institutions whereas in Belgium,(Flanders), they are formulated by the universities. In theUnited Kingdom, the standards are the goals and objectives ofthe given institutions and the external reference pointsdeveloped in agreement with the sector. In Denmark, thecriteria used are those of the individual institution and"general criteria of good practice". In Germany (AKK), thestandards are the "goals as formulated by the institution, theprofession, and AKK".

In, response to the INQAAHE questionnaire as to who hadmost influence in setting the criteria for an external evaluation,the indications were that the Agency (including for this


purpose the external evaluation team) had the most influencein determining the focus of individual evaluations.

4.2.12. Evaluation Methods and InstrumentsA strong consensus exists throughout Western Europe,irrespective of process and of the purpose of the externalevaluation, as to the methods and instruments used. Theelements common to all systems are:

a self evaluation/study;an external review with an on-site visit;a report;

and increasingly, the use of other instruments such asstatistical data provided either from national sources orindividual institutions;qualitative information from the internal qualityassurance processes of the institution itself;performance indicators;user surveys labour market/students/employers (e.g.,in Denmark and Finland);external examiner reports (Ireland, The United Kingdom,and Denmark);structured follow-up processes.

Agreement on these common instruments does not,however, imply similar implementation practices. There arevariations in the nature and coverage of the self-evaluationreflecting the purpose of evaluation; the criteria and standardsof evaluation, the aspects of provision under evaluation; theduration and structure of visits; the identification,appointment, training, and role of experts; and the judgmentsmade. Some examples are included in Annex 4.

4.2.13. The Self-Evaluation StudyThe one common feature about the self-evaluation study isthat unlike external evaluation reports, self-evaluationdocuments are not published by any existing evaluation

PROGRAMME DESIGN 49

agencies and remain confidential unless the institutionchooses to release them. While self-assessment is a centralelement of all institutional evaluations, the prescriptiveness ofthe parameters for the self-study as set out by the evaluationagency either in its handbook or through other guidelinesvaries. Sometimes the self-evaluation document, especially forstudy programmes, will be predicated on legislation and will bedesigned to show how the programme achieves not only theaims and purposes established by the institution offering theprogramme but also provisions in appropriate legislation, e.g.,the FH Council, Austria (see Annex 4.1.1.). In Norway, thehigher education institution may choose the methodology andorganizational approach to suit its purposes and interests. Theposition in Finland is similar but with the proviso that the self-study reflect the interests of consumers (students, externalstakeholders, the surrounding community, etc). In Sweden, inthe new quality assessment process at programme level,institutions are encouraged to demonstrate "what is uniqueand especially characteristic about their programmes and byso doing highlight... good practice" (France, 2002).

The British and Irish models have shared a more directiveapproach. In Ireland under the former NCEA, there wasconcentration on organizational matters such as:

mission and goals;governance and strategic planning;academic policies;the use of institutional resources;

all of which were to be reflected in the self-assessment.The new institutional audit method which is being

established in England will, in addition to self-evaluationdocuments at institutional and discipline level, also draw oninformation which all institutions are to be required to produceannually from their internal quality assurance processes. Anexcerpt from the Draft Handbook for Institutional Audit inEngland (April 2002) Annex C: "Guidelines for Producing Self-Evaluation Documents for Discipline Audit Trails" is


reproduced in Annex 4.1.2. The institution being audited isalso expected to demonstrate the extent to which it addressesthe points of reference for quality and standards provided byparts of the "Code of Practice and the Higher EducationQualifications Framework" of the QAA.

No matter what model of self-evaluation is adopted, theremust be clear and effective guidance to institutions as to whatthey can or must do. The guidance (which is normally in theform of a handbook or other written guidelines and is generallypublicly available) should assist institutions to undertakereflection and analysis rather than directing them to focus ondescription. However, it has been recognized that merelyproviding a handbook or other guidelines may not be sufficientpreparation for external evaluation, especially wheninstitutions have little capacity for, or experience of,conducting self-evaluations (Genis, 2002). A programme oforientation and capacity building exercises, such asworkshops on writing self-evaluations and informationsessions on new QA processes before they are introduced, mayresult in dividends in terms of the quality of self-evaluationswhich will in turn enhance the external evaluation process.

4.2.14. External Experts: Selection, Training, and BriefingThe success of peer review depends heavily on the training,experience, and professionalism of teams (Kells, 1993, p. 104).Practices in terms of the training and briefing of teams varyconsiderably in Western Europe, but a common feature is abriefing meeting before evaluation visits take place, as areguidelines or handbooks on protocol and evaluationprocedures. The nature of training and briefing to some extentdepends on the pattern of evaluations. It is easier to coordinatetraining and briefing activities if a pool of experts is selectedand appointed either for the duration of the cycle or at least fora specified period. Practice varies as to who selects the experts.In some cases, it is the agency, with either the professionalstaff or appropriate Committee making the decisions; inothers, there may be nominations from other sources. The use

PROGRAMME DESIGN 51

of foreign experience is common in some of the smallercountries, particularly when there is a shared language, e.g.,The Netherlands and Belgium (Flanders) and in the Nordiccountries, but is also a feature of the activities of the CNE(France) and of some of the German Accreditation Agencies.Examples of practices regarding the selection and training ofexperts are included in Annex 4.3.

4.2.15. Reports and JudgmentsIn all systems, some kind of public report is generated at theconclusion of the evaluation, but the nature and extent of thisreport varies. In those countries in which all provision isreviewed at the same time, subject reports may be produced:e.g., the VSNU in the Netherlands (see <http://www.vsnu.nl>for examples in English). However, the VSNU also produces aconfidential report for individual institutions on theirprogrammes. Other agencies work entirely in the publicdomain and publish reports in full, reflecting their aim toinform stakeholders, e.g., QAA (in The United Kingdom)<http://www.qaa.ac.uk>. In some countries, reports orsummaries of reports are published in English even when it isnot the national language, e.g., Denmark (<http: / /www.eva.dk>)and the Swedish National Agency <http://www.hsv.se>.Making reports available on the Internet is a cost-effectivemeans of publishing, but alternative means must be availableto ensure that those who do not have access to the Internetcan still obtain the information.

Practices vary from agency to agency as to who draftsevaluation reports. Sometimes it is the agency staff (EVA,CNE); in other cases, the agency staff may comprehensivelyedit the report written by the experts but never alter thejudgments, e.g., QAA. In other cases, the experts may write thereports. In the latter cases, the training and briefing of expertsshould take account of the need to assure consistency andquality in reporting.

The nature of judgments also varies depending on thepurpose of the evaluation. There may be yes/no decisions


concerning accreditation and a series of recommendations forimprovement and/or commendations for good practice forevaluation. No Western agency "ranks" programmes orinstitutions, but evidence from published reports, especiallywhen they include graded judgments, is often used by othersto create ranking. Recently, commentators in the United States(Jones, 2002) have observed that different stakeholders ineducation are looking for different types of information fromevaluation processes which present a real challenge forevaluation agencies to meet.

4.2.16. Aspects of Programme DesignQuality assurance agencies are not responsible for the designof individual programmes of study but may influence themeither through the establishment of external reference pointsor standards of quality or through the recommendations orjudgments made in the external evaluation of programmes.

In Western Europe, programme design has ranged frombeing either the sole responsibility of individual institutions(British and Irish universities are examples) to theresponsibility of institutions within a fairly prescriptive legalframework which may, however, allow for a certain degree offlexibility, e.g., in terms of optional courses in addition to acompulsory core curriculum (e.g., Spain). In France, theMinistry of Education develops degree programmes leading tonational awards according to its own guidelines and is incharge of the accreditation of degrees and institutions. Thisstipulation does not preclude institutions from designing andoffering their own programmes, but such programmes do notlead to national awards. In other countries, institutions haveautonomy over programme design and content but areprovided with reference points such as national descriptors forqualifications and programmes either in terms of credits(Sweden for academic degrees) and/or study hours and abroad subject outline. However, the impact of the BolognaDeclaration and especially the desire to create greatertransparency about qualifications for international purposes

PROGRAMME DESIGN 53

have set in motion a frenzy of rhetoric, reflection, and activityin re-structuring qualifications and higher education provision.Much of the activity is incomplete, and there is as yetinsufficient evidence on which to base judgments as to thesuccess of changes in enhancing the transparency ofqualifications.

In some systems, the use of credits or credit points eitheron the basis of a national system, e.g., Sweden, Finland, andthe Netherlands, or using ECTS (the European Credit TransferSystem <http://www.europa.eu.int/commdeducation>), e.g., Denmarkand Italy is a requirement in designing programmes. In others,use of a credit system is optional. However, credits, per se,while a useful tool to organize curricular frameworks, have norelationship to quality or standards.

4.2.17. Two Cycles of EducationAs in Central and Eastern Europe, the Bologna process has setin motion a series of changes in relation to the definition ofqualifications which in turn impinges on programme design.The changes include:

measures to introduce qualifications based on two cyclesof education Bachelor's/Master's Degrees in countrieswhere these are "new" qualifications, e.g., Italy, theNetherlands, and Germany. Sooner or later, dependingon the country, these qualifications will replace thetraditional long, first-degree programmes (See Trends IIReport for more details at <http://www.oph.fi/publications/trends2>). Where binary systems of highereducation exist, it appears that there will be two types ofBachelor's and Master's Degree programmes academicand professional (The Netherlands). An example ofdifferent descriptors for two orientations ofBachelors/Masters degrees in Germany is given inAnnex 5.2., re. the standards of ZevA).the redefinition or refinement of qualificationsframeworks in which two cycles of higher educationalready exist. For example, in the United Kingdom,


Higher Education Qualifications Frameworks wererecently agreed upon and have been established as oneof the reference points for external evaluation. In Ireland,there is currently consultation on a new nationalqualifications framework (<http://www.nqatie>. In thiscase, an interesting aspect of the proposal is for theinclusion of professional and international awards (thatis, provision being offered in Ireland but not by Irishinstitutions) in the framework.

There are two Europe-wide initiatives in the framework ofthe Bologna process which may have an influence onprogramme design and curriculum content through thedefinition of subject specific learning outcomes and the genericdefinition of qualifications (Bachelor's and Master's Degrees)frameworks: the Tuning project (<http://www.relint.deusto.es>)and the Joint Quality Initiative (<http://www.jointquality.corn>).They are described in Annex 5.

4.2.18. Emerging Trends in Quality Assurance and the Designof Programmes in Western Europe

expansion and diversification of the scope and type ofwork that evaluation agencies are carrying out.Examples include the establishment of agencies with acomprehensive range of functions in relation to qualityand the recognition of qualifications, especially insmaller higher education systems (the Nordic countries);a greater emphasis on enhancement activities such asthe introduction of follow-up processes after evaluation;and the undertaking of thematic reviews on wideraspects of quality and standards.work on developing instruments to assure and evaluatethe quality of new modes of provision and new providersof higher education. This work includes the developmentof codes of practice, both national (e.g., The UnitedKingdom Code of Practice on Collaborative Provision) and

PROGRAMME DESIGN 55

international (the Council of Europe/UNESCO-CEPES<http: / /www. cepes. ro>), on transnational education.the review and updating of evaluation processes to takeaccount of the experiences of both institutions andevaluation agencies after the completion of a full roundof reviews. For example, the shift in emphasis frominstitutional audit to programme assessment in Swedenand from programme assessment to institutional auditin England.the development of new external reference points forquality and standards such as subject benchmarkstatements and new generic descriptors for BA/MAqualifications at regional, national, and internationallevel based on student learning outcomes. While thiswork had already started in some countries beforeBologna, the Declaration has had a catalytic effect as hasthe drive for international recognition of qualifications.the introduction or development of accreditation systemsat national (the Netherlands), regional and subject-based(Germany), and international (FIBAA, EQUIS) level.an increase in networking in quality assurance andenhancement activities:

at agency level the development of internationalnetworks of quality assurance agencies (ENQA,CEENetwork, INQAAHE, CHEA) - all listed in Annex6, bilateral links between agencies, e.g., VLIR andVSNU, and the exploration of mutual recognitionbetween agencies, e.g., a recent project involving EVA(Denmark) and FINHEEC (Finland);at institutional level often on an international basis

the emergence of university networks andbenchmarking clubs, such as Universitas 21<http: / /wvvw.universitas2 1 .com/ collaborative . html>and the Coimbra group <http: / /www.coimbra-group .be> ;

at subject level, e.g., Socrates thematic networks;


by the professions, e.g., the mutual recognitionactivities in engineering by accreditation agenciesthrough the Washington. Accord (See, Annex 4.7.2. forinformation).

greater involvement of stakeholders, especially students,in the external evaluation process. Examples of thisinvolvement include the participation of the EuropeanUnion of Students (ESIB <http: / /www.esib.org>) asmembers of ENQA, student representation either on thecouncils of agencies (e.g., in the United Kingdom) or onvisiting panels (Sweden). But, this involvement is morethan "representational". Recent revisions of externalevaluation have put the interests of students at theircore. "Students are central both to the principal focusesof institutional audit and to the audit process itself.Audit teams scrutinize a range of matters directlyrelevant to students, including the quality of theinformation provided for them, the ways in which theirlearning is facilitated and supported, and the academicstandards they are expected to achieve, and achieve inpractice. In each audit, students are invited toparticipate in the key stages of the process...." (QAA,2002). 'The student perspective is also very important.They constitute one of the most important groups thatthe peer group team meets at visits and are alwaysrepresented on the expert groups as well" (Franke,2002);the provision of more and better information on qualityand standards which is prompting:

debates about changes in the nature of judgmentsmade by agencies; whether or how to makejudgments without ranking. For many existingaccreditation systems, such a shift would implychanging from summative to formative judgments;debates about disclosure or wider publication ofevaluation reports. In many existing systems,

PROGRAMME DESIGN 57

"publication of reports" means only sending them tothe institution that has been evaluated or making asummary of the report available more generally.Wider disclosure need not mean enormouspublication costs but does imply the setting-up andmaintenance of efficient Website and negotiationswith institutions. The degree of expense hasimplications for both national systems andinstitutions. International students are becomingincreasingly sophisticated in searching forinformation about study opportunities and frequentlyuse the Internet to do so. Wider publication of reportsmay also provide an opportunity for the spreading ofinformation about good practice, thus contributing tothe enhancement aim of external evaluation;the need to improve the "language" about quality formany agencies, the public is the ultimate constituent,and much of the language of quality and standards isjargon-ridden and not immediately useful toconstituents such as government or students,(examples include publication by QAA of leaflets forstudents; the CHEA glossary of terms).

closer links at national and international level betweenquality assurance agencies and bodies responsible forthe recognition of qualifications for academic andprofessional purposes. At international level, an exampleof such a linkage is the co-operation between ENQA andthe ENIC and NARIC networks (more information aboutENICs from < http : / /www.support4leaming.org.uk/education/narics.htm>). The outcomes of two meetingsbetween representatives of the networks was theidentification of specific issues for further work:improving the communications between the networkmembers both at national/regional level as well as atnetwork level and how to improve the definition ofquality and recognition issues in non-formal education.


the emergence of other "non-formal" commentators onquality and standards who "rank" institutions andprogrammes. Examples of this phenomenon are therankings of universities by newspapers such as DerSpeigel (Germany), the Financial Times (business schoolsaround the world), The Guardian (British universities),etc., even when national quality assurance agenciesthemselves make no rankings. While this trend may notbe viewed as positive, it does, nevertheless, demonstratean interest in and demand for information on qualityand standards.

However, "a national evaluation system can only best be asupporter of the activities taking place in the educationalprocess" (Franke, 2002), and there is a growing awareness ofthe need to strengthen internal quality assurance andmanagement in higher education institutions in many WesternEuropean countries. There is still a perception in manyinstitutions that quality assurance is an external activity orcontrol imposed on them. A recent initiative in response to thisperception is a proposed project, to be managed by theEuropean University Association, to focus on the developmentof internal quality cultures in European Universities.

This initiative also brings into focus an issue which is notabout quality management per se, but is closely related to it,i.e., the perpetuation in some Western European countries ofrigid governance and outdated management structures forhigher education which mean that tertiary educationinstitutions are not themselves in a position to exerciseeffective control over the principal factors affecting the qualityand costs of their own programmes (World Bank, 2002).Matters such as granting institutions more freedom andauthority over curricula, teaching methods, the appointmentand promotion of staff, and the selection of students, as well asover the improvement of leadership and management are alsoessential in order to create institutions which will be able todeliver on the targets set by the Ministers of Education atBologna and Prague.

PROGRAMME DESIGN 59

4.3. QUALITY ASSURANCE AND PROGRAMME DESIGN INTHE CENTRAL AND EASTERN EUROPEAN COUNTRIES:THE EVOLUTION AND CURRENT STATE OF AFFAIRSDESCRIPTION AND COMPARATIVE ANALYSIS

After the fall of communism in and after 1989-1990, all theCentral and Eastern European countries began to restructuretheir higher education systems. Acts on higher education wereintroduced which gave tertiary education an autonomy it hadnot known for half a century or at all, allowing the respectivesystems to set up internal decision-making bodies(institutional and faculty senates, research committees, etc.),giving them hiring and firing powers, and involving students ininstitutional governance. At the same time, the acts and otherlegislation installed a number of control mechanisms.

The hiring and firing of academic and administrative staff,who are public sector (Le., government) employees, forexample, is also regulated by public employment laws. Thelegislation, moreover, has outlined the minimum requirementsfor a higher education institution with regard to its functioning,structure and organization, finance and governance, activitiesand services, and has usually defined the types of institutionsthat may operate in a given country. The laws describe therequirements for study programmes with regard to structureand minimum content as well as to different levels ofeducation. In many countries (Bulgaria, Estonia, Hungary,Latvia, Lithuania, Romania, Russia, Slovenia, and Ukraine),the laws stipulate the promulgation of national qualificationrequirements or subject benchmarks which establish thegeneral content, examination schemes, and criteria for degreeprogrammes.

In designing their study programmes, higher educationinstitutions have thus had a high degree of autonomy toaccommodate the needs of their stakeholders andenvironments. At the same time, they are bound by regulationsas to both the way their programmes are made up and ex anteand/or ex post quality checks.


The establishment of democracy in the Central and EasternEuropean countries released tremendous energies and affectedall walks of life, including higher education. The time framecoincided with such global trends in higher education as thesharp increase in the demand for study places, the emergenceof a large variety and flexibility of study programmes andschemes, the entry of information technology into the marketfor educational provision, the increasing interest in qualityassurance and government demands for accountability in theuse of public monies, and calls for stakeholder protection.

Different movements were set in motion that gainedmomentum in Europe through various European Unionprogrammes (e.g., TEMPUS, SOCRATES-ERASMUS, PHARE,NARIC), the Council of Europe, and UNESCO/CEPES (e.g., theLegislative Reform Programme, ENIC), OECD, and others. TheBologna Declaration is perhaps the most far-reaching of allEuropean initiatives, which in turn has set into motion variousmovements to design the future of European higher education.

The Central and Eastern European countries have had toface the challenge of taking advantage of the opportunitiesoffered by the various trends and to adapt them to their ownhistorical, political, cultural, and social environments. FifteenCentral and Eastern European countries, all of them, with theexception of Serbia/Montenegro, have, in place, some sort ofnational-level quality assurance body for higher education. Themechanisms they assign to assessing quality are similar. Theterminology is largely the same. The implementation and itsramifications are both similar and different in each country.

The question as to why all the Central and EasternEuropean countries chose accreditation as their mechanism ofquality control in higher education, while the WesternEuropean model in the early 1990s was assessment/evaluation-oriented, has been analysed by various authors(Kristoffersen et al., 1998, pp. 21-22; Tomusk, 2000, pp. 175-185). The reasons cited include the need, with the emergenceof democracy, to establish comparability with Western highereducation; the necessity to re-evaluate the curricula to rid

PROGRAMME DESIGN 61

them of politically distorted content; and the urgency tomodernize programme content and approach as well as tointroduce more flexible programme structures. "Before 1990,higher education was still an 'elite' sector with rigid entrancerequirements and, in many places, semi-autonomous faculties.Everyone knew the "rank" of a college or university,... newrules of survival in the institutional hierarchy, a brain-drain...appeared" (Rozsnyai, 2001, p. 3). Answers, such as a steep risein private institutions in some countries and in pressure toallow access for a large number of students to a previouslyelite sector, may also be part of the explanation; however,these trends were experienced in Western Europe as well,albeit more gradually.

There has been the blunt postulation that accreditation isthe approach most suited to a region accustomed to anautocratic mentality (Tomusk, 2000, pp. 175-185). It appearsthat all of the above have been factors in the preference of theCentral and Eastern European countries for accreditation overevaluation or assessment.

The systems of no two countries, East or West, North orSouth, are alike. All have evolved according to a handful ofmodels, but have adapted these models to suit their ownhistorical and cultural contexts. The fact that several Westerncountries are introducing accreditation as a form of qualitycontrol and accountability does not mean that the approachesthere are the same as those in the Central and EasternEuropean countries, nor are they the same in any two Westerncountries. The fact that the political regimes changed utterly

in the Central and Eastern European countries almost at thesame time may also explain why almost all opted foraccreditation.

The threat to close down institutions, which is inherent inaccreditation, has rarely been used in the Central and EasternEuropean countries (even though more so where stateaccreditation is viewed as a tool to control the extensiveproliferation of new private institutions, e.g., Romania andRussia). Various forms of warning, giving institutions or


programmes time to improve, were implemented everywhere.In fact, a trend can be observed that quality assuranceagencies and committees in almost all countries areincreasingly stressing the importance of an improvementorientation and relying more on the internal quality controlmechanisms in higher education institutions.

An interesting explanation for the function of accreditationwas given for Poland, but it holds true for most of the Centraland Eastern European countries.

The reason why accreditation is proving itself so popular inPoland is that, like elsewhere in Central and Eastern Europe, itis probably the only known mechanism in the world with anelement of certification, a classification of institutions intothose meeting the set standards and those failing to do so....On the other hand, the accreditation committees are notmerely stating whether [or not] the various curricula meet theminimum requirements, but they also look at the degree towhich these requirements are exceeded and compare this"excess" against the achievement of set goals. (fitness forpurpose). In this sense, the work of accreditation committeeshas got more to do with the kind of quality assessment carriedout in many Western European countries, with accreditationbeing merely a form of utilizing the results of this assessment.The name, "accreditation committee", is thus misleading, with"assessment and accreditation committee" being a more aptdesignation (WOjcicka and Chmielecka, 2001).

The INQAAHE questionnaire was filled out by almost all theCentral and Eastern Europe Network member countries. Theresponses show that in Albania, Bulgaria, Estonia, Hungary,Lithuania, and Romania, the initiative to establish the agenciescame from the government (or from Parliament). The same isclearly true for Russia and Ukraine, which did not submitquestionnaires. In all these countries, the government alsoholds ownership of the agency, except in Estonia (where it is afoundation). (Latvia marked both government and highereducation institutions as holding ownership, while the Czechssaid that their agency was an independent body. Both

PROGRAMME DESIGN 63

presumably had an unclear understanding of the questionsimilar to Macedonia which indicated that both thegovernment and higher education institutions established theagency. In these three cases, the government was the maindriver, while clearly the higher education institutions wereinvolved to some degree in most countries). In Macedonia, theownership of the agency is an inter-university conference.Poland set up agencies initiated by higher educationinstitutions in addition to the government one. The initiative inSlovenia also came from higher education institutions, butthese indicated that the government holds ownership.

To understand the questionnaire correctly, however, it mustbe noted that many Central and Eastern European countriesdistinguish between "agency" and "council" (or "committee" or"commission"). The "agency" is understood as the office incharge of coordinating evaluations, which may mean anythingfrom mere administrative activities (e.g., the Czech Republic) toactually being involved with evaluation teams. The commissionis the body of experts which approves accreditation decisionswhich may be based on the recommendations passed on bythe agency. The two must be taken as a unit and, in fact,many countries responded with this understanding, eventhough the "agency" may play a quite active role in the process(e.g., Hungary).

The degree of independence of quality assurance agenciesor committees vis-a-vis the governments of their countriesvaries. In some countries, higher education institutions havetaken the initiative to establish quality assurance bodies(Macedonia, the Polish agencies, the Slovenian QAC), but thegovernments in all of the Central and Eastern Europeancountries have some control over quality assurance. Someagencies operate within their ministries (the Czech Republic,the Polish State AC, Russia, the Slovenian Council, Ukraine) orare situated, more or less loosely, under ministerialsupervision (Croatia, the Estonian Council, the LatvianAccreditation Commission, Hungary, Lithuania, Slovakia), orunder that of the respective Council of Ministers (Albania,


Bulgaria) or the parliament (the Latvian Council, Romania).The various jurisdictional arrangements are not a directindication as to the degree of independence. The committeesmaking decisions as to the quality of institutions, programmes,or other higher education issues are almost always guaranteedautonomy by law. In Hungary, the Minister of Education maytake a decision opposed to the opinion of the AccreditationCommittee only if he or she publishes his or her reasons fordoing so. In the Czech Republic, the decision of theAccreditation Commission is even more binding. If itrecommends not accrediting a programme, the minister maynot override its judgement, and if the Commission adopts apositive decision, the Minister may refuse accreditation only onexplicitly stated grounds. Likewise, in Slovakia, the Ministermust publish dissenting decisions, and in Estonia, he or shemay modify a decision made by the council or the commissiononly on legal grounds. In Romania, the Parliament orGovernment adopts decisions relative to the accreditation ofinstitutions and programmes, respectively, and does not, as arule, diverge from the decisions taken by the commission, eventhough it potentially could. In Bulgaria, Croatia, Estonia(Council), Lithuania, Macedonia, and Slovenia (NationalCommission), on the other hand, assessment or accreditationdecisions are advisory and not binding on the Minister. InAlbania and Macedonia, moreover, one or more ministry orgovernment delegates participate in commission meetings. InRussia and Ukraine, the accreditation body is located in theMinistry; however, particularly in Russia, an agency assists inthe procedure (and some forms of education are accredited bylocal/regional authorities).

All commissions work with external pools of experts.Estonia recruits its experts from foreign countries, in part byapproaching accreditation agencies to recommend them. Theexperts are scholars and researchers in the fields to beevaluated. They remain in Estonia for a week during whichthey visit all institutions teaching in the field and write theirreports. The Hungarian Accreditation Committee, on the other

PROGRAMME DESIGN 65

hand, selects peer reviewers on the recommendation ofcommittee members and those of its subcommittees dealingwith issues in individual study fields. Foreign reviewers areinvited in fields in which there are no or only a limited numberof experts in the country.

Higher education institutions in most countries have someform of internal quality assurance mechanisms. The provisionis stipulated in the Higher Education Act in the CzechRepublic, and higher education institutions must publish theoutcomes. In Hungary, the Higher Education Law decreed thatinstitutions must set up quality audit commissions by mid-2000. The Slovenian Quality Assessment Commission was setup in order to assist institutions in implementing self-evaluation. Bulgaria has just completed a PHARE project tointroduce internal evaluation at higher education institutionsand may, in the future, revise its external assessment toinclude the internal results. The new higher education law inMacedonia lays great emphasis on institutional-level qualityassurance. Certainly, self-assessment is part of the qualitycontrol process in all countries.

The procedures for evaluation/accreditation in the Centraland Eastern European countries are in line with thepredominant model, i.e., they employ self-evaluations, externalreviews with on-site visits, and written reports. The content ofand approach to self-evaluation vary from mostly in-putoriented to analytical results (Croatia, Estonia, Latvia, andLithuania are examples for the latter). The focus of theevaluation/accreditation reports by the commissions alsovaries. All *countries (no data available for Ukraine) stress theimprovement orientation of their quality assurance systems,even though the degree and mode of achievement of this aim isinterpreted differently within the various cultural environments.Reports may be published (Albania, Czech Republic, Hungaryafter 2002, Latvia, Macedonia); however, in some countries,whether or not publication occurs is conditional on permissionbeing granted by the individual higher education institution(Romania). In other countries, reports are intended for internal


institutional and/or government use (Bulgaria, Croatia,Estonia, Hungary until 2001, Lithuania, Poland, Russia,Slovakia, the Slovenian Commission, and Ukraine), while thefinal decision and a brief explanation may be made publicly(Bulgaria, Hungary until 2001, Lithuania, the SlovenianCouncil, Ukraine).

In what follows the conditions for programme design invarious Central and Eastern European countries aredescribed, followed by an overview of their approaches toquality assurance. Annex 2 offers a country-by-country reviewof higher education in fifteen Central and Eastern Europeancountries with a focus on their quality assurance systems; anda subsequent set of tables provides an overview ofaccreditation/quality assurance agencies, but first, a briefdescription of the methodology used to compile thisinformation.

4.3.1. Methodology

Much of the information concerning quality assessment andaccreditation in the fifteen Central and Eastern Europeancountries studied was gathered by means of a questionnairewhen the Central and Eastern European Network of QualityAssurance Agencies was established in 2000. This information,that is available on the Network Website, is updated regularly,the updating being based on information supplied by thecountry agencies. Most Central and Eastern Europeanagencies have written short reports for the Central and EasternEuropean Network Website. These offer a brief description ofthe higher education of the countries and an overview of theirquality assurance systems. A great deal of additionalinformation was provided very generously and in great detailby contact persons in the respective countries, at qualityagencies, research centers, and ministries. A set of questionswas compiled for this purpose but answers were not limited tothese. An exception is Ukraine, in which no contact could beestablished and which has not joined the Network. Theinformation on Ukraine given in this publication comes mainly

PROGRAMME DESIGN 67

from a study published by the German Rectors' Conference(see Part III).

After the tables and country profiles on higher educationand quality assurance systems had been completed, they weresent to the contact persons in the respective countries forchecking. The replies made by these persons are incorporatedin the pertaining sections.

One set of listings in Section 2.1 of the Bibliographies inPart III provides descriptions of the higher education andquality assessment systems of certain countries. Another setconcerns more than one country or general topics related toquality assurance and higher education. Much, but not all, ofthe listed literature was provided by the contact persons of thequality assurance agencies that are part of the Central andEastern European Network.

4.3.2. Programme DesignCommon to all higher education systems in Central andEastern Europe is that the tertiary study programmes theyoffer:

are defined in national legislation;are comparable within a given country and, for the mostpart, are compatible with those in other countries;issue closing documents which certify qualification in agiven field and indicate the level of education achieved,and enable persons who have successfully completed therequirements to seek employment based on suchdocuments;are subject to quality control.

Before opening a study programme, higher educationinstitutions in most countries must submit applications forlicensing or accreditation to their national quality assessmentagencies or commissions (Albania, Bulgaria, the CzechRepublic, Hungary, Latvia, Macedonia, Romania, Slovakia, andSlovenia). Some systems are very flexible with regard to jointprogrammes among different institutions (e.g., Slovenia), while


in others (e.g., Hungary) such initiatives are less common,even though they exist.

4.3.3. Levels of StudyThe stipulation in the Bologna Declaration that all highereducation systems in Europe offer "a system of easily readableand comparable degrees [whereby] access to the second cycleshall require successful completion of the first cycle studies...."(Joint Declaration..., Bologna, 18 June 1999 <htip://www.mab.hu/english/a links.html>) has created a bustle in some countries inwhich the two-level or -tier (Bachelor's/Master's Degree)system was not applied. They have had to search for models inorder to restructure their tertiary provision. The challenge ismore pronounced in those countries with binary highereducation systems in which higher education and degrees areoffered in two distinct types of institution, either at universitiesor at non-university establishments.

Most prominently Germany, the home of the "Humboldtianuniversity model" (pursuant to one of Wilhelm von Humboldt'smain principles behind his university reform, the "unity ofteaching and research"), has traditionally regarded universitiesas educational institutions with a strong link to (theoreticaland methodical) research, while the relatively newerinstitutional types of the country, the Gesamthochschulen(comprehensive universities) and especially the Fachhoch-schulen (universities of applied sciences), were introduced toexpand the provision toward more applied education andtraining. The binary system was introduced relatively late inGermany as compared to countries that had relinquished theunitary system much earlier in their modem history or hada more diverse structure essentially from the beginning. Therewas no provision for changing from one type of institution toanother.

Under Soviet dominance, many CEE countries, whichearlier had followed the German model, installed the Soviet-type binary system with so-called specialized higher educationinstitutions, which were of the college level with an applied

PROGRAMME DESIGN 69

orientation in a particular general field. Under the Sovietmodel, pure research was assigned to the national academiesof sciences and their institutes. Universities providedtheoretically-oriented education while non-university institutionsor colleges offered applied education. The possibility forstudents to move between the two types of institution was notbuilt into the system. The systems of Albania, the CzechRepublic, Hungary, Romania, Slovakia, Russia, and Ukraineare of this type, as was Macedonia until the Higher EducationLaw of 2000 made provision for non-university professionalinstitutions. Whereas under the Soviet model, doctorates wereawarded by the academies of sciences, after 1990, all countrieshave (re)introduced Doctoral studies at universities.

Recently, a number of countries have adopted the two-levelsystem of studies. In recognition of the importance ofcomparability, mobility, and the message of the BolognaDeclaration, as well as the will to be part of the Europeanhigher education area, the sector is groping for ways toestablish two-level study systems. While so-called Bachelor'sDegree and Master's Degree programmes are on offer in manycountries, they often mean only that the former are short,usually practical-type programmes, and the latter, longer,research-oriented programmes, sometimes at different types ofinstitutions. "Master's" studies may also designate two orthree-year programmes offered for holders of Bachelor'sDegrees and also one-stream, five or six-year programmes,depending on the field (usually medicine, some engineeringprogrammes, and some social sciences or humanitiesprogrammes). Doctoral studies, offered at universities andoften in collaboration with external research institutes, in fact,constitute the third level of higher education stipulated in theBologna Declaration.

With respect to institutional types, the two-level highereducation system also exists in Albania, but only the nursingschool is non-university-level to date. At the same time,universities already offer many different levels and diplomas ordegrees, with one to two-year Master's Degree level studies


built on Bachelor's Degree qualifications, attained after four tosix years, followed by Doctoral studies. The implementation of acredit transfer system is impending.

In Bulgaria, the binary system predominates, with separatestreams for Bachelor's and Master's Degree level studies.However, in some fields it is possible to earn a Master's Degreeafter one year of study after completion of a Bachelor's Degree(4 years). The Bulgarian higher education system includes theeducational qualification degree, "Specialist in...", a degree ofprofessional higher education attained after at least threeyears of studies at a college; the Bachelor's Degree (first degree)attained after at least four years of studies in a university or ahigher school; the Master's Degree (second degree) attainedafter at least five years of training or at least one year after theBachelor's Degree in a university. The training for the DoctoralDegree is provided along research specialties following anearned Master's Degree in a university or in the BulgarianAcademy of Sciences, the Agricultural Academy, or otherresearch organizations.

For its polytechnics, Croatia has two-level higher educationprogrammes offered at the undergraduate level (2-4 years) andat the graduate level (1-2 years). University degrees can beearned after four to five years; Science Master's Degrees, aftersix to seven years, altogether.

In the higher education system in the Czech Republic,Bachelor's Degree programmes require three to four years tocomplete, and most Master's Degree programmes require fiveor five-and-a-half years. It is also clear from the fact that in1999, 71 percent of the programmes offered were Master'sDegree programmes that these are one-level studies. A varietyof choices are evident; however, most higher educationinstitutions offer Bachelor's Degree Programmes (however, notin certain fields, such as medicine or law). Nevertheless, two orthree-year Master's Degree programmes are accessible toholders of Bachelor's Degree diplomas. The Czech Parliamentis elaborating a law to introduce the two-level system acrossthe board.

PROGRAMME DESIGN 71

The government of Estonia issued a regulation in June2000, the "Higher Education Standard", which determinesBachelor's Degree (3-5 years) and Master's Degree (1-2 years)studies. Admittance to the latter is based on successfulcompletion of the former. The regulation sets down theminimum requirements for the two levels, as well as the lower,vocational ("diploma", three to four years), and the higher,Doctoral level. Some fields, such as human and veterinarymedicine or architecture, are covered in one stream of five tosix years.

In Hungary, a clear division is made between "colleges" and"universities". The Higher Education Act defines collegedegrees as comparable to Bachelor's degrees and universitydegrees as comparable to Master's Degrees (Hungarian HigherEducation Act LXXX/1993 [last amended in 20001, Section 24§ 95 (6)a) and b). So-called "complementary undergraduate"programmes, stipulated in the 1996 Amendment to the HigherEducation Act, are university (or teacher training) programmesfor college graduates. Yet with the exception of a limitednumber of bilateral agreements between specific universities totake on graduates from specific colleges, it is very difficult for acollege graduate to enroll in university studies at a comparableyear level. Various options for the introduction of two-levelstudies are being discussed. A preference is emerging forrunning both models together, which would retain full, five orsix-year programmes in certain study fields (e.g., medicine),and change the curriculum of others (e.g., computer science)to two-level programmes. Hungary also has two-yearvocational higher education programmes, one third of whichmay count toward college or university studies.

The Latvian Law on Higher Education Establishments laysdown the three levels of higher education studies: Bachelor'sDegrees, Master's Degrees, and Doctoral Degrees. The durationof a Bachelor's Degree programme may not be less than fouryears, while a Master's Degree can be attained after five toseven years, altogether. Some fields offer Master's-level studiesin one stream.


In 1993, Lithuania introduced Bachelor's Degree programmes,which, as of 2002, are extended from three-and-a-half to four-and-a-half years, and one-and-a-half or two-year Master'sDegree programmes, while some fields, such as medicalstudies, continue as six-year programmes. There are alsovocational one- or two-year programmes. Ordinarily, inuniversity-type institutions, there are three levels of studies: (i)undergraduate, leading to a Bachelor's Degree and/orprofessional qualification; (ii) graduate, leading to a Master'sDegree and/or a professional qualification; and (iii) postgraduate,leading to a Doctor's Degree or residency (for medical doctors)or "meno aspirantura" for artists. In non-university typeinstitutions, studies lead only to professional qualifications.

Macedonia also has the three levels of study: Bachelor'sDegree, Master's Degree, and Doctoral Degree. Bachelor'sDegrees (diplomiran) are issued after four to six years. Thelonger studies include fields such as medicine and are actuallyof the German-type university level. Master's Degrees(magister) are awarded after two years of study following aBachelor's Degree.

Poland, while having a two-level Bachelor's Degree (licencjator inzynier) and a Master's Degree system, retains one-levelstudies in certain fields. Universities offer professionallyoriented programmes of between three and four years afterwhich a two- to two-and-a-half years Master's Degreeprogrammes may be entered. In fields such as law, pharmacy,or medicine, there is one stream of studies of five to six years.

Romania is undergoing a fundamental structural reform ofeducation, begun in 1997. It soon introduced Master's Degreeprogrammes, understood as post-graduate studies after auniversity degree, which is considered a Bachelor's Degree.The reforms also extended to the restructuring of PhDprogrammes. Traditionally, short, three-year programmes havebeen offered in some fields at university colleges, which lead toa university-college diploma. Graduates with a university-college diploma may take entrance examinations to enter thethird year of long programmes. Licenta diplomas are degrees

PROGRAMME DESIGN 73

awarded in certain fields after four to six years, at universities,academies, and conservatories. Graduates with licentadiplomas may continue their studies in one-year specializedprogrammes or one- or two-year Master's Degree programmes.One-year Master's Degree study may count toward study forthe PhD degree.

Russian higher education may be of the non-universitylevel, not leading to an academic degree, and of universitylevel, leading to an academic degree. The former consists of"technical schools" (that offer courses in the humanities, thesocial and natural sciences) following three years of generaleducation and two to three years following secondaryeducation and colleges offering course programme runningthree-and-a-half to four-and-a-half years. The latter are offeredat universities, academies (for one area of study), andinstitutes (independent or part of a university or academy, forpost-graduate studies in any field). Russia has retained severallevels of Doctoral degrees. These are not conferred by theAcademy of Sciences but by the Academic Councils of highereducation institutions. There is no course work, onlyindependent research with a consultant followed by adissertation and its defense.

The one-level system is also prevalent in Slovakia, withwhat is considered Master's Degree-level diplomas representingas much as 95 percent of the graduates to date. At the sametime, Master's Degree studies that are based on Bachelor'sDegrees, as well as programmes leading to independentBachelor's Degrees, are gradually being introduced. The newHigher Education Act, in effect since 1 April 2002, definesBachelor's, Master's, Engineering, and Doctoral levels.Bachelor's degrees are earned in three to four years; Master'sDegrees generally require an additional two years, butdepending on the field, programmes may be offered only at theMaster's Degree level in four to six years. Engineers receive thetitle of "engineer in..." the particular field. Medical studies leadto the doctor's title with the speciality given. Both are "secondgrade", i.e., Master's-level studies. The third level is the PhD


and other Doctor's Degrees, usually requiring three years tocomplete.

Slovenia, on the other hand, already had a two-level systemin 1995 (The Higher Education Act, in, Zgaga and Jurkovid,1995) in its first higher education law (before Bologna). In fact,a statistic shows that "MSc"-designated programmes havebeen offered in this country since 1970 (Zgaga and Jurkovid,1995). Bachelor's Degree-level programmes are in part offeredby faculties of universities that were originally junior collegesthat awarded professional diplomas after two years of study.Following their integration into the universities in 1995, theyoffer three-year studies.

Higher education in Ukraine has four accreditation levels: (i)junior specialist courses of a maximum of three years ofduration (at vocational and technical schools); (ii) four-yearBachelor's Degree courses offered at colleges; (iii) specialistdegree courses lasting an additional two to three semestersafter a Bachelor's Degree, or five to six years as fullprogrammes offered at institutes, conservatories, academies,and universities; (iv) Master's Degree programmes, requiringthe defense of a research thesis, of six years for secondary-school graduates, of two to three years for those with juniorspecialist diplomas, or of one to two years on top of Specialistor Bachelor's Degrees offered at institutes, conservatories,academies, and universities. In addition, there are three-to-four-year Doctoral programmes leading to "Candidate ofScience" Degrees. The subsequent "Doctor of Science" Degreesare offered after completion of a dissertation followingindependent research and sometimes course work and areconferred by a High Certification Committee.

4.3.4. The Legal Setting for Programme DesignQuality assurance agencies are not directly involved indeveloping study programmes. Assessment does, however,check compliance with the legislative requirements regardingprogrammes. Indirectly, moreover, the assessment has aninfluence not only on the quality of programmes but their

PROGRAMME DESIGN 75

content as well, in that the evaluation report points out thestrengths and weaknesses of a programme.

All of the Central and Eastern European countries havepassed legislation governing higher education. This legislationdescribes, to varying degrees, the requirements and standardsfor study programmes, which is the condition for staterecognition of programmes. Within the boundaries of the legalframework, the higher education institutions in all countriesare free to design their own study programmes. Beforelaunching them, however, the programmes must generally bepassed by the respective accreditation body and approved bythe Ministry or Higher Education Council.

In Albania, the Ministry of Education and Science isresponsible for approving new educational programmes.Institutions are redesigning their programmes and structuresaccording to standards approved by the ministry.

In Bulgaria, new programmes must be approved by theAccreditation Council before they can be launched. In addition,various sections of the Higher Education Act describe therequirements for programmes, including examinationprocedures and the educational goals of programmes. Withinthis framework, institutions design their own programmes inaccordance with their internal regulations.

In Estonia, institutions design their own study programmesin compliance with the general legal framework. A governmentregulation of June 2000 governs the levels of study and ofstudy fields in eight categories. It also elaborates minimumduration, content, and staff qualifications as well asexaminations.

In the Czech Republic, the Ministry of Labour monitors theunemployment distribution of graduates so as to identify studyfields in which prospects for finding work are positive ornegative. Higher education institutions also conduct internalevaluation of their activities, including their programmes.

In Hungary, a government decree has grouped fifty-sixdisciplines into eight areas of science and the humanitiesunder which all study programmes are categorized. This


categorization has importance in that universities and collegesare defined by having the capacity to teach in a set number ofareas, and Doctoral schools, for example, may be launched ifthe university is accredited in the discipline to which theDoctoral school is assigned. Another set of government decreesconcerns national qualification requirements. They set downthe standards for the general content of a study programme,the examination schedule, and, in general, the knowledge to betransmitted by means of it. Qualification requirements for newstudy programmes or programmes to be launched at a highereducation institution are submitted to the HungarianAccreditation Committee for a quality check.

In Latvia, institutions also design their own programmes,but separate legislation governs education, higher educationestablishments, and the general principles of accreditation,and hence programmes standards. The 1998 Education Lawand the Law on Higher Education Establishments provide thebroad framework within which higher education institutionsmay develop their study programmes for all types and levels. Alicensing requirement must be met before programmes can belaunched.

The Lithuanian Law on Higher Education stipulates thedevelopment of subject area benchmarks describing therequirements for educational programmes, based on whichhigher education institutions develop their programmes andcurricula. The studies in higher education institutions arebased on the programmes included in the State Registry ofStudies and Programmes, which was established in 1997.Studies are organized in accordance with the curriculaprepared by institutions of higher education which abide bythe regulations.

In Macedonia, the Accreditation (also translated as the"Licensing") Board licenses new study programmes, which aredeveloped by higher education institutions. The legislativeframework for study programmes is laid down in the HigherEducation Act.

PROGRAMME DESIGN 77

Romania, in line with its education reform launched in1997, revised its list of university specialization. Recentlegislation, including two government decrees in 2000, assertthe right of universities to develop their own curricula.Curricular standards are regulated by a ministerial ordinance(1998).

Russia has state educational standards, which set down theminimum content of study programmes, student workload,and requirements students must meet to receive degrees. Thestate requirements are the core, to which regionalrequirements may be added. Based on these, higher educationinstitutions develop their own study programmes.

As described in the new Slovak Higher Education Act,higher education institutions can develop and implement theirown programmes. The Ministry of Education administers thesystem of study fields in which higher education institutions inSlovakia may provide education.

In Slovenia, the Higher Education Council of the Republic ofSlovenia, founded in 1994, is responsible for defining thestandards for the design of curricula.

The 1996 Law on Education in Ukraine sets down thegeneral standards of education. The Ministry is in the processof developing individual standards for each level of educationand speciality. Higher education institutions are awarded theright by law to design their own curricula in line with therespective legislation.

4.3.5. Quality AssuranceIt appears that in the Central and Eastern European countries,the predominant choice of "fitness for purpose" as the qualitystandard has emanated from the understanding that thepurpose of a higher education institution or programme isdefined in the pertaining legislation and that fitness ismeasured against the degree of compliance with the legislatedrequirements. That the mission statements of higher educationinstitutions have been factored into quality evaluations fromthe beginning of the quality assessment process (e.g., in


Albania, Bulgaria, Estonia, Hungary, Latvia, Romania,Slovenia) was an indication of awareness that autonomousinstitutions must become more proactive in designing theirfutures rather than in checking up on how much they live upto their self-defined missions.

In all the Central and Eastern European countries, qualityassurance was introduced with the following purposes:

to protect stakeholders;to define quality standards and levels;to assure, to varying degrees, the comparability of studyprogramme content and levels with those in WesternEurope; andin many countries, to control the quality of education atproliferating private institutions.

The expansion in the number of private higher educationinstitutions and consequently the concern for the quality ofeducation provision were factors in introducing qualityassessment, and specifically accreditation, at the system levelin most Central and Eastern European countries (Albania,Bulgaria, the Czech Republic, Estonia, Latvia, Romania,Russia, and Ukraine). As an example, in the Czech Republic,twenty private institutions have been established since 1999;however, they only cater to 2.1 percent of the students in thecountry (Individual country reports: Czech Republic, 2002<http: / /www. ceenetwork. hu >).

In Hungary, the movement to establish private highereducation institutions has not been as pronounced as in manyother countries of Central and Eastern Europe. A few initiativeswere thwarted by the Hungarian Accreditation Committee onthe grounds that the quality of provision was not up tostandard. Slovakia has only one private higher educationinstitution. At the same time, however, the country faces aproblem of maintaining the standard of quality of educationbecause of a rise, over the last ten years, in the number ofpublic universities. Slovenia, with only two universities, hadtwo "free-standing higher education institutions" in 1995, and

76

PROGRAMME DESIGN 79

seven, by 2001. However, the need for a methodical qualitycontrol mechanism is still under discussion.

But while the proliferation of private institutions has givencause for concern with regard to quality education, the PolishUniversity Accreditation Commission deals only with researchuniversities, which private institutions are not. The qualitycontrol of private institutions in Poland will likely be among theresponsibilities of the newly established National GeneralAccreditation Commission.

Steep increases in higher education enrollments have beenregistered in Poland and Hungary: a 3.5-fold increase between1990 and 2000 in Poland (WOjcicka and Chwirot, 2001) from10 percent to over 30 percent of the 18 to 23-year age cohort,and in the same period in Hungary (Statistical Yearbook ofHungary, 2000), from 112,000 students (Moko§in, 1999, p. 11)to 215,207 students (Individual Country Reports: CzechRepublic, 2002 <http: / /www.ceenetwork.hu>). In Poland,however, there is an oversupply of study places so that highereducation institutions must compete for new students.

The universities that are considered to offer the highestquality education are only beginning to face this problem asthe age cohort decreases. In the Czech Republic, "the totalnumber of students graduating from Master's degree studyprogrammes has risen by about 62 percent since 1990"(Individual Country Reports: Czech Republic, 2002, <http:/ /www.ceenetwork.hu >). Even so, the number of applications,at least in the public universities, still exceeds the number ofopenings.

In the other countries, the growth of student enrollments hashad similar effects. It must be noted, however, that many expectthe expansion to level off in the near future (e.g., Estonia) inresponse to demographic development. While most countriesstill have an oversupply of applicants for available study placesat system level, some higher education institutions or studyprogrammes are competing for students. Generally speaking,however, accreditation is not yet viewed as a publicly recognizedfactor for choosing a higher education institution.


It is evident that the preference in all the Central andEastern European countries for accreditation rather thanquality assessment alone occurred because, at the time oftransition, a priority was to establish some sort of qualitycontrol for the higher education sector. This priority wasdefined by higher education policy makers and stakeholders,both in government and in institutions. The question of howmuch external stakeholders acceded to this approach deservesmore analysis. It is likely that at a time when, with the start ofa new political system, new structures in higher educationwere developing at an accelerated pace, those "close to thefire", L e., the policy makers and those near them, had moresway than external individuals.

Nevertheless, the quality assurance process as a whole isperceived as serving all stakeholders including those in theCentral and Eastern European countries. The quality of highereducation is of general concern. The methodology for assuringit continues to be debated and reconsidered.

4.3.6. Quality Assurance Co-operation between Central andEastern European CountriesThe Baltic States Co-operation for Quality Assurance in HigherEducation was launched in 1994 within the Legislative ReformProgramme for Higher Education of the Council of Europe. Theproject, which involved meetings between rectors and ministryofficials of Estonia, Latvia, and Lithuania, led to a co-operationagreement in the area of quality assessment between theministers of education of the three countries. In accordancewith the initial aim of the project, the three countries set upnational quality assessment agencies in 1994 and 1995. Theproject has helped to clarify criteria, standards, and questionsof methodology. In 1996, the Baltic Higher Education Co-ordination Committee adopted its statute. The nine members ofthe Committee are nominated by the national qualityassessment body of each country, the respective Rectors'Conference, and the respective Minister of Education (thenominee of the latter being the ENIC representative of the

78

PROGRAMME DESIGN 81

country). The objectives of the Committee are to stimulate co-operation in institutional and programme assessment (toolsand procedures for international reviewers), stimulate studentand staff mobility, promote internationalization at highereducation institutions, develop common higher educationstandards, publish assessment outcomes, and organize jointconferences and workshops. The Baltic co-operation has beensuccessful in providing feedback into the quality assessmentprocesses of the three countries and especially in the exchangeof peer reviewers, which is ongoing.

The Network of Central and Eastern European QualityAssurance Agencies in Higher Education (a Central and EasternEuropean Network) was established in 2000, at the same timeas the Regional Sub-Network of the International Network ofQuality Assurance Agencies in Higher Education. Although thelink to INQAAHE was only an informal one, the membersdecided after a year to continue as an independent organizationin name as well. The intention of Network member agencies is tomutually share experiences and to foster co-operation, toexchange information, to recommend experts, to serve as aclearing house for quality assurance issues in Central andEastern Europe, and to assist each other in elaboratingmeasures for harmonizing quality assurance activities withinthe European higher education area. Member agencies are fromAlbania, Bulgaria, Croatia, the Czech Republic, Estonia,Hungary, Latvia, Lithuania, Macedonia, Poland, the RussianFederation, the Slovak Republic, and Slovenia. The applicationfor membership of the Austrian Accreditation Council will be theobject of a decision at the Assembly of the Central and EasternEuropean Network in October 2002.

In 2001, the European Network for Quality Assurance(ENQA), an organization overseen by the EuropeanCommission, extended the possibility for membership to theassociated countries of the European Union. Several Centraland Eastern European Network member agencies wereadmitted to ENQA at its General Assembly in May 2002. Aspart of the application process for membership, a discrepancy


regarding the concept of independence from governmentbecame evident between the European Union and the Centraland Eastern European agencies. As described above, theCentral and Eastern European agencies vary in this respect,from operating directly under ministries to acting as advisorybodies to the respective minister on accreditation decisions butmaking quality judgments autonomously. It appears that thedefinitions behind the common terminology require clarification,without disregard for the social and historical context.

4.3.7. Expected TrendsIt is very difficult to say in which direction the legislativeprocess regarding higher education is going. It seems that inthis sector, too, the initial feeling that "everything is possible",which pervaded public sentiment in the early days ofdemocratic government in Central and Eastern Europe, wasnot able to shed the frame of mind ingrained both by decadesof control and a more authoritative tradition relative to, say,northern Europe in general. Often, more autonomy in highereducation is at once coupled with detailed regulation, which,with the inevitable amendments to the new laws, is becomingmore intricate rather than more general. To steer highereducation institutions towards "defining their own mission andestablishing direct links to society and its changing needs"(Tomusk, 2000, p. 182); moreover, to give them the financialindependence that this objective requires is a prerequisite forthe organic development of the sector.

An outlook on higher education developments and currenttrends in each CEE country goes beyond the scope of thisexample. Hungary may serve as a case study. Here, a financialreform project for higher education has been commissioned bythe government, in which the weaknesses of the overlyregulated system have been defined (Matolcsy, 2001, pp. 19-20). These weaknesses include low cost effectiveness andefficiency; no provision for an external board to oversee aninstitution while the main governing body, the academicsenate, has a stake in upholding the status quo; educational

PROGRAMME DESIGN 83

provision decided by tradition and academic staff preferences;very limited own income and consequently little impact onsociety on the part of institutions; personal "entrepreneurship"on the part of the academic staff using institutional facilities,such as running after-hours courses or teaching at severalplaces with full pay from each; a frequently changing legislativeenvironment making long-term planning difficult; anddependence on the national budget law which earmarksspending and precludes innovative financial strategies(Matolcsy, 2002, p. 22).

Another development in the arena of higher education inHungary is a debate conducted in the commentary section ofthe weekly Elet es Irodalom [Life and Literature] (18 January2002 et seq.), launched with the publication of a book by twoHungarian academics (PolOnyi and Timar, 2001) on the state ofhigher education in the country. They cite an OECD (SIALS)survey on reading comprehension in twenty-two countries.While Hungarians generally are proud of the highly educatedindividuals they have given to the world of science, the studyputs Hungarian 14-to-64-year-olds in last place. The analysisconcludes that, on the one hand, a narrow elite is and wasindeed well educated, but that, on the other hand, educationin Hungary is geared toward cramming factual informationrather than toward problem-solving and innovative thinking.

A similar trend, incidentally, may be observed elsewhere inthe region (Tomusk, 2000, p. 183). Half a dozen educationresearchers, sociologists, economists, and others have joinedthe debate in the paper to reflect not only on the commonlyknown miserable state of the higher and public educationinfrastructure and the consequences of under-funding thesector overall, but on the structural distortions that havehardly been addressed up until now and certainly not asbroadly as in the current discussion.

With regard to the higher education sector, it appears thatthere is a natural curve of development, from launching a newlegislative system to "improving" it until it becomes saturated,to recognizing the failure of the model, to identifying its

81


weaknesses, and finally, to restructuring the system. Hungarycan be said to be approaching the peak of the curve.Legislation is still being "refined", but the core weaknesses ofthe model are already being identified.

With regard to programme design, a problem with theintroduction of the two-level study system is that theemployment market in the CEE countries is not ready toabsorb graduates of the first level. This situation may,however, be a consequence of the fact that the outcome incontent of the first-level degree has not been defined. So-called"Bachelor's" Degree holders, i.e., college graduates in dualsystems, are not faced with this problem. Therefore it can beexpected that as curricula are revised to accommodate thetwo-level structure of studies and bachelor-level graduates ofthis type enter the employment market, it will be ready.

It is generally the case for the higher education sector inCentral and Eastern European countries that the greatestproblem continues to be the lack of adequate financialresources. While that problem is the object of prevalentcomplaints in the West, the funding in real terms is far higherthere than in Central and Eastern Europe. Moreover, thecondition of the infrastructure and the cost-effectiveness of thesector overall lag far behind. In Central and Eastern Europe,the average salary of academic staff members is a fraction ofthat of their Western colleagues, which has far-reachingconsequences on many aspects of higher education. It shouldbe remembered, however, that enormous progress has beenmade in the last decade concerning structural developments inhigher education, as has been described, accompanied by asteady rise in the GDP in many Central and Eastern Europeancountries. Together, the two trends may provide the sector inthe region with opportunities to define its future, both in thecountries individually and as part of the European highereducation area.

8'

Part II

Annexes

83

Annex 1

Resources Chart fromthe December 2001 INQAAHE Survey

This chart summarizes published responses to two of thequestions in the December 2001 INQAAHE survey of itsmembers asking:

At whose initiative the external evaluation agency wasestablished?What, in the view of the Agency, was the relativeimportance of the purposes of their evaluations? (Thereplies were rated on a scale of 1-5, with 5 being mostimportant.)It indicates which providers of education are covered bythe evaluation processes of the Agency.It only lists respondents (from European Union memberstates) who are full members of INQAAHE.

They are also members of ENQA.

8487


Agency: name and Established Constituencycountry/region at

Accountability Accreditation Enhancement Benchmarking Info

AustriaOsterreichischer Government

Akkreditierungsrat initiative

Austria- Geschehalisselle GovernmentFachhochschulrat initiative

BelgiumVlaarnse Government

Hogescholenrad initiative

DenmarkDanmarks Government

Eualueringsinstitut initiative

FinlandFinnish Higher

EducationEvaluation Council GovernmentFINHEEC initiative

FranceConte National Government

d'Eualuation initiative

Germany JointAkkrediterungsrat Government

National andinstitutionalinitiative

Norway- Network NorwayCouncil

Portugal- INAFOP

NetherlandsHBO -Raad

Netherlands- VSNU

SwedenNational Agency

for HigherEducation

Private universities

Fachhochschulen

Hogescholen

All levels of education

Polytechnics anduniversities

All higher educationinstitutions

Accreditation agencies

Governmentinitiative

Government Initial teacherinitiative education programmes

Hogscholeninitiative

Universityinitiative

Governmentinitiative

United Kingdom JointQuality Assurance government/

Agency for Higher fundingEducation council and

highereducationinstitutionsinitiative

Hogescholen

Universities

Universities/university colleges andprofessionalaccreditation

Universities, colleges ofhigher education, andhigher educationfurther educationcolleges

5

5

3

3

3

5

3

5

5

5

Not until2002-2003

Not until2002-2003

3

4

5

5

5

4

5

3

5

5

5

5

4

5

3

3

4

2

5

3

4

4

1

5

BEST COPY AVAILABLE

85

Annex 2

Descriptions of Higher Education Institutionsand Quality Assurance in Central and Eastern

European Countries

2.1. ALBANIA

Albania has eight universities, three of which are in Tirana,and five, in other parts of the country. There are also twoacademies, for art and for sports, and one "higher school" fornursing. In addition, there is a military and a police academy.There are no private higher education institutions, eventhough the legislation makes their establishment possible. Aseparate law on private institutions is foreseen. All institutionsmust have state recognition to operate. The number ofapplying students exceeds the number of study places overall;however, as everywhere in the world, there is little interest incertain subjects and an overabundance of applications forsuch popular subjects as economics and law. The governmentdetermines the number of study places as proposed by theinstitutions. Tuition fees have recently been introduced, ofwhich the institutions can retain up to 90 percent with thebalance going to the central budget. These fees areaccompanied by a grant system that is based on family incomeand merit.

The Accreditation Agency of Higher Education, along withthe Council of Accreditation, was formally established in 1999,with the passing of the higher education law and specifically adecision by the Council of Ministers. However, Albania is justlaunching the accreditation process with the assistance of ajust-ending, three-year TEMPUS project. The aim is for thesame evaluation team to assess similar programmes in thewhole country. At programme level, the desire is to assess botheducational and research activities. The scope of assessmentsis very broad, encompassing both institutional and programme

86 89


levels, and within these, an extensive number of factors, fromgoals and aims to the opinion of society at large. One canexpect that once the agency has gained some experience thescope will become more focused. The Agency aims to evaluateinstitutions and programmes every four years.

The Accreditation Council has eleven members. They aredistinguished academics, including two representatives fromindustry and one from the Ministry of Education and Science,who are named by the Council of Ministers on the proposal ofthe Minister of Education and Science. The AccreditationCouncil approves the criteria of self-evaluation and externalevaluation and decides on the accreditation report. TheAccreditation Agency has a staff of five, one director, threeprogramme officers, and one administrator. The programmeofficers participate as secretaries in the review visit andcontribute to writing the accreditation report. The report,which describes the strengths and weaknesses of theevaluated institution, is submitted to the Council ofAccreditation, and the decision of the latter is presented to theMinister, who has the final decision on granting accreditation.The report is published but distributed only within the sector.

The Agency prepares the criteria for evaluation to beapproved by the Council. The Agency issues two manuals, onefor self-evaluation, used by the institutions, and the other forthe review team. It is interesting to note that the Agency trainsthe self-evaluation team at the institutions prior to launchingthe self-evaluation exercise. Equally interesting is the provisionin the law that the self-evaluation should create the basis for adevelopment plan for the institution. The Agency also trainsthe external reviewers who, as stated in the law, come from theorganizations of employers, industry, and other universities.

2.2. BULGARIA

Bulgaria has forty-nine higher education institutions thatinclude thirty-seven state universities and specialized higherschools, four private universities, two state, and six privatecolleges. All higher education institutions operating in Bulgaria

87

QUALITY ASSURANCE 91

must be recognized by the National Assembly. The training forDoctoral Degrees provided in connection with researchspecialities may also be offered by the Bulgarian Academy ofSciences, the Agricultural Academy, and the other researchorganizations in the country.

The National Evaluation and Accreditation Agency conductsinstitutional and programme accreditation. The law stipulatesthat programme accreditation can only be conducted ininstitutions that have already been accredited. Programmeaccreditation is conducted for programmes leading toeducational and professional degrees ("Specialist in..."),educational qualification (Bachelor's and Master's Degrees), aswell as for education and research (Doctoral Degrees). TheAgency also accredits new programmes to be launched inaccredited institutions. Doctoral education may be conductedat universities and research organizations which have receivedthe highest of four ratings in their accreditation of researchspecialities for Doctoral Degrees.

The Agency evaluates projects for the opening andtransforming of a higher school, a faculty, an institute, abranch, or a college as well as for the opening of newspecialities.

The Accreditation Council consists of nine recognizedacademics, including two representing the Minister ofEducation and Science. The Council Chairperson is also theChairperson of the Agency. The Council is appointed for a six-year term. The Council sets up standing committees of three toseven members appointed by the Chairperson for a three-yearterm. The role of the standing committees is to review thereports of the expert groups that have conducted site visits.The Standing Committee evaluates the report on the strengthsand weaknesses of the institution or programme described andproposes an accreditation grade.

The Agency has a full-time staff of twenty persons who havean administrative and coordinating role. They do notparticipate in the review visits, but do contribute to theaccreditation report. The Agency develops its evaluation

92 C. CAMPBELL and C. ROZS1VYAI

criteria and procedures in accordance with the law and otherlegislation, both for institutional and programme reviews. Italso organizes workshops for training its review teams. Theresults of accreditation are published annually in the StateGazette, while the reports are published periodically in theBulletin of the Agency and are distributed to higher educationinstitutions, research organizations, ministries, and other stateorganizations.

2.3. CROATIA

Croatia has four multi-faculty universities, which also includeacademies and polytechnic schools. In 2001, the Ministry ofScience and Technology decreed that polytechnics might alsobe established as independent institutions. Consequentlyseven polytechnics and eight independent schools have beenestablished.

The national system of quality assurance is set down in the1993 Higher Education Act. It was amended in 1996 toseparate the functions of assessment and accreditation. A newdraft law is currently under discussion.

The Ministry initiates the accreditation of studyprogrammes and makes accreditation decisions, based on therecommendations of the National Council for HigherEducation. The Council has eighteen members and aPresident. On nomination by the Rectors Conference andhigher education institutions, the members and the Presidentof the Council are appointed by the Parliament of the Republicof Croatia for a four-year term. The role of the Council is topropose the experts, and following their assessment, toformulate a recommendation to the Ministry regardingaccreditation. The same review team conducts the review of allprogrammes in the same general subject area in the country.

Higher education institutions are evaluated without a"formal" accreditation decision. Nevertheless, the Council maypropose the closing of an institution that does not meet qualitystandards. The Council devises guidelines for self-evaluationand the framework for review, but the Ministry approves the

99


procedures. The Ministry has the function of a quality agencyin that it provides the logistics for reviews. The review team isappointed by the Ministry on the proposal of the Council. Itconsists of academics, some of them (mostly of Croatian origin)are from abroad. It submits its evaluation report to theministry, which forwards it to the Council. The Council thendebates the content of the team's report, based on the self-evaluation report, the opinions of professional organizationsand international experts, and may accept it as is or requireadditional information. The Council then formulates one offour proposals: from ruling that the institution meets thequality standards, to requesting the implementation ofchanges by a given date, to changing the status (funding) ofthe institution, or to closing it. The Ministry makes the finaldecision.

The tasks of the Council, in addition to quality assessment,also extend to commenting on higher education policy andstrategy and proposing to initiate new or to abolish existingcourse programmes and higher education institutions.

2.4. THE CZECH REPUBLIC

The Accreditation Commission of the Czech Republic wasestablished by means of the Higher Education Act in 1990. Itstwenty-one members, including two from business and threefrom foreign countries, are appointed by the Government, onthe nomination of the Minister of Education, Youth, andSports, from higher education institutions, the Academy ofSciences, and the Research and Development Council of theGovernment. The Minister must consult with representativebodies of the three groups. Commission members areappointed for four- or six-year renewable terms. Additionally,chairpersons of permanent working groups cover disciplinesnot represented by commission members. Ministryrepresentatives may attend Commission meetings. A personfrom the applying institution or programme may be invitedwhen the Commission discusses the application.

90


In 1999, a Secretariat was set up and has since expandedto include a staff of six, including the Secretary. It istechnically a unit of the Department for Science and HigherEducation of the Ministry of Education, Youth, and Sport. TheCommission Secretary is appointed by the Minister on theproposal of the Chairperson of the Commission. The staffperforms a coordinating and administrative function, while theCommission members, who meet to discuss and vote onapplications and higher education issues "at least three timesa year" (The Statute of the Accreditation Committee, Ministryof Education Youth and Sports of the Czech Republic. Article 2

p. 1) but in fact more often, carry the full load of work inpreparing the evaluations.

At the end of 2001, the Czech Republic had forty-eighthigher education institutions of which twenty-four were public,four were state institutions (one police academy and threemilitary institutions), and twenty were private institutions. Thepublic and military institutions are universities with a researchorientation and offer Bachelor's, Master's, and Doctoralprogrammes. The Police Academy and all the privateinstitutions are non-university vocational institutions offeringBachelor-level degrees.

In 1998, the Higher Education Act was amended so as togreatly expand the mandate of the Commission. While earlier,only new programmes were accredited without provision forongoing quality control, evaluation cycles were nowintroduced. The scope of the commission is to evaluate"applications for the accreditation of study programmes and...for the accreditation of procedures for obtaining venium docenti(habilitation) or procedures for appointment of professors in agiven field, ...[the] establishment, merger, amalgamation,splitting, or dissolution of a faculty of a public or state highereducation institution, granting the state permission for a legalentity desiring to operate as a private higher educationinstitution, determining the type of higher educationinstitution" (The Statute of the Accreditation Commission,Ministry of Education Youth and Sports of the Czech Republic.

91


Article 2 [3] and [4b-d], pp. 1-2). The AccreditationCommission also has an indirect influence on higher educationpolicy in that it is asked to "assess other issues concerning thesystem of higher education presented to it by the Minister andexpress its standpoints [regarding] these issues" (The Statuteof the Accreditation Commission, Ministry of Education Youthand Sports of the Czech Republic, Article 1 [1] b, p. 1).

The commission and its working groups devise their ownstandards for evaluation and accreditation, based on theHigher Education Law and pertinent legislation.

While the denial of accreditation can lead to the closing ofan institution or programme, the commission invokes thisright only in cases in which no hope for improvement isperceived. It may also propose to restrict accreditation so thatno new students are admitted, or withdraw accreditation sothat no degree may be awarded until the programme isimproved.

Reports include a description of strengths and weaknessesand contain a number of specific provisions for improving thequality of a programme or institution. Higher educationinstitutions are required by law to have internal quality controlmechanisms and to publish the outcomes of their reviews oncea year (a provision in the 1998 amendment to the Law). In itsexternal evaluation, the Accreditation Commission considersthe capacity of the given institution to control its quality.

2.5. ESTONIA

Estonia has six public universities, seven state applied highereducation institutions (polytechnics), six state highervocational institutions, eight private higher vocationalinstitutions, and twenty-two private higher educationinstitutions. They are required to obtain a license from theMinistry of Culture and Education to operate and may issuestate-recognized degrees only for study programmes that havebeen accredited.

The Higher Education Accreditation Center was establishedby the Government in 1997 as a foundation. Its primary


activity is to organize the accreditation of course programmes;however, it has also reviewed some institutions. The Centerorganizes research evaluation separately.

The Center functions as the coordinator for accreditationprocedures with a staff of three: a Head and two AssistantHeads, one for research and the other for evaluation. The roleof the Center is to identify experts and to organize their reviewvisits. The review teams are made up of only foreign experts.The review team schedules are organized so that the team isable to review all the programmes in the broad given field inthe country within the one week allotted to their visit, whichincludes writing the review report.

The Higher Education Quality Assessment Council ofEstonia was established by the 1995 Law on Universities, to"operate under the jurisdiction of the Ministry of Culture andEducation". It has twelve members, including members fromother Nordic countries, who are "representatives from amongstresearch and development institutions, state foundations thatfinance research and development activity, and fromprofessional associations". They are appointed by theGovernment for a three-year term that may be renewed once.The Council votes on the review team reports and isresponsible for devising the criteria and methodology foraccreditation which are approved by the Minister. Governmentregulations set down the general standards for curricula andlevels of study. The accreditation decision is either for fullaccreditation (valid for seven years), conditional accreditation(valid for two years), or not accredited. If over one-third of theprogrammes of a university are not accredited, its license isrevoked. (For research evaluation the possible grades that areawarded are excellent, good, satisfactory, or unsatisfactory).The judgments of the Council are advisory to the Minister ofEducation, who makes the final decision . regardingaccreditation of a programme or an institution. The Ministerhas veto power over Council decisions but may not changethem and is required to explain his or her reasons. Reportsdeal with the strengths and weaknesses of the evaluated

93


programme or institution. They are not widely diffused, butprinted copies are distributed to universities and studentorganizations.

2.6. HUNGARY

The Hungarian Accreditation Committee was established asthe result of the first Hungarian Higher Education Law in1993. Its scope is to accredit higher education institutions,which, in the first round, that ended in 2000, it did byassessing all study programmes at given evaluatedinstitutions. In the coming round of evaluations, the focus willbe more on the institution as a whole, and only a selection ofprogrammes will be evaluated. The evaluation of entire fields atone time and by expert reviewers from a consistent pool is alsobeing considered, the purpose being the comparability of studyprogrammes in the country. The Hungarian AccreditationCommittee adopted a strategic plan in early 2002 and isproposing a new procedure in consultation with highereducation institutions. The Committee conducts programmeevaluation according to the standards devised by its expertcommittees for each discipline.

In addition, the Hungarian Accreditation Committeeevaluates applications to establish and grant state recognitionto higher education institutions and faculties, on new studyprogrammes, and on national qualification requirements for alldegree programmes taught in Hungary. The Committee alsoadvises on habilitation regulations. According to theamendment of 2000 to the Higher Education Act, theCommittee is also charged with evaluating applications forprofessorships.

Hungary has eighteen state universities, twelve colleges, fivechurch-maintained universities, twenty-one church-funded/managed colleges, and six private colleges. Hungaryhas a dual tertiary system with universities providingresearch-based and research-oriented education and withcolleges offering applied training. Student movement from onesystem to the other, although provided for by law, has been


difficult and infrequent, except in the few cases in whichbilateral agreements between certain institutions facilitate it.The sector is discussing a number of options to introduce theBachelor's/Master's model. The introduction of an ECTS-compatible credit system is underway.

In order to receive state recognition and to award degrees,private higher education institutions must undergo accreditation.Church-maintained higher education institutions receive statefinancing similar to state institutions and are accredited, butthe mandate of the Committee only pertains to secularprogrammes.

The decisions of the Committee regarding doctoral schoolsis binding. All its other decisions are advisory, but in case theMinister of Education deviates from the assessment of theCommittee, he or she must publish the reasons for his or herdecision. The scope of the Hungarian Accreditation Committeeis accreditation oriented toward improvement. Accreditation forhigher education institutions is contingent on compliance withthe Higher Education Act. In the first round of institutionalaccreditation, the evaluated programmes were graded on afour-level scale. Roughly one third of the programmes wereawarded accreditation for a specified period with prescribedrequirements to be met by that date. These requirements werereviewed in a monitoring procedure. In the next institutionalaccreditation round, programmes will be given one of threegrades: "yes" (may be accredited), "conditional yes" (may beaccredited but is required to meet stated requirements by astated date), and "no" (may not be accredited).

In the first round, only the final accreditation decision waspublished along with a brief explanation. In the followinground, the full report will be published.

The Hungarian Accreditation Committee consists of thirtyfull members delegated by higher education institutions,research institutes, and professional organizations. Theyreceive their letters of appointment from the Prime Minister fora three-year term, which may be renewed once. There is onenon-voting student member. Additional non-voting members


are appointed in order to cover all the main disciplines. AnInternational Advisory Board, mandated by law, reviews thework of the Committee at an annual meeting and makesrecommendations which are also sent to the Minister.

The Secretariat has a staff of sixteen programme officers,four of whom are part-time, and nine administrative staff. Theprogramme officers are assigned several expert committees thework of which they prepare and guide from inception tocompletion of the report. In addition, they are responsible forseveral institutions; they prepare and participate in site-visits;and they assist in writing the accreditation report.

In 2000-2001, the Hungarian Accreditation Committee wasevaluated by an international review team under the auspicesof the European Rectors Conference (CRE, now the EuropeanUniversity Association EUA). Its report, together with the replyof the Committee, have been published (<http://www.mab.hu/english/ a_links.html>) .

2.7. LATVIA

There are forty higher education institutions in Latvia, ofwhich twenty-three are state institutions and seventeen areprivate. All five universities are state institutions.

The Higher Education Quality Evaluation Center, Ltd., isowned by shareholders. These are the Ministry of Educationand Science and five higher education institutions. A five-member board supervises its activities. One member of thisboard is appointed by the Ministry. The shareholders elect theDirector of the Center who hires the staff. The staff includestwo academics and consists of four full-time equivalentpositions, including the Director, which are filled by two full-time and four part-time employees.

The Center has a primarily coordinating function. It sets thetimetable for reviews in consultation with higher educationinstitutions and brings together the evaluation teams asrequired by the Council of Higher Education (for institutions)or the Accreditation Commission (for programme assessment),in consultation with the institutions or persons responsible for


programmes to be reviewed. The staff of the Center does notparticipate in the visits. While there is no national-levelassessment of all study programmes in a field, the Center doesattempt to use the same experts to evaluate similarprogrammes. The Center has guidelines for self-evaluation andexternal assessment, but the evaluation team has greatautonomy in deciding on its own methodology and procedures,so long as they comply with the legislated standards. Theteams are made up only of academics, one from Latvia and atleast two foreign team members. Only for college-levelprofessional education may all the experts be from Latvia. Thereview team reports are published without changes, and theteams are fully responsible for their content. The evaluationreport is sent directly to the higher education institution aswell as to the Council for Higher Education and theAccreditation Commission, and is published.

The Council for Higher Education is set up by Parliament,and the Accreditation Commission, by the Ministry. The twobodies approve the review teams set up by the Center anddecide whether accreditation should be granted. The Councilfor Higher Education Institutions and the Commission forStudy Programmes approve the reports of the independentevaluation teams organized by the Center. They submit theirdecisions to the Minister, who may refuse to grantaccreditation simply on the grounds of a legal violation.

The Council for Higher education is an independentinstitution. It has ten members, including its full-timeChairperson who is prohibited from holding a leadershipposition elsewhere. They are delegated by bodies representinghigher education, research, professional organizations, andindustry. The Minister of Education and Science also attendsthe meetings. The members are approved by Parliament forfour-year terms. In addition to adopting decisions on theaccreditation of higher education institutions, the Councilundertakes a wide range of duties regarding higher educationstrategy, quality, and financing.

9 7


The Accreditation Commission is set up by the Minister toassess study programmes and to propose an accreditation .

decision to him or her. Its members are delegated by highereducation institutions, research institutes, and the Ministry,as well as by unions, professional associations, and otherministries.

Institutions are granted permanent accreditation statusexcept in cases in which improvement is needed. In suchcases, accreditation is given for two years. New studyprogrammes must receive accreditation within two years ofstarting activities. Study programmes are accredited for sixyears, but may receive a two-year accreditation once if theyneed to be improved. The Council or Commission may alsodecide that accreditation should be refused. After repeatednegative accreditation, the institution or study programmemust be reorganized or closed, depending on the severity of theproblems. Negative as well as positive decisions are published.

New study programmes must be granted licenses beforethey can operate and must receive accreditation within twoyears. Study programmes must be accredited every six years,at the least.

2.8. LITHUANIA

During the 2001-2002 academic year, there were twenty-twostate and thirteen non-state higher education institutionsoperating in Lithuania. They include nineteen university-typeinstitutions (fifteen state and four non-state) and sixteen non-university-type institutions colleges (seven state and sixnon-state).

The Lithuanian Center for Quality Assessment in HigherEducation was established, and is financed, by government,but the agency functions independently. The Minister forScience and Education appoints the Director of the. Center ona competitive basis to serve a five-year term.

The Agency staff organizes institutional and programmeassessment. In addition, it conducts research evaluationseparately. The activities of the Center focus on quality


assessment and improvement, even though the Centerforwards its assessment reports to the Ministry.

The Ministry sets up a Council of Experts. The latter isresponsible for summarizing the conclusions of theassessments. The external evaluation schedule for studyprogrammes, institutions, and the subdivision is determinedby the Center in consultation with the Rectors' Conference, theCollege Directors' Conference, the Directors' Conference ofState Research Institutes, and the Science Council. Theassessment plans are approved by the Minister. Institutionaland programme accreditation decisions are taken by theMinister of Science and Education, based on the report of thereview team, approved by the Council of Experts. Accreditationis mandatory.

The agency functions as the coordinator of reviews. It alsoorganizes seminars for self-assessment. The regularassessment of education programmes began in 1999. TheAgency appoints the review teams in consultation with theinstitution to be assessed. The review teams, which consist ofacademics and non-academics, prepare their reports for theevaluated institution. For the subject areas (especially thesocial sciences), international expert groups are formed. One ortwo local experts are included in each group. The review teamsmay or may not be the same for similar programmes. However,the same review team is used to evaluate all studyprogrammes in the same general field in all higher educationinstitutions. Programmes, which receive temporary orrestricted assessment, are re-evaluated within two years. Theteams hold responsibility for the report, and only a summary ispublished. The Center forwards the report to the evaluatedinstitution and to the Council.

While assessing the research and development of researchand higher education institutions (or their subdivisions),efficiency of research activity may be assessed considering notonly quantity indicators, but also quality parameters forresearch efficiency. The most important research results aredetermined following a debate on their significance. After every

9 9


assessment, the Agency publishes the research anddevelopment efficiency indicators of institutions of researchand higher education taken from their self-assessment reportsand supplemented with corrections according to the results ofexternal assessment.

The qualitative assessment of the whole institution ofresearch and higher education embraces all its academicsubdivisions as well as the whole combination of all itsacademic (educational and research) activities. Both aninternal and external assessment of the whole institution arebased on the previous assessment reports concerning theeducational programmes as well as those of research indifferent subject areas, with the reports being summarized,and strategic matters, considered. Only in the case of a smallinstitution may a general qualitative assessment of researchand higher education coincide with an assessment of itsresearch or study programmes. Summarized qualityassessments include grading on a four-point scale.

It is interesting to note that the Agency shares its offices,administration, Director, and Deputy-Director with theNational Recognition of Qualifications and Information Center,the ENIC/NARIC, which allows the organizations to share theirpools of experts. The Agency works with a staff of eight, butshares its administration with the Recognition Center. Thequality assessment unit has a staff of four, including its Head,an academic holding a part-time position.

2.9. MACEDONIA

Macedonia recognizes four types of higher educationinstitutions: universities, faculties, art academies, and highervocational schools. Currently, the country has two state-recognized universities. There is also one private, recognized,foreign university. However, altogether, there are thirty-fivehigher education institutions. All must have licenses tooperate. For the most attractive programmes, there are moreapplicants than available study places.


Currently, a project with international co-operation, theinvolvement of the evaluation agency, the Ministry ofEducation and Science, and the two state universities, isunderway to design the quality assessment landscape ofhigher education in the country. The main focus is onintroducing self-evaluation into higher education institutions.

After ten years of preparation, the Law on Higher Educationwas adopted in August 2000. It called for the internationalcomparability of higher education and introduced qualityassessment and accreditation. Both institutions andprogrammes are to be evaluated and accredited.

The Law set up two bodies for quality assessment, theAccreditation Board (also translated as "Licensing Board") andthe Evaluation Agency. The Board is an independent body offifteen academics delegated by the Inter-University Conference,the Academy of Science and Art, and the Government for afour-year term, renewable once. It is responsible fordetermining whether or not a higher education institution iscomplying with the general requirements set for it. It decideson the accreditation of new study programmes, keeps recordson licensed higher education institutions, and assesses thequality of higher education in general. The Board mayestablish expert committees.

The Evaluation Agency has nine members who areacademics proposed by the Inter-University Conference andelected for a four-year, once renewable term by the Board bysecret ballot. They represent proportionally the number offaculties and scientific institutions involved in highereducation at all levels. The Agency functions as anindependent and separate body but is convened by thePresident of the Board. It is responsible for conducting externalevaluations based on guidelines adopted on the proposal of theInter-University Conference. It should also review the quality ofthe academic staff in the higher education institutions of thecountry. It must monitor the quality of work at higher educationinstitutions. Once every five years, it assesses the quality ofhigher education institutions; it gives recommendations for


improvement; it assesses the quality of the academic staffthroughout the country and makes suggestions forimprovement; it publishes its reports on assessment resultswhich it transmits to the institutions, the Board, the Ministry,and the Parliament. Based on its findings, it recommends theextension or the withdrawal of the accreditation of giveninstitutions.

The law requires higher education institutions to conductself-evaluations via committees set up for this purpose.General Guidelines for Evaluation and Evaluation Proceduresat Universities have been published, and the AccreditationBoard as well as the Evaluation Agency have issued theirrespective by-laws.

2.10. POLAND

In Poland, the proliferation of private higher educationinstitutions has been enormous. Over 200 such institutionswere established over the last decade (WOjcicka and Chwirot,2001, p.'7). This result, in part, contributed to theestablishment of a number of accreditation bodies. BusinessSchools for Quality of Higher Education (FORUM) was set upin 1994. The Accreditation Commission of Higher VocationalEducation was established by the Law on Higher ProfessionalSchools in 1997 and by a ministerial decree of 1998. TheAccreditation Commission for Medical Universities was set upin 1997; the University Accreditation Commission, in 1998; theAccreditation Commission for Schools of Physical Educationand one for Pedagogical Schools, in 1999. The AccreditationCommission of Technical Universities, one for AgriculturalUniversities, and the Accreditation Commission of theFoundation for the Promotion and Accreditation of Studies inEconomics were set up in 2001. The aim of these commissionsis to control the standard of study programmes in their givenfields.

The General Council on Higher Education was establishedas a result of the Polish Higher Education Law of 1990. One ofits functions was to set the standards for new higher education


institutions, faculties, and study programmes. But "deprivedof... organizational support and adequate financial means [it]has not been able to conduct systematic quality evaluation ofprovision in the hitherto existing and newly establishedinstitutions of tertiary education" (VVOjcicka and Chwirot,2001, p. 7). In the 2001 amendment to the Higher EducationLaw, the likely successor to the General Council, the NationalGeneral Accreditation Commission, was established. It beganto function in January 2002 and is expected to work outthreshold standards for quality. "In this situation, it seems thatone of the more important tasks of the State AccreditationCommission [now translated as "National General AccreditationCommission "] is, in addition to seeing to the authorization ofactivities undertaken by the academic milieu via its officialcertification and limited co-ordination role, also assuring theprovision of certain minimal quality standards by each tertiaryeducation establishment. Such activities will allow for (i)

developing important social initiatives in the academic milieupossessing self-regulatory character, (ii) distinguishing andpromoting institutions granting excellent provision, and (iii)accelerating the inclusion into the evaluation system of allhigher education institutions in Poland" (WOjcicka andChwirot, 2001, p. 8).

Following years of discussion about setting up a qualityassurance system, the Conference of Rectors of PolishUniversities (CRPU) took the initiative and established theUniversity Accreditation Commission (UAC) in 1998. Itsmission is the "enactment of quality standards in educationcomparable to those prevailing in the European Union; theupgrading of the quality of education; [and the] promotion ofhigh-quality courses of studies and of schools offering them" (2001,<http: / /www. main. amu edu. pl / -ects/uka/uka-eng.html>) . Themembership is made up of seventeen "Vice-Rectors orrepresentatives of the schools-signatories of the Agreementappointed by the Rectors of their schools and one persondesignated by CRPU. CRPU appoints the President of the


UAC. The tenure of UAC members is three years"(<http: / /www. main . amu.edu pl / -ects/uka/uka-eng.html>) .

The UAC specifically aims to set up a system ofaccreditation for degree programmes and to establish commonstandards of quality. Other institutions that are not membersof the agreement may apply for accreditation of theirprogrammes by the UAC but the UAC deals only with researchuniversities. Some private institutions have, in fact, undergoneaccreditation by UAC. In their evaluation, for which theapplication is voluntary, the UAC looks at the curriculum, theacademic qualifications of the teaching staff, the student/teacher ratio, the adequacy of infrastructure, including libraryholdings and laboratories, as well as research or scholarlycontacts, and the existence of student evaluations of thetaught courses.

At least five universities must apply for the accreditation ofa particular study field in order to launch the accreditationprocedure. The university applying for accreditation proposesfive experts with documented qualifications to participate inthe evaluation (two from the university and three from anotheruniversity). An expert group of six to twelve persons, whosetask is to establish specific standards for evaluating the studyfield, and a three- to five-member evaluation team for eachapplying programme, that reviews the programme and writesan evaluation report, is set up. The evaluation team mayrecommend that the UAC grant accreditation (for two to fiveyears), prescribe certain requirements before accreditation maybe granted, or refuse accreditation. The expert group reviewsand updates its standards at least every two years. The UACemploys two full-time and one part-time staff members. Theseperform coordinating and administrative functions.

The proliferation of accreditation commissions hasgalvanized their members into establishing an umbrellaorganization. It is currently developing. Its members are thechairpeople of the seven academic accreditation commissions.Its tasks are to co-ordinate the activities of its seven membercommissions, particularly to observe that accreditation


standards and procedures are correct and comparable, andthat fields of study offered by different schools are accreditedunder the same standards and procedures. It should alsorepresent the seven commissions vis-à-vis the National GeneralAccreditation Commission and in international organizations.Finally, it is expected to speak out publicly on issues ofaccreditation and the ranking of higher education institutions.

2.11. ROMANIA

Romania has fifty-seven public and eighty-three private highereducation institutions. Higher education institutions can beuniversities, academies, polytechnics, institutes, or colleges.Already in 1990, Romania had 101 private faculties, a numberthat more than doubled in the next decade, reaching 221faculties by 2000, with over 30 percent of students in highereducation attending private institutions. With tuition feesbeing charged in public institutions, the private sector is highlycompetitive. However, accreditation is mandatory and veryrigorous, and the 1993 law on accreditation requires, amongother things, that at least 51 percent of students at a privateinstitution pass the state examination at a public university.By 2001, only eight private higher education institutions hadbeen accredited, thus authorized to grant state-recognizeddegrees as well as to be eligible for state funding.

The National Council for Academic Assessment andAccreditation has nineteen to twenty-one members appointedfor four-year terms. They are nominated by higher educationinstitutions, the list of which is published in the daily press soas to invite public challenges. From the resulting short-list, theMinister of Education proposes the final membership list to thecabinet for approval by the Parliament. The members elect apresident, a vice-president, and a secretary, on the proposal ofthe Minister.

The Council sets up fifteen expert committees of seven tonine members, who are appointed for a four-year term. Theexpert committees cover the basic disciplines and includesome foreign members. They may involve additional experts for

0 5


reviews who may be non-academics and/or foreigners. TheCouncil has a full-time staff of fifteen professionals and tenadministrators and coordinators.

Standards and criteria for accreditation are set down in the1993 Law on Accreditation. Quality assurance encompassesongoing internal evaluation of higher education institutionsand external assessment. The latter begins with the licensingof new institutions, accreditation three years after licensing,and repeated assessment every five years.

New programmes also must be initially licensed. Accreditationis awarded after several assessments. The final decision onaccreditation is a parliamentary decision, if it concerns a newinstitution, and governmental, if it concerns a new programme.The periodic institutional assessment covers all studyprogrammes. The institutions that do not meet therequirements are given a year to improve.

The reports of the initial review team are confined to factsand are first given to the university for verification. The nextstage of the report is written by the expert committee and isbased on the self-evaluation report of the institution and thereview team report. The final report is discussed and approvedby the full Council and may include recommendations for theuniversity. The report is then submitted to the Minister whomay return it for review to the commission. The report of theCommission and the comments of the Minister are sent to theinstitution, and if the latter agrees, may be made public. Thefinal decision on accreditation is made by the Government (forprogrammes at accredited institutions) or by the Parliament(for new institutions).

2.12. THE RUSSIAN FEDERATION

The Russian Federation, embracing fifteen post-Soviet stateswith 89 regions, has over 1,100 higher education institutions.Of these, 431 are licensed non-state institutions providing 9percent of the student population. State institutions werefounded before 1990. There are also municipal and privateinstitutions that were founded after that date. Higher


education is offered at four levels: vocational, non-university,university, and postgraduate, including Doctoral level.

Quality assurance in Russia is based on three consecutiveprocedures, licensing, state accreditation, and attestationbased on state standards. The process was made the object oflegislation in the 1992 Law on Education. State standardsdescribe minimum programme content, workload, andrequirements. Licensing, granted by state or local authorities,gives institutions the right to offer study programmes in thefield and at the levels for which licensing is granted. Theexperts check whether or not the conditions for education havebeen met, whether the institution and its programmes meetlegislative and infrastructural requirements, and whether staffqualifications are adequate. Licensing is an administrativeprocess conducted by external experts and federal and localeducational authorities.

Accreditation is the legislated product of a decree of theState Committee for Higher Education. It confers on both stateand private institutions the right to issue state-recognizeddegrees. Accreditation testifies type of institution (professionalschool, technical school [also humanities and social sciences],college, institute, academy, university) and its level (generaleducation, vocational education, non-university or universityhigher education), and whether or not the contents and qualityof training comply with the State Educational Standards. Theprocess of accreditation combines institutional and programmeassessment and extends to all the programmes of theevaluated institution. Accreditation is granted for a maximumof five years for institutions and programmes and encompassesself-evaluation by higher education institutions and anexternal peer review. The review team, made up of academicsand non-academics, writes a report, which forms the basis foraccreditation. Non-universities are accredited by stateauthorities in the region in which the institution is located.Universities are accredited by the Ministry of General andProfessional Education.


Attestation is the task of an expert commission, which alsoinclude non-academics. Its purpose is to assess the knowledgeof graduates as compared to the State Educational Standards,and is regulated by the 1994 decree of the State committee forHigher Education on State Final Attestation of Graduates ofHigher Education Institutions. Attestation is a rigorous exitexamination procedure for graduates, which includes a finalexamination in the graduate's discipline, a final examination inthe specialization, and the defense of a thesis or project,conducted by a state attestation commission, involvingacademics and non-academics, set up by the body responsiblefor accreditation. Attestation is a precondition for the award ofindividual degrees as well as part of the process ofaccreditation at the institutional level.

The focus of the National Accreditation Center is to provideinformation to the Accreditation Council. The Council,established in 1997, is a unit of the Ministry of Education. Ithas forty-two members drawn from the Ministry and fromvarious educational and non-state organizations. It organizesreviews and invites review teams. Review teams evaluate theinstitution and all its programmes and consist of five to fifteenexperts in different fields, including management, education,and research. The role of the center is to compiledocumentation for the Ministry that makes accreditationdecisions based on the report of the Accreditation Council.

The Center works with a staff of about fifty full-time andabout twenty part-time employees. The permanent staff of theCenter includes five academics and eight top managers withone director and three deputy directors. There are sixdepartments at the headquarters in Yoshkar-Ola and onebranch in Moscow. A major part of the work of the Center isto compile data on higher education institutions, theirprogrammes, and the accreditation status of both. TheCenter also conducts research into accreditation and qualityassessment. It designs assessment procedures and developscomputer software for different stages of quality assessment.Another important service offered by the Center is to provide


information on accreditation status and procedures. Thisinformation can be obtained via the Website of the Center<http://www.nica.ru>. The Center staff includes academicsand professionals with academic degrees.

2.13. SLOVAKIA

Slovakia has twenty-three higher education institutions, ofwhich twenty-two offer Master's Degree level and Doctoraleducation, while one private institution leads only up to theBachelor's Degree level. Ten of the higher education institutions,including the newly established Catholic university, areuniversities; three are technical universities with a range offields; and three are professionally oriented universities. Thereare also three university-level academies of art, one police, andtwo military academies. The latter three are considered "state"institutions, while the other non-private ones are "public"institutions. All higher education institutions, whether public,state, or private, must have state recognition to operate.

More students apply for admission to higher educationinstitutions than there are study places available. At the sametime, institutions compete for students in less fashionablefields.

The first Slovak Law on Higher Education following the1989 change of regime was introduced in 1990. A new HigherEducation Act went into effect on 1 April 2002 reflecting twofundamental changes. Whereas in the past faculties were legalentities, nowadays it is the university that is a legal entity. Itretains ownership of its properties, for the oversight of which,and some other strategic decisions, it must set up a Board ofTrustees. The 2002 Law has changed some of theresponsibilities of the Accreditation Commission. Moreover, theMinister of Education must now approve its regulations andstandards.

The Commission has twenty-one members, includingforeign ones, who are appointed by government on theproposal of the Minister following consultations with the highereducation institutions, the research institutes of the Academy


of Sciences, and other bodies, the latter possibly beingrecommended by various ministries. The Commissionevaluates the capacity of institutions to implementprogrammes, including doctoral programmes, in non-highereducation institutions, and to conduct habilitations and tonominate professors. In addition, it issues statements relativeto proposed changes in a higher education institution (e.g., oncreating a new faculty), about the setting up of institutions,about defining its type (university, non-university, or researchuniversity), about changing the structure of the study fields ofgiven institutions, and about conducting the "complex"accreditation of an institution. It acts as an advisory body togovernment.

There is a distinct separation between programme andinstitutional accreditation. Programme accreditation grants theinstitution the right to award degrees at the level specified andto habilitate and nominate professors according to theprocedures that are also accredited. If the accreditation ispositive, it may be granted without a time limit, but will be re-assessed in the context of "complex" institutionalaccreditation. If problems were identified, accreditation is givenfor one or two years, depending on the severity of the problemsin question. The evaluation aspect in this process is negligible,and accreditation is based on threshold criteria.

"Complex" institutional accreditation is conducted at six-year intervals. It encompasses the institutional level, includingthe procedures for habilitation and nomination of professors,as well as all programmes. The procedure should focus onimprovement, with the Commission pointing out strengths andweaknesses. As part of complex institutional accreditation, theCommission also evaluates the performance of a given highereducation institution in research and development (except infields of art). As per the new law, the quality of research is nolonger graded.

It is interesting to note that the same procedure forevaluating research is applied to the Academy of Sciences aswell as to state and private research institutes. This fact


implies that all of these institutions will vie for research fundsfrom the state budget on the basis of accreditation.

The decisions of the Commission are formulated asopinions addressed to the Minister, or, in the case of certainspecific tasks, to the Government or Parliament. However, ifthe decision of the Minister regarding accreditation is differentfrom that of the Commission, he or she must, by law, publishhis or her reasons.

Currently, the technical staff of the AccreditationCommission consists of one academic-professional person andone administrator. A government decree will lay down thebasic conditions for operation.

2.14. SLOVENIA

In 1993, Slovenia had two comprehensive universities and two"free-standing professional higher education institutions".Today, the number of universities remains unchanged (with,altogether, thirty-seven faculties), but there are also sevenprivate higher education institutions. The "free-standing"schools are required to have state recognition.

The Higher Education Act of 1993 set up a Council forHigher Education. Its standards are specified in the Act, andits criteria for evaluation, in its "Criteria and Procedures onAccreditation of Study Programmes and Higher EducationInstitutions" (in, Zgaga and Jurkovie, 1995).

Six of its fourteen members and its President are appointedby the Government, after nomination by the higher educationinstitutions, for four-year terms. They are joined by thePresident of the Academy of Sciences and Arts and the rectorsof the universities.

Among the responsibilities of the Council are those ofestablishing the criteria for assessing the quality of teachingand with approving study programmes. It is also expected todetermine whether or not an institution is fulfilling the legalrequirements to be established and operated, and whether newstudy programmes can be set up, as a precondition forfunding. The criteria for assessing higher education


institutions are also set down in the "Criteria and Procedures".Moreover, the Council is responsible for judging whether or nota faculty member may be awarded a title, based on therequirements of the law and the opinion of external experts.

In 1996, the higher education institutions took the initiativeto establish the Quality Assessment Commission. It wasrestructured in 2000. Its main purpose is to develop a self-evaluation methodology and to assist higher educationinstitutions in setting up their internal quality controlmechanisms. The Commission basically collects anddisseminates information about evaluation at institutions. TheCommission has eighteen members and a president. There isonly one administrator and no programme officers.

The overall tasks of the Commission are defined in the 1997amendment to the Higher Education Act as follows:

The quality and effectiveness of teaching, research, art, andprofessional activities of higher education institutions shall bemonitored and assessed by a Quality Assessment Commissioncreated by higher education institutions in the Republic ofSlovenia. The Commission shall be composed of representativesof all scientific and art disciplines and professional fields. TheCommission shall also obtain the advice of students. TheCommission shall conduct business according to the rulesdetermined in co-operation with the senates of highereducation institutions and criteria defined by the Council ofHigher education in co-operation with the Council for Researchand Technology. Once a year, the Commission shall report tothe Senates of the higher education institutions, the Council forHigher education, and the Council for Research andtechnology. The report shall be publicly disseminated (Article89, Higher Education Act of the Republic of Slovenia, amendedin 1997).

With more students applying than there are places availablefor them, at least in some areas of study, there is competitionamong higher education institutions for students. Accreditation,however, is not yet mandatory.

The law in Slovenia has, in effect, established two bodies,one, which answers to the Government, and another, which


answers to the higher education institutions (but, as anationally legislated body, also to government), with the twoexpected to share criteria. With initially only a handful ofhigher education institutions the sector felt little pressure toorganize its system of quality assurance. The sharp rise in thenumber of institutions in the last two to three years is boundto speed up the process.

2.15. UKRAINE

In 2000, Ukraine had 809 state and 161 private highereducation institutions. The Law on Education of the countrywas passed by Parliament in 1991 and, amended in 1996,establishing the broad legislative framework for education. Theelaboration of a higher education law began in 1996; however,no such law has yet been passed. In line with the ongoingreform, the Ministry of Education and Science was reorganizedin 1997. It is responsible for over half the state highereducation institutions, while the rest are under otherministries. The respective ministries are involved in setting theeducational content and the accreditation of the highereducation institutions under their authority, while theeducation ministry retains responsibility for the educationprocess. It is also in charge of conducting the accreditation ofhigher education institutions, setting admission standards,and issuing licenses and certificates. The autonomousRepublic of Crimea has a separate Ministry of Education, butit abides by the regulations of the Ukrainian ministry. A StateAccreditation Commission, as a unit of the Ministry, isresponsible for carrying out the accreditation procedures.Accreditation was introduced with the 1992 law and iscurrently regulated according to a Regulation of the Cabinet ofMinisters (No. 200, 12 February 1996).

The accreditation of higher education institutions isawarded up to one of four levels pertaining to the institutionaltype and diploma or degree offered. Level I is for technical andvocational schools leading to the certificate of "juniorspecialist". Level II is for colleges leading to a Bachelor's


Degree. Levels II and IV are for institutes, conservatories,academies, and universities leading to "Specialist" andMaster's Degrees.

The level of accreditation determines the degree ofautonomy of the institution as well as its funding, with onlythe highest level receiving full financial support. Of the 809state higher education institutions in 2000, 220 were of LevelsIII and IV, while 589 were of Levels I or II. Of the 161 non-statehigher education institutions, only forty-one were fullyaccredited. A shift to the higher levels is apparent (from 161 in1995 to 220 in 2000).

New higher education institutions are first licensed by theState Accreditation Commission. Licensum does not representa qualitative judgment but merely allows the institution tobegin operations. The succeeding step is accreditation, whichgives the institution the right to issue state-recognized degrees,issued after completion of education based on nationalstandards. A course programme can be submitted foraccreditation only after it has been offered for five years.Accreditation covers all study programmes and must berenewed every ten years. Institutions are awarded the level ofaccreditation that has been awarded to 75 percent of itsprogrammes.

The Commission is regulated by the Constitution, bylegislation passed by the Parliament, by the President ofUkraine, by the Cabinet of Ministers, and by the Ministry ofEducation. The Commission sets the criteria for accreditationand publishes accreditation results. The members of theCommission, including civil servants from the education andother ministries, as well as state executives, academics, andresearchers, must be confirmed by the cabinet of Ministers.The Minister is the Chairperson of the Commission. TheCommission has a network of local and regional expertcouncils who report on the individual programmes submittedfor accreditation. The Commission adopts the final decisionrelative to the level of accreditation awarded.


2.16. COUNTRY OVERVIEW OF ACCREDITATION ANDQUALITY ASSESSMENT AGENCIES

2. 16. 1. Clarification of Terminology

NAME OF AGENCY/COMMISSION, YEAR ESTABLISHED: the officialname of the agency or commission at present and theyear the agency or commission was set up.

INITIATIVE TO ESTABLISH AGENCY/ COMMISSION: either thegovernment or a group of higher education institutionswhich joined to initiate quality assurance in the countryor for members of the group.

OWNERSHIP OF AGENCY /COMMISSION: To whom does the agencyand/or commission report?

MEMBERSHIP, STAFF: Who delegates/names, the members of thecommission, their number, and their background; howmany staff members and their role?

SCOPE OF ACTIVITIES, ORIENTATION: What are the main tasks ofthe commission/agency, what types ofassessment/accreditation do they conduct, and theobjective of the assessment?

CYCLE OF ASSESSMENT: For how many years is accreditationvalid; how often is assessment repeated?

METHODS: The common approach would be self-evaluation,peer review, a report, but other methods may bepossible.

GRADING: Is the evaluated institution or programme given aquality mark on a scale used by the agency/commission,or are there different levels of accreditation, yes or no?

CRITERIA FOR ASSESSMENT: Who develops the criteria, standards,and methods to conduct assessment?

REVIEW TEAM TRAINING OR BRIEFING: Which or neither, perhapsonly written information?

FOREIGN COMMISSION MEMBERS, REVIEWERS: Are foreignnationals members of the commission and/or are foreignnationals invited to participate in the evaluation; if yes,which of them?

REPORT PUBLISHED: Yes or no? Clarification if needed.


2.16.2. Schematic Presentation of Accreditation and QualityAssessment Agencies by Central and Eastern European Countries

ALBANIA

Name of agency/ commission, year established

Initiative to establish agency/ commission

Ownership of agency/ commission

Membership, staff

Scope of activities, orientation

Cycle of assessment

Methods

Grading

Criteriafor assessment

Review team training or briefingForeign commission members, reviewersReport published

Accreditation Agency of Higher Education,and Accreditation Council, 1999GovernmentGovernment (independent decisions onquality but advisory to minister)11 Council members(academics); 5 Agencystaff with collaborating, coordinating,administrative roleInstitutional and programme-levelaccreditation. Improvement oriented4 yearsSelf-evaluation according to Agencyguidelines, review team with on-site visitafter agency guidelinesYesSet by Agency and expert team based onlegislationTrainingNoYes

BULGARIA

Name of agency / commission, year established



Membership, staff


Cycle of assessment

Methods

Grading



National Evaluation and AccreditationAgency, and Accreditation Council, 1996GovernmentGovernment (independent decisions onquality but advisory to minister)9 Council members (academics), 20Agency staff with coordinating andadministrative roleInstitutional, subsequently programmeaccreditation, with accountability andimprovement orientation5 years, 1-3 years if assessed as"Satisfactory"Self-evaluation according to Agencyguidelines, review team with on-site visitaccording to Agency guidelinesYesCriteria set down by Agency based onlegislationBriefingCouncil no, review team sometimesYes

3EST COPY AVAILABLE 116

120

CROATIA

C. CAMPBELL and C. ROZSNYAI

Name of agency / commission, year established National Council for Higher Education, 1993,1999


Ownership of agency/commission

Membership, staff


Cycle of assessment

Methods

Grading



GovernmentGovernment (unit in, and advisory to,Ministry, which organizes reviews)19 Council members (academics), 3 full-time,1 part-time staff in Ministry departmentEvaluation of institutions, accreditation ofprogrammes, with improvement orientationNot setSelf-evaluation according to Councilguidelines, review team with on-site visitaccording to Council guidelinesYesCriteria set down by Council based onlegislationBriefingCouncil no, Review teams yesNo

THE CZECH REPUBLIC

Name of agency/ commission, year established Accreditation Commission,1990Initiative to establish agency/ commission

Ownership of agency/commission

Membership, staff


Cycle of assessment

Methods

Grading

Criteria for assessment

Review team training or briefingForeign commission members, reviewers

Report published

GovernmentGovernment (independent decisions onquality but advisory to Minister)21. members (11 academics and 10professional experts); 6 staff withcoordinating and administrative roleAccreditation of programmes and institutions.Improvement orientatedMax. 2 x length of programme (doctoral max.10 years),Institutions have independent internalevaluation. Self-evaluation according toagency guidelines, review team with on-sitevisit according to agency guidelinesYes (Le., 3 levels of negative decision)Elaborated by review team based onCommission criteria and legislationWritten informationYesYes, both internal and external evaluationreports


ESTONIA


Initiative to establish agency/ commissionOwnership of agency / commission

Membership, staff


Cycle of assessment

Methods

Grading


Review team training or briefing

Foreign commission members, reviewers

Report published

Higher Education Accreditation Center,Archimedes Foundation, 1997; HigherEducation Quality Assessment Council, 1995GovernmentGovernment (Council); foundation (Center)12 Council members (academics); 3 Centerstaff with coordinating and administrativeroleProgramme, some institutional accreditation.Improvement oriented (separate researchevaluation)7 years for full, 2 years for conditionalaccreditationSelf-evaluation according to Agencyguidelines, review team with on-site visitaccording to Agency guidelinesYesDetermined by Council based on legislationand approved by MinisterWritten informationIn Council no, Review team only foreignmembersNot broadly but sent to institutions and tostudent organizations

HUNGARY

Name of agency/commission, year establishedInitiative to establish agency/ commissionOwnership of agency / commission

Membership, staff


Cycle of assessment

Methods

Grading



Hungarian Accreditation Committee, 1993GovernmentGovernment30 full members (academics andprofessional body representatives), non-voting members to cover spectrum ofdisciplines, 1 non-voting student; 26 staffwith collaborating, coordinating,administrative roleAccreditation of institutions including itsprogrammes. Improvement oriented8 yearsSelf-evaluation according to agencyguidelines, review team with on-site visitaccording to agency guidelinesYesSet by agency, with national qualificationrequirements legislatedBriefingSometimesYes

122

LATVIA

C. CAMPBELL and C. ROZSNYAI




Membership, staff


Cycle of assessment

Methods

Grading


Review team training or briefing

Foreign commission members, reviewers

Report published

Higher Education Quality Evaluation CenterLtd., 1994, Council for Higher Education,1995, Accreditation Commission, 1995GovernmentMinistry and 5 higher education institutionsare shareholders of agency6 agency staff with coordinating,administrative role; 10 member AccreditationCommission; 10 member Council (academicsand non-academics)Institutional and programme accreditationwith improvement orientationPermanent or 2 years for institutions, 6 years(or 2 years only once) for programmesSelf-assessment based on agency guidelines,site-visit based on methodology by teamRecommended for programmes but rarelyusedEstablished by agency based on legislationWritten information, sometimes training orbriefingYes (except for review of college-levelvocational higher education institutions)Yes

LITHUANIA

Name of agency /commission, year established


Ownership of agency / commission

Membership, staff


Cycle of assessment

Methods

GradingCriteriafor assessmentReview team training or briefingForeign commission members, reviewersReport published

Lithuanian Center for Quality Assessmentin Higher Education, 1995Ministry of Education and Science of theRepublic of Lithuania; Lithuanian ScienceCouncilGovernmentExpert evaluation division head (academic,part-time) + 3 full-time agency staff withcoordinating roleInstitutional and programme assessment.Accreditation by Minister. (Separateresearch assessment also). ImprovementorientationNot set, 8 years for first cycle for all studyprogrammes, reassessment within 2 yearsafter temporary or restricted assessmentSelf-evaluation according to agencyguidelines, review team with on-site visitaccording to agency guidelinesForeseenEstablished by agency based on legislationWorkshops and briefingReview teams for specific subject areasReviews yes


MACEDONIA

Name of agency/ commission, year established Evaluation Agency, and Accreditation Board,2000

Initiative to establish agency/ commissionOwnership of agency/commission

Membership, staff


Cycle of assessment

Methods

Grading



Inter-University ConferenceGovernment9 Agency members with coordinating,administrative role; 15 Board members(academics)Institutional and programme accreditation.Improvement orientationFive yearsSelf-evaluation according to Agencyguidelines, review team with on-site visitaccording to Agency guidelinesNo

Established by Board and by Agency-basedon legislationWritten informationSometimesYes

POLAND

Name of agency/ commission, year established University Accreditation Commission (UAC),1998


Membership, staff


Cycle of assessment

Methods

Grading



Rectors ConferenceNon-governmental18 members (academics, i.e., 17 vice-rectorsor representatives of signatory universities +1 Rectors Conference delegate); 2 full-timeand 1 half-time staff, coordinating,administrative roleAccreditation of member and non-memberuniversity programmes at researchuniversities, voluntary. Improvementorientation2-5 yearsSelf-study based on agency guidelines,external review with on-site visitNo

Standards set by own expert committee,reviewed every two years, methodologyworked out by commissionBriefingReview team no, expert committee sometimesNo, only for evaluated unit


POLAND (cont.)

University Accreditation Commission (UAC),Name of agency /commission, year established ty1998

Name of agency /commission, year established


Membership, staff


Name of agency/commission,



Name of agency / commission,



Name of agency/ commission,

Name of agency/ commission,

Name of agency/commission.

year established

year established

year established

year established

year established

year established

year established

year established

year established


Membership

National General Accreditation Commission,January 2002. Expected to succeed GeneralCouncil on Higher Education, 2000 (Law:1990)GovernmentGovernment (unit in ministry)

Intention was to evaluate without formalaccreditation at institutional and programmelevel, accountability, some improvementorientation (The General Council had anumber of other tasks not directly related toquality control)Business Schools for Quality of HigherEducation (SEM FORUM) (private), 1993Accreditation Committee of Medical Schools(State), 1997Accreditation Commission of Schools ofHigher Vocational Education (State andprivate), 1997Accreditation Commission of Schools ofPhysical Education (State), 1999

Accreditation Commission of HigherPedagogical Schools (State), 1999

Accreditation Commission for TechnicalUniversities (State), 2001Accreditation Commission for AgriculturalUniversities (State), 2001Accreditation Commission of the Foundationfor Promotion and Accreditation of EconomicStudies (State), 2001Accreditation Commission of the Rectors ofPolish Academic Schools, 2001Rectors ConferenceNon-governmentalThe 7 state academic accreditationcommission chairpersons + a president , anumbrella organization


ROMANIA

Name of agency/commission, year established National Council for Academic Assessmentand Accreditation, 1994


Membership, staff


Cycle of assessment

Methods

Grading



GovernmentParliament19-21 council members (academics); 25staff with coordinating and administrativeroleInstitutional and programme accreditationwith improvement orientation5 yearsSelf-evaluation according to agencyguidelines, review team with on-site visitafter agency guidelinesNo

Established by expert commissions basedon legislationTrainingReviewers sometimesIf institution agrees

THE RUSSIAN FEDERATION

Name of agency / commission, year establishedNational Accreditation Center, 1995;Accreditation Council (Ministry for Generaland Professional Education), 1997


Membership, staff


Cycle of assessment

MethodsGrading



GovernmentGovernment50 full-time and 20 part-time Center staff; 42Council members (academics and non-academics)Institutional and programme accreditation.Improvement orientationMaximum 5 years for institution orprogrammeDeveloped by ministryNo

Developed by evaluation team, based onlegislationWritten informationNo

Yes but not widely distributed


SLOVAKIA

Name of agency/ commission, year establishedInitiative to establish agency/ commissionOwnership of agency /commission

Membership, staff


Cycle of assessment

Methods

GradingCriteria for assessmentReview team training or briefingForeign commission members, reviewersReport published

Accreditation Commission, 1990GovernmentGovernment27 members(academics); old commission had2 staff members with coordinating role andhead contributes to reportInstitutional and programme accreditation,improvement orientationProgramme accreditation without time limit,but 1-2 years if shortcomings observed, 6years for institutional accreditationSelf-study based on agency guidelines,external review with on-site visit based onagency guidelinesNoEstablished in law and by commissionBriefingCommission yes, review team sometimesNo

SLOVENIA

Name of agency/commission, year established Council for Higher Education, 1994Initiative to establish agency/ commissionOwnership of agency/ commissionMembership, staff


Cycle of assessment

Methods

GradingCriteria for assessmentReview team training or briefingForeign commission members, reviewers

Report published

Name of agency/ commission, year established

Initiative to establish agency/ commissionOwnership of agency/commissionMembership, staff


GovernmentGovernment + Rectors Conference15 (academics)A government advisory body. Institutionaland programme accreditationNot setSelf-study, external review with on-site visitbased on agency guidelinesYesLaw + agencyNo

No

Only decision and summary reason withnegative decisionsQuality Assessment Commission of Slovenia,1996, reorganized 2000Rectors ConferenceGovernment19 (academics + 2 students), 1 administratorNo external evaluation, but advisory toCouncil for Higher Education. Improvementoriented


SLOVENIA (cont.)

Name of agency/commission, year established Council for Higher Education, 1994

Cycle of assessment

Methods

Grading



Not set

Self-study based on institution's ownapproach, external review with on-site visitserves to advise on self-evaluationNoAgency, according to standards by Council ofHENoNo

No

UKRAINE

Name of agency / commission, year establishedState Accreditation Commission(Ministry of Education and Science), 1992


Membership, staff

Scope of activities, orientationCycle of assessmentMethodsGradingCriteria for assessmentReview team training or briefingForeign commission members, reviewersReport published

GovernmentGovernmentMembers from various ministries, academics,researchers, state executives, chaired byeducation ministerProgramme and institutional accreditation10 years for programme accreditationSet by commission based on legislationNo dataSet by commission based on legislationNo dataNoOutcomes yes

2.16.3. Internet Links for Quality Assurance in the Centraland Eastern European Countries

<http://www.ceenetwork.hu>individual agency links (access from

<http: / /www.ceenetwork.hu>)information on CEE agencies also at

<http://wwvv.inqaahe.nl>


2.16.4. Questionnaire Sent to Quality Assurance Agencies inFifteen Central and Eastern European Countries

a) Could you please send me a bibliography on qualityassurance and your schools' systems to list in thebibliography section of the book;

b) How many higher education institutions in the country?Number of universities: Number of others? Number of stateinstitutions? Number of private institutions? Do they needstate recognition to operate?

c) Do you see QA as a tool for controlling the proliferation ofprivate institutions?

d) Do more students apply to higher education institutionsthan there is room for them?

e) Do you feel that there is competition among highereducation institutions to enroll students? If yes, isevaluation/accreditation a publicly recognized factor forchoosing a higher education institution?

j) Who is involved in making up study programmes, thehigher education institution, the ministry, your agency?

g) Scope of your agency: accreditation, evaluation, or both? Ifaccreditation, for institutions, or programmes, or both?

11) Do you evaluate institutions and/or programmes? If both,in separate procedures or in a single procedure? Isevaluation mandatory or voluntary?

1) By whom is the accreditation/evaluation commissionnominated/appointed? By government (i.e., minister, primeminister) or higher education institutions (i.e., rectorsconference)?

j) What is the size of office staff and what are the functions ofthe members (programme officers, administrators)? If youhave programme officers, do they have coordinating roles,participate in visits, contribute to reports?

k) Do you use a grading scale (such as Excellent, Good,Satisfactory...) in your assessment? If yes, for institutionaland/or programme evaluation?

1) Who devises the methodology and criteria for evaluation?Who devises questions for the institutional self-evaluation?


What is the ratio of data requested and of the analyticalpart? How long is the self-evaluation? Are higher educationinstitutions consulted?

m) What is the aim of the evaluation: accreditation? fitness ofpurpose? improvement? compliance with laws? (more thanone possible)? Does your agency advise the Minister andhow binding is a decision by your commission (does theMinister have to publish his reasons for not agreeing withthe Commission)?

n) Do you prepare or train the review team? How?o) Are its reports published? How detailed is your report

(anything from "complies with the law" to detaileddescription of strengths and weaknesses)?

p) Do you include foreign members in review teams?Sometimes? Always?

q) Who decides on the schedule of the visit? When to visit aninstitution and/or a programme and the make-up of thereview team? For how many years is the evaluation/accreditation valid?The authors hereby wish to thank the contact persons of

the CEE Network agencies, who contributed considerable timeand effort to reply to the questionnaire and to do even more.

Annex 3

Glossary

The definitions appearing below have been derived from anumber of sources. In particular, those marked with a singleasterisk (*) are based on definitions appearing in QualityAssurance in Higher Education Manual of Quality Assurancein Higher Education: Procedures and Practices (Turin: EuropeanTraining Foundation, 1998, p. 12). Those marked with adouble asterisk (**) are derived from the questionnaire for asurvey on quality assurance agencies conducted, in December2001, by the International Network of Quality AssuranceAgencies in Higher Education (INQAAHE). See<http: / /www.inqaahe.nl>.

Accountability: The assurance of a unit to its stakeholders thatit provides education of good quality.

Accreditation: The award of a status which signals approval,recognition, and sometimes a license to operate and isbased on pre-defined standards*. It is a formal, yes/no(sometimes also conditional) decision based on explicitminimum (threshold) requirements**.

Agency: Organization or office responsible for preparing, co-ordinating, and carrying out evaluation or accreditationprocedures but not actually taking decisions onoutcomes. The term, agency, is often used collectively,when in fact, a committee passes the final evaluationdecisions.

Benchmark: A reference point or criterion by which to measuresomething.

Benchmarking: Setting levels against which quality ismeasured or a pro&ss of identifying and learning fromgood practice in other organizations.

Binary (or dual) system When higher education in a country isoffered in two separate and distinct types of institutions

131

127


universities and non-university establishments (cf.,unitary system).

Criteria: "Checkpoints, the benchmarks for assessing thequality of the input and process**. Often usedsynonymously with "standards".

Evaluation: Any process leading to judgments and/orrecommendations regarding the quality of a unit*.

Evaluation team: Same as a review team.Ex-ante assessment/evaluation: Assessing/evaluating quality

before a programme or institution is launched, often as acondition for granting a license to operate.

Ex-post assessment/evaluation: Assessing/evaluating qualityafter a programme or institution has been in operation inorder to establish strengths and weaknesses.

External evaluation: Evaluation of the quality of a unit carriedout by a selected team of experts who are not connectedto the unit evaluated.

External expert Also: peer inspector. Member of a selectedteam evaluating a unit but who is not linked to it.

Fitness for purpose: One of the possible criteria for establishingwhether or not a unit meets quality, measured againstwhat is seen to be the goal of the unit.

Higher education: Part of tertiary education leading to a degreeor equivalent diploma.

Improvement orientation: Evaluation or assessment describingthe strengths and weaknesses of a unit to facilitateimprovement, and possibly including suggestions as tohow to achieve it.

Inspector. Also: peer, external expert. Member of a selectedteam evaluating a unit but who is not connected to it.

Internal evaluation: Same as self-evaluation, done by a unit asa form of quality management or in preparation forexternal evaluation.

128

GLOSSARY 133

One-level degree structure: Also: one-tier, or one-cycle, wherebyone course of studies leads to one level of degree (eitherin unitary or binary systems) as opposed to a two-tierdegree structure, which offers a shorter degreeprogramme and a choice of post-graduate programmes.

Licensing: Award to operate a new unit granted if it meetsestablished criteria for quality.

Peer. Also: external expert, inspector. Member of a selectedteam evaluating a unit but who is not connected to it.

Programme officer. Upper-level employee of an evaluation oraccreditation agency coordinating, assisting, andcarrying out the administration of evaluation oraccreditation procedures.

Quality assessment Same as quality review. Evaluation butincluding its external dimension*. The "evaluation ofquality itself,"** either on the institutional or programmelevel.

Quality assurance: Policies, processes, and actions to maintainquality. The focus is on accountability to thestakeholders* of a unit.

Quality audit Evaluation of an institution's processes forquality management*.

Quality commission, committee: Standing body of expertschosen by government or a group of higher educationinstitutions for a specified term to establish the quality ofinstitutions or programmes and frequently also to stateits opinion on matters concerning higher education.

Quality control: Like evaluation, but stressing the internalmeasurement of quality of a unit. Often usedsynonymously with quality management*.

Quality control: Aggregate of measures taken regularly toassure quality of a unit, but the emphasis is on assuringthat a prescribed threshold of quality is met.

Quality enhancement Procedures taken to improve quality.


Quality management Aggregate of measures taken regularly toassure quality of a unit. Emphasizes the goal to improvequality.

Quality review: Same as quality assessment. Evaluation butincluding its external dimension*.

Review team: Same as (external) evaluation team, expertcommittee, visiting team, or peer review team. A group ofexperts not linked to the evaluated institution, unit orprogramme, who report on its quality based oninformation provided by the unit and on-site visits.

Se If-evaluation: Same as internal evaluation, done by a unit asa form of quality management or in preparation forexternal evaluation.

Staff Employees of an evaluation or accreditation agency co-ordinating, assisting, and carrying out the administrationof evaluation or accreditation procedures. Staff mayconsist of programme officers with coordinating andassistance functions and of office assistants oradministrators with secretarial functions.

Stakeholder. Students, society, and government participatingin or benefiting from the provision of education.

Standards: Expected outcomes of education, competenciesexpected from graduates. May be general standards for adegree level (e.g., Bachelor's or Master's Degree) orsubject specific standards.** Often used synonymouslywita "criteria".

(Subject) benchmark statement Represents general expectationsabout standards (levels of student attainment) at a givenlevel in a particular subject area.

Tertiary education: Any education entered after successfulcompletion of secondary education, which may includevocational post-secondary education (leading to acertificate) and higher education (leading to a degree),even though the designation is often used synonymouslywith higher education.

130

GLOSSARY 135

Unit An institution, a faculty, or programme of study*.Unitary system: Higher education in a country offered in one

type of institution, consisting of universities oruniversity-like establishments (cf., binary system).

Annex 4

Quality Assurance: Aspects of Processes,Approaches, and Networking

There is no "one size fits all models" for a quality assurancesystem. Each existing system is adapted to the cultural, social,and political context in which it has been established.However, a small sample of information collated frompublished documents, project reports, workshop sessions, andWebsite, is included here to illustrate some aspects of externalevaluation and current developments in quality assurance inEurope. The selection is limited by the availability ofinformation in English and the ease of accessibility for follow-up as well as by the fact that many systems of qualityassurance in higher education are, at the moment, in theprocess of development or change. Information has beenincluded from quality assurance systems in small countries,from sector specific agencies, those in which legislation is afactor in setting standards and those in which it is not.Inclusion is not an indicator of best practice, neither isexclusion an indicator of less good practice.

Readers are advised to go to the Website of the networks forquality assurance (see Part III) and follow links to individualQuality Assurance and Accreditation agencies, where muchmore information, including details of contact persons, isavailable. While several agencies may not have their guidelinesavailable on the Web they will make them available in hardcopy on request.

The authors would like to thank all of those who haveprovided information or have agreed to allow it to be madeavailable.

137

132


4.1. QUALITY ASSURANCE AGENCIES: PRINCIPLES,VALUES, AND PROCESS STANDARDS

Some examples of statements concerning current andproposed principles, values, and process standards of QualityAssurance Agencies are presented below.

4.1.1. FH Conference/ FH Council Austria: External Evaluation ofFachhochschulen in Austria <http://www.fhr.ac.at/english>

PRINCIPLES

1. The scientific evaluation pursuant to Section 13Paragraph 2 FH Studies Act as amended consists of a self-evaluation report of the FH study programme and themaintainer (provider), the peer report, and the statementof the maintainer.

2. In the preparation of the self-evaluation report (abbr. SEReport), the division of responsibilities between the FHstudy programme and the maintainer has to be taken intoaccount in view of the autonomy of the FH studyprogramme pursuant to Section 12, Paragraph 2, Sub-paragraph 5, and Section 12, Paragraph 4, Sub-paragraph2, FH Studies Act as amended.

3. Prerequisite for the preparation of the SE report is theexistence of an internal quality management system thatshould already be installed and documented in outline atthe time of preparation of the SE Report.

4. The choice and design of the internal quality managementsystem is the duty of each individual Fachhochschulestudy programme.

5. The quality system, FC Conference/FH Council, is basedon the internal self-evaluation by the Fachhochschulestudy programme and the maintainer in accordance withthe prescribed general conditions of the fifteen sections.Based on this evaluation report, an external peer groupexamines the activities of the FH study programme andthe maintainer.

133

PROCESSES, APPROACHES, AND NETWORKING 139

6. The peer group consists of four peers and an assistant.The peer group elects a group leader who nominates anassistant. The four peers are nominated jointly by the FHCouncil and the FH Conference and are appointed by theFH Council. The peers are assigned by the maintainer ofthe FH study programme to be evaluated.

7. Th.:: external evaluation takes place in a form that allowsthe assessment of the achievement of the goals and thefulfillment of the targets of the approved programme to beevaluated.

8. The assessments and the recommendations of the peers,after examination by the FH Council for their compliancewith the FH Studies Act and the decisions of the FHCouncil, can be taken into account in the maintainer'sapplication for extension of recognition.

9. The evaluation procedure is mainly founded on the qualityconcept, "fitness for purpose", that is, the quality of astudy programme is measured by the degree to which theprescribed and its own objectives have been fulfilled.

10. The main task of the evaluation is establishing andassessing the discrepancies between the prescribed aim orthe intended quality of the application and the actual stateof the FH programme. This does not rule outrecommendations by the peers.

4.1.2. Denmark: Danmarks Evalueringsinsitut (EVA)<http: / / www.evadk>

CENTRAL VALUES

All EVA evaluations are based on the principles ofimprovement, visibility, and professionalism.

IMPROVEMENT

Evaluations must identify strengths and weaknesses and thuspromote the improvement of teaching and learning based onthe objectives of the field of education - not on alreadyaccepted standards.

140. C. CAMPBELL and C. ROZSNYAI

VISIBILITY

Evaluations must make the quality of teaching and learningvisible, highlighting what is already successful and what maybe improved.

PROFESSIONALISM

Evaluations must be based on the expertise already availablewithin the educational sector concerned. For this reason,experts familiar with the specialized field are always invited toparticipate in the evaluation. Evaluation officers from EVAcontribute to the evaluation know-how.

4.1.3. Germany: Accreditation, Certification, and QualityAssurance Institute (ACQUIN) <http: / /www.acquin.org /english>The main characteristics of ACQUIN are independence,objectivity, and high quality. The agency is oriented towardsthe realization of academic principles and is neither influencedby the state nor by lobbyists. Academic freedom and academicautonomy are respected: higher education institutions mayregulate their own quality and standards, but at the same timethey must guarantee transparency of process and publicaccountability in discharging this self-regulation.

4.1.4. The United Kingdom: The Quality Assurance Agency forHigher Education (QAA) <http: / / www.qaa.ac.uk>

Draft Handbook for Institutional Review: Annex J: TheAgency's Operational Principles and Process Standards

BACKGROUND

1. The Agency's approach to undertaking institutional auditsdraws upon the practices and process standards developedand enhanced by its predecessor bodies. Since those bodiesbegan their work, good practice in auditing (guided bypublished standards of auditing practice) and requirementsrelating to accountability and reporting have developed


considerably. The Agency recognizes that some of theprocess standards it has observed in the past have beenimplicit rather than explicit and that the institutional auditprocess should be underpinned by a more explicitstatement on operational principles and process standards.

2. In developing its operational principles and processstandards, the Agency has taken note of the principlesunderpinning the AA1000 series accountability standardand the "Seven Principles of Public Life" developed by theNolan Committee.

PRINCIPLES

1. The Agency seeks to observe and promote several generalprinciples within both the strategic and operational levels ofits work. The principles are:INCLUSIVENESS taking into account the needs of allstakeholder groups and facilitating their participation inaspects of the Agency's work.OPENNESS transparency in the work and methods of theAgency, to build trust and confidence among stakeholders,and to provide information about the Agency's work to thewider public.ACCOUNTABILITY demonstrating that the Agency is using itsresources to good effect and with probity; conducting itswork with integrity and impartiality; and ensuring thatstakeholders are able to depend on the informationprovided.TIMELINESS the need for regular, systematic, and timelyaction in all reporting processes to support the decision-making of the Agency and its stakeholders.COMPARABILITY using experience drawn from within theAgency and other organizations as a means with which toinform future work.RELEVANCE ensuring that the information provided by theAgency is useful to, and understood by, all stakeholders.

136


2. In due course, these principles will be used to developexplicit service standards for institutional audits.

QUALITY ASSURANCE MECHANISMS

1. The Agency is committed to the regular monitoring andevaluation of its policies, procedures, and processes, to ensuretheir ongoing credibility, and to continuously improve itsperformance in response to the results. In respect ofinstitutional audit, this commitment includes providing theopportunity for participants in the process, including students,to provide structured feedback on their experiences.

4.1.5. United States of America CHEA <http:/ / www.chea.org>Statement on Good Practices and Shared Responsibility in theConduct of Specialized and Professional Accreditation Review(Note: specialized accreditors are those who accreditprogrammes only and on a national rather than a regionalbasis. Some of these accreditors also operate outside theUnited States)

Key issues addressed by the statement:clear and direct communication between specializedaccreditors and institutional leaders;enhanced understanding by specialized accreditors ofthe larger context of institutional needs and direction;enhanced understanding by institutional leaders of theperspective and needs of specialized accreditors; andaffirmation that the relationship between resources andaccountability is grounded in meeting accreditationstandards.

Full text on <http://chea.org/research/good-practices.cfm>.


4.2. QUALITY ASSURANCE: SELF-EVALUATION DOCUMENTSAND REPORTS

4.2.1. United Kingdom: Guidance to Institutions on Producinga Self-Evaluation An Example from the "Draft Handbook onInstitutional Audit in England" (April 2001, full text ofHandbook available at http://www.qaa.ac.uk)ANNEX C: GUIDELINES FOR PRODUCING SELF-EVALUATIONDOCUMENTS FOR DISCIPLINE AUDIT TRAILS

11. 1. After the audit team has confirmed the disciplineaudit trails to be pursued during the audit visit, theinstitution is asked to provide a self-evaluation for eachdiscipline to be trailed (the discipline SED). A recentinternal review report (or similar) on the discipline,accompanied by the relevant programme specifications,may be provided instead of a discipline SED, providing thatit covers the matters listed below, in Paragraph 2, or issupplemented with a note covering these matters.

0

LENGTH, STYLE, CONTENT, AND STRUCTURE

1. If the institution wishes to prepare a discipline SEDspecifically for the purposes of the audit trail, it is suggestedthat the document [be] about 3,000 words in length andcover the following:

EDUCATIONAL AIMS OF THE PROVISION a statement of theoverall aims of the programme or cluster of programmescovered by the discipline audit trail;LEARNING OUTCOMES evaluation of the appropriateness, tothe educational aims, of the intended learning outcomes ofthe programme or of each of the clusters of programmes,making reference to internal and external reference pointssuch as Subject benchmark statements and the FHEQ(Framework for Higher Education Qualifications England,Wales, and Northern Ireland);

CURRICULA AND ASSESSMENT evaluation of the ways in whichprogramme content and methods of assessment support


achievement of the intended learning outcomes of theprogramme(s); how curricula and assessment togetherdetermine the academic level of the award(s) to which theprogramme(s) lead; the extent to which students achieve theprogramme aims and intended learning outcomes.QUALITY OF LEARNING OPPORTUNITIES, which can be furtherdivided into:

TEACHING AND LEARNING evaluation of the effectiveness ofthe teaching and learning strategies employed by theprogramme(s) for providing students with good learningopportunities to support achievement of the intendedlearning outcomes and academic standards;STUDENT ADMISSION AND PROGRESSION - evaluation of theways in which student progression through theprogramme(s) is supported and monitored, from intaketo completion;

LEARNING RESOURCES evaluation. of effectiveness of thedeployment of the resources, human and material, thatsupport the learning of students, and of the effectivenessof their linkage to the intended learning outcomes of theprogramme(s).

MAINTENANCE AND ENHANCEMENT OF STANDARDS AND QUALITYevaluation of the effectiveness of procedures formaintaining and enhancing the quality of provision andthe security of academic standards in respect of theprogramme(s).

ANNEX D PROGRAMME SPECIFICATIONS FOR EACH PROGRAMMECOVERED BY THE TRAIL

1. The emphasis in the discipline SED [self-educationdocument] should be on evaluation of student achievementof the appropriate academic standards, and of the learningopportunities offered to students to support theirachievements. Description of the programme(s) should bethe minimum necessary to enable the audit team tounderstand the background of the self-evaluation. The


programme specifications may be able to cover most or allof the descriptive material that is needed.

2. Discipline SEDs should:be balanced and relevant;be concise and accessible to the audit team;be appropriately balanced between analysis and description.

3. Institutions will note the advantages of ensuring that theirinternal processes are capable of generating reports thatcan serve, with minor adaptation, as discipline SEDs.

SUBMISSION

1. The institution is required to submit discipline SEDs to theAgency seven weeks before the audit visit.

CONFIDENTIALITY

2. Discipline SEDs remain confidential to the Agency and theaudit team; however, when appropriate they will be madeavailable to specialist advisers who are asked to provideadvice to the team, and to subject specialist reviewersundertaking reviews in the institution. It is likely that theaudit report will refer to and include quotations from thediscipline SEDs. The institution is strongly encouraged toinvolve students in the preparation of discipline SEDs andto make the completed documents available to them.

4.2.2. Germany: FH Conference/FH Council: Quality System<http: / / wwwjhr. ac. at/ english>(In this example, reference is made to legislation, the FHStudies Act, so as to show how the agency is addressing therequirements of the legislation.)


FORM AND CONTENT OF THE SELF-EVALUATION REPORT AND THEPEER REPORT

1. MISSION AND GOALS

Main mission derived from the FH Studies Act and theapplication for a FH study programme and the goals derivedthere from:

Normative concept with the "vision" of the studyprogramme;Strategic concept with the short, medium, and long-termstrategies and the definition of the core competencies;Operative concept with the implementation of themeasures derived from points 1 and 2 as well ascommunication of the mission.

Section 3, Paragraph 1: "Fachhochschule study programmesare programmes at higher education level which serve toimpart scientifically sound professional education". Aprerequisite for the recognition as a FH study programmepursuant to Section 12, Paragraph 2, Sub-paragraph 1 is thatthe "goals and the guiding principles for the design ofFachhochschule study programmes (Section 3) are met.

2. PEDAGOGICAL DIDACTIC CONCEPT

pedagogical didactic basic concept;description of forms of teaching and theirimplementation;balance between types of courses;measures for students from second chance education.

Section 3, Paragraph 2, Sub-paragraph 1: "Fachhochschulestudy programmes have to take into account the variety ofscientific study methods". Section 3, Paragraph 2, Sub-paragraph 8: 'The didactic concept of the courses has tocorrespond to their purpose and the educational level of thestudents".


Target group specific FH programmes (Compare informationfor applicants as amended). The use of distance studyelements is obligatory for this special form of studyprogramme: the presentation of a matching scientific anddidactic concept as well as an appropriate quality assurancesystem is essential.

Section 4, Paragraph 2, Second sentence: "If the scientificconcept of an FH study programme is based on professionalexperience, the access to this FH study programme may berestricted to an appropriate target group". If the access isrestricted to a target group, "then the duration of studies is sixterms; the FH study programmes have to be established asdistance studies" (Section 3, Paragraph 2, Sub-paragraph 2).

3. CURRICULUM AND MODIFICATION PROCEDURE

Teaching goals, teaching content, didactic structure perdepartment and course;(Special impact of the required professional experienceon the curriculum in target group-specific studyprogrammes);How is the topicality examined, which body in the studyprogramme decides on proposed modifications to thecurriculum;Coordination of courses and teaching contents.

4. EVALUATION OF THE TEACHING

To what extent will the general objectives (Point 1) andthe detailed teaching goals (Point 3) be achieved?Self-evaluation by the teachers;Course analysis by the students and the studyprogramme management (collegium);Consequences, measures, and their implementation;Results of current improvement processes;Evaluation of examinations.


5. ADMISSION PROCEDURE

Application of the admission criteria according to theapplication for an FH study programme;Body of experts (team or individual person);Permeability of the education system;Admission statistics of the last years;Recognition of proven knowledge.

6. ANALYSIS OF COURSE OF STUDIES

Number of study applicants and admitted students,drop-out rate, marks, etc.;Measures in case of [for instance], a high drop-out rate,unsatisfactory analysis of graduates, unsatisfactoryaverage mark;Measures to ensure that the prescribed duration ofstudies also corresponds to the actual duration.Analogous to the statistical analysis [for] the FH Councilin accordance with the "Regulation on Making AvailableInformation on the Studies as Amended".

7. RESOURCES/ EQUIPMENT/ FINANCES

Key data on facilities and physical assets according tothe application for an FH study programme;Library [and] teaching aids... including EDP equipment,student guidance and infrastructure (guarantee in caseof study programmes for working people?);Financial expenditure for teaching, administration,equipment;Costs per student.

8. TEACHING STAFF

Selection of teachers;Compliance with minimum requirements according tothe FH Studies Act;

-- Qualification profile of teachers;


Measures for personnel development and further educationof teachers and employees;Number of full-time teachers.

9. PRACTICAL TRAINING AND DIPLOMA THESES

Evidence of practical training places relevant to studyprogramme;Placing of trainees with regard to industrial law andsocial legislation;Documentation of practical guidance;Documentation and guidance of thesis and details of thethesis profile.

10. ANALYSIS OF GRADUATES

Integration and success on labour market;Profession-related activity;Duration of job search;Salary;Doctoral programme.

1 1 . RESEARCH) & D(EVELOPMEIVT) ACTIVITIES

Publications;R&D projects (number and project volume from projectsfunded by third parties and project expenditure);Services (training and consultation).Conferences and events.

12. ACTIVITIES AT HOME AND ABROAD

Co-operation partners at home and abroad (number andname of institutions);Student mobility;FH lecturers mobility;International and national meetings, conferences, andevents (both participation and organization);


International and national projects and co-operation(e.g., development of curriculum, ECTS, intensiveprogrammes, etc.);Public relations (information and guidance by FH studyprogramme).

13. ORGANIZATION

Structure and sequence (study programme, maintainer);Competencies and structure of responsibilities;Autonomy of the study programme according to the FHStudies Act.

14. gUALITY) MANAGEMENT) SYSTEM

Characteristics of the quality control policy;Presentation of the applied quality management system(system description, control systems, responsibilities).

FH STUDIES AC 7' COMMENTARY

The achievement of the goals and the guarantee of theprinciples of Fachhochschule study programmes requireinternal measures for quality assurance which should beintegrated into an extensive quality management system andlinked with each other.

15. SELF-EVALUATION REPORT

Point 15 of the peer report should give its view on the quality ofthe self-evaluation report and assess the report.

4.3. THE SELECTION, APPOINTMENT, AND TRAINING OFPEERS: SOME EXAMPLES OF CURRENT PRACTICE

4.3.1. Austria: FH Conference/ FH Council<http://fhr.ac.at/english>

COMPOSITION OF THE PEER GROUP

1. The peer group consists of four peers and an assistant andis made up as follows:


a representative from the pool of experts of the FHCouncil;a representative of the Committee for quality matters ofthe FH Conference;an expert of the professional field;a representative of a subject-related educationalinstitution from abroad or with relevant experienceabroad;an assistant of the administration/management of anAustrian FH study programme (with right of proposalbut not of voting).

2. The guidelines for the nomination of the peers.The peer of the FH Council should have a subject-relevant or pedagogical didactic qualification.The FH study programme to be evaluated has a single,justified veto right concerning the peer from the Qualitymatters of the FH Conference. (An example would be thegeographic proximity of a peer from a subject-related FHstudy programme.)In order to guarantee the impartiality of the externalevaluation, close relationships of peers with the FHprogramme to be evaluated have to be avoided. Explicitlyexcluded are teachers of the FH study programmes,members of the development team, and persons incontractual dependence.

WORKSHOP FOR PEER GROUPS

Workshops are designed and commissioned for allmembers of peer groups.Two weeks before the workshop they receive comprehensivewritten documentation on the general statistical analysesand the regulatory and control mechanisms of theAustrian FH sector, the QS system FH Conference/FHCouncil, and the legal foundations.The workshop is carried out in discursive form andincludes the experience of senior peers.


4.3.2. Denmark: Evaluation Group (EVA) <http: / /www. evadk>For each evaluation, EVA appoints an evaluation group. Thegroup is made up of individuals with special expertise in thefield concerned. The quality and integrity of the members ofthe evaluation group are crucial. All members must beindependent of the programmes/institutions being evaluated.As a general rule, EVA tries to recruit at least one Nordicmember for each evaluation.

4.3.3. The United Kingdom: The Quality Assurance Agency (QAA)<http: / / www. qaa. ac. uk>

All auditors, audit secretaries, and specialist advisors areselected by the Agency on the basis of published selectioncriteria and generally from nominations made by institutions.All are provided with induction and training to ensure thatthey are familiar with the aims, objectives, and procedures ofthe audit process and with their own roles within it. Thequalities required in auditors, etc., are outlined in the QAADraft Handbook for institutional audit, and every attempt ismade to ensure that the cohorts of auditors and specialistadvisors reflect appropriate sectoral, discipline, geographical,gender, and ethnic balance. Training for auditors, auditsecretaries, and specialist advisers is undertaken by theAgency in collaboration with appropriate training providers.The purpose of the training is to ensure that all:

understand the aims and objectives of the audit process;are acquainted with the procedures involved;understand their own roles and tasks, the importance ofteam coherence, the Agency's expectations of them, andthe rules of conduct governing the process;have an opportunity to explore and practice thetechniques of data assimilation and analysis, thedevelopment of programmes for visits, the constructionand testing of hypotheses, the forming of judgments andstatements of confidence, and the preparation of reports.

PROCESSES, APPROACHES, AND 111E7WORKATG 153

4.4. INTERNATIONAL BENCHMARKING AND EVALUATION:THE DANISH EVALUATION AGENCY (EVA) ANINTERNATIONAL COMPARATIVE EVALUATION OFPROGRAMMES IN AGRICULTURAL SCIENCE

IntroductionThis example presents an innovative project which could proveuseful, particularly for co-operation among small highereducation systems and especially when only one or twonational institutions offer programmes in a particular subjectarea.

Although there have been some pilot projects on evaluatingprogrammes according to a shared methodological framework,there have been few attempts to set up transnationalevaluations with a comparative perspective.

The Danish Evaluation Institute (EVA) decided in its actionplan for 2001 to initiate an international comparativeevaluation within the field of higher education. The scope ofthe evaluation is a pilot project involving BSc programmes inAgricultural Science in institutions in four countries theRoyal Veterinary and Agricultural University of Denmark,Waginingen University in the Netherlands, the University ofHohenheim in Germany, and University College, Dublin,Ireland.

In developing the methodological framework for the pilotproject, EVA has drawn on the lessons learned from therelatively few international evaluations which have beenconducted during the past ten years and the substantialexperiences gained by the cycle of evaluations of highereducation programmes conducted by the predecessor of EVA,the Danish Center for Quality Assurance and Evaluation ofHigher Education. The criteria for selecting the institutions toparticipate were as follows:

The institutions should have a record of commitment tothe internationalization of higher education.The institution should be expected to be motivated toparticipate in the evaluation and accordingly to allocate


the necessary time and human resources involvedprimarily in the self-evaluation process and in the followup on evaluation.The institutions should be able to appoint representativesfor all relevant groups of stakeholders who are able andwilling to communicate in English.

Purpose of the EvaluationThe purpose of the evaluation is two-fold. The project willpartly support the development of a common framework forinternational comparative evaluations and partly provide theparticipating institutions with substantial reporting on thequality of their teaching and learning in the field of agriculturalscience. Following this [plan], the specific objectives of theevaluation are to:

develop and test a common methodological framework andcommon quality criteria for comparative internationalevaluations of programmes within higher edutation;stimulate discussions on what constitutes good qualitywithin higher education across countries;establish mechanisms for continuous quality improvementand co-operation between participating institutions.

Areas of FocusIn its evaluations, EVA usually covers a broad range of aspectsrelated to the programmes being evaluated. As a pilot projectwith a strong methodological focus, the internationalevaluation will only include a few aspects. This choice reflectsconclusions drawn from earlier international evaluationsstressing the importance of limiting focus when conductingevaluations across different educational cultures. There will bethree areas of focus:

1. CORE COMPETENCIES

The assessment will include the characteristics of methodsrelated to competencies as well as to competencies specifically


related to agricultural science gained through the completionof a degree in agricultural science.

2. LEARNING AND QUALITY ASSURANCE MECHANISMS

The assessment will include three elements:

the existence of mechanisms for quality assurance andmonitoring of on-going activities;the existence of mechanisms and systems fordocumentation and dissemination of experiences andlessons learned;the capacity to transform gained experiences and lessonslearned into changed practices and strategies.

3. INTERNATIONALIZATION

The assessment will include:

the degree of internationalization in terms of the contentof the programme [for] international co-operation, amongother things, [and] the level and scope of internationalexchange of students and staff;existing procedures for exchanging best practices andbenchmarks of quality.

4. ORGANIZATION

A team of evaluation officers from EVA will be responsible forthe practical and methodological planning and implementationof the evaluation, while a panel of international experts (fromGermany, the Netherlands, the United Kingdom, Norway, aswell as Denmark) will be responsible for the professionalquality of the evaluation. The specific tasks of the expertsinclude mainly involvement in the formulation of commonquality criteria and responsibility for drawing conclusions andrecommendations based upon the documentation from theinstitutions involved in the evaluation. The composition of thepanel will thus involve persons with strong methodological


skills as well as persons with expertise within the field ofagricultural science.

5. EVALUATION METHOD

This includes a number of different elements generally used byEVA.

PRELIMINARY STUDY

EVA conducts a preliminary study (desk study) to collectand review existing materials relating to the field ofeducation (agricultural science) in the countries involved.EVA also collects and studies the findings andmethodological considerations of other internationalevaluations with a similar focus on comparative analyses.The preliminary study concludes with the formulation of theterms of reference and the appointment of the panel ofexperts.

it SELF ASSESSMENT

The participating institutions (programmes) conduct a self-assessment analyzing and assessing their own strengthsand weaknesses in relation to the three selected focus areas(core competencies, learning and quality assurancemechanisms, and internationalization).

iii. VISITS

The international evaluation panel will visit the involvedinstitutions. The visit will be planned in co-operation withthe institutions and will, together with the self-assessmentreports, constitute a substantial part of the background forthe findings and conclusions of the evaluation.

iv. REPORTING

The analysis, assessment, and recommendations of theevaluation will be documented in a report. A draft reportwill be sent to the institutions involved prior to completionof the final report. The final report will be published.


v. Follow-Up

The follow-up process will be determined later together withthe institutions concerned but will possibly consist of aninternational conference focusing on the methodological aswell as the programme related outcome of the evaluation.There will be no standard setting, and the assessment willnot result in any ranking of the programmes or theinstitutions. Rather, the assessment will focus on thestrengths and weaknesses of the priorities of the individualprogrammes in relation to the areas of focus in theevaluation. The intention of using the criteria in this way isto encourage and stimulate the development of theprogrammes involved in the evaluation. The EVA stressesthat the methodological approach of the evaluation andapplication of common quality criteria must not beinterpreted as an attempt to set up and to use anaccreditation procedure. Although accreditation requiresthe use of common quality criteria, application of criteriadoes not, per se, imply accreditation.Further details, including the final reports, in English, onthe evaluations will be available at a later date from EVA;see <http://vvww.eva.dk>.

4.5. APPROACH TO QUALITY ASSURANCE: THE EXAMPLEOF AN AGENCY IN A CENTRAL AND EASTERN EUROPEANCOUNTRY THE HUNGARIAN ACCREDITATIONCOMMITTEE (HAC)

The Hungarian Accreditation Committee (HAC) is preparing for itssecond eight-year cycle of institutional accreditation. Inpreparation for developing its new guidebook and to set downits general aims for the coming years, it has produced a strategicplan <http://www.rnab.hu/english/a_regulations.html> (summary).

I. THE MAIN ISSUESThe mission of the HAC is to contribute to the improvement ofthe quality of life of Hungarian society through improving thequality of higher education.

152


The general aim of the HAC is the protection of the qualityof Hungarian higher education. Quality is interpreted asfitness for purpose, which includes the capacity to meetstakeholder expectations.

The HAC has the following powers:

a. It makes recommendations in the field of educationalpolicy based on discussions pertaining to general issuesof the quality of Hungarian higher education.

b. It issues opinions to the Minister, to the HigherEducation and Research Council, and to the highereducation institutions on matters determined by theHigher Education Act (accreditation tasks).

c. It advises the institutions and programmes to assistthem in their quality enhancement (as part of theaccreditation processes). [This function has not beenemphasized so far.]

d. It issues decisions on launching and operating doctoralschools and on the disciplines in which an institutionmay offer post-graduate and doctoral training.

H. INTERNATIONAL TASKS

The HAC will initiate mutual recognition with foreignaccreditation and/or quality assurance agencies. The HACinitiates the harmonization of the work of CEE qualityassurance organizations.

The HAC maintains and, as far as possible, develops itsresearch and information activities concerning foreign qualityassurance activities and organizations. The results of theseactivities must be utilized in updating quality requirementsand procedures, in producing various guidebooks anddocuments, and in improving its operation.

III. INSTITUTIONAL ACCREDITATION

Institutional accreditation will have four essential, interrelatedcomponents in the future:

public presentation of the general and specific datarelated to the quality of institutions (made by the

.153


institutions themselves, on their homepages, updatedyearly) [New feature].self-evaluation report by the institution based on itsSWOT and constraints analyses, with special regard tothe process and the outcomes of teaching and learning[Includes new features].site visit and formative report by ad hoc visitingcommittees.user surveys by the institutions to determine usersatisfaction [New feature].

The HAC is determined to streamline the process ofinstitutional accreditation. A new Accreditation Guidebook isbeing written. Fewer data will be asked for in the self-evaluation report, while the analytical, evaluative character ofit will be strengthened (SWOT analysis). Institutions will beasked to name and evaluate their best and worst programmesand fields of operation in their self-evaluation. Institutions areasked to put quality-related data and information on theirWebsite. The aim is to serve not only accreditation purposesbut to provide information to students, other stakeholders,other (competing) institutions, and the general public.Moreover, easily accessible public data will improve thetransparency and accountability of the system.

The HAC will evaluate the mission and goals of theinstitutions and also other factors determining the quality ofthe institution as a whole (e.g., management, internal qualityassessment system). Moreover, the HAC will lay moreemphasis than before on evaluating the process and outcomesof teaching and learning.

As for programme accreditation performed within theframework of institutional accreditation*, the HAC will not

*In the first cycle of institutional accreditation, all running programmes of theinstitution were evaluated in detail. By "programme accreditation performed within theframework of institutional accreditation", this type of activity is meant. By programmeaccreditation in itself we mean a separate activity, the accreditation of programmes to beestablished or launched (this is called "preliminary accreditation" according to theterminology of the new government decree on the HAC).


examine all programmes as was the case in the first cycle. Thefollowing programmes will be evaluated in detail:

programmes judged in the first cycle to requiremonitoring;programmes launched after the first cycle institutionalevaluation report;selected programmes from among the best and worstnamed in the SER;programmes selected at random.

The institutional accreditation report will contain:an evaluation of the institution;discussion of the internal quality assessment system;detailed evaluation of selected programmes;discussion of data and information coming from users.

4. PROGRAMME ACCREDITATIONProgramme accreditation continues according to earlierpractice. An important new feature will be that the HAC willlaunch pilot study programme evaluations of programmes inthe same disciplines at various institutions, probably in 2003.The procedure and the standards of excellence must beworked out by end of June 2003.

After the full accreditation of doctoral schools is completed,a comprehensive report will be produced by end of June 2002.

5. TASKS RELATED TO THE OPERATION AND ORGANIZATION OF THE HACThe most important tasks of the HAC for improving itsoperation are the following:

creating an internal system of quality assurance (by theend of June 2003);yearly review of its operation;detailed analytical self-evaluation every three years,discussed with stakeholders (due in 2003);review of the current subcommittee structure (due in2003).


6. METHODOLOGYThe activities of the HAC will continue to have a double focus,in the future, from the point of view of methodology. On theone hand, the HAC performs accreditation, i.e., it defines thequality of institutions, faculties, and programmes in relation topredetermined threshold quality requirements, i.e., minimumstandards (judgments: "yes", "conditional yes", and "no"). Onthe other hand, above the threshold level, the HAC gives adetailed assessment and recommendations as to theenhancement of the quality of the given institution, faculty, orprogramme. That means that beyond the so-called "control"function (giving opinions to the Minister), the HAC alsoperforms an advisory function to promote quality enhancement.It is the definite intention of the HAC that in the future thelatter function be more emphatic than the former.

An important change in this respect is that in the secondcycle of institutional accreditation the HAC will no longer gradedegree programmes. (In the first cycle we used the grades"Excellent", "Strong", "Adequate", "Inadequate".)

7. PUBLICITYThe activities of the HAC should be publicized more widely andeffectively in the future. The aim of the communication strategyof the HAC is to improve the image of the organization and todeliver the mission, the aims, and the outcomes of the HAC'soperation to various target groups (higher educationinstitutions, students, the Ministry of Education, and thegeneral public).

4.6. QUALITY ASSURANCE FROM THE PERSPECTIVE OF AUNIVERSITY NETWORK ASSOCIATION: A POLICY STATEMENTOF THE EUROPEAN UNIVERSITY ASSOCIATION (EUA)

1. BACKGROUNDThis EUA policy paper on quality assurance arises from fivekey developments that have taken place in Europe over thepast few years:


The Magna Charta Universitatum (1988) which upholdsuniversity autonomy, must be the precondition for fosteringthe adaptability of universities to the ever-changingrequirements of today's society.

The meeting of Ministers at the Sorbonne's 800thanniversary (1998) referred to the central role of highereducation in the development of Europe through the creationof a European Higher Education Area.

[Through] the Bologna Declaration (1999), ...the twenty-nine signatory states agreed to act in concert to increase thecompetitiveness of Europe through a range of measures aimedat creating a European Higher Education Area. These includethe adoption of a system of easily readable and comparabledegrees and a system of credits. The objectives of such toolsare to promote mobility, European co-operation in qualityassurance, inter-institutional co-operation, and integratedprogrammes of study, training, and research.

The Salamanca Convention (2001) of European highereducation institutions considered quality as a fundamentalbuilding block of the European Higher Education Area andmade it the underlying condition for trust, relevance ofdegrees, mobility, compatibility, and attractiveness.

Similarly, the recent Communiqué of the EuropeanEducation Ministers (Prague, 2001) regards quality as a majorfactor in determining the competitiveness and attractiveness ofEuropean higher education.

In this context, the European University Associationconfirms the central role of quality in higher education andaffirms that the evaluation of quality should:

be based on trust and co-operation between institutionsand evaluation agencies;take into account the goals and missions of institutionsand programmes;consider the balance between tradition and innovation,academic excellence and socio-economic relevance, thecoherence of curricula, and students' freedom of choice;


examine teaching and research as well as managementand administration;include responsiveness to the needs of students and theprovision of non-educational services.

Quality assurance refers to a set of procedures adopted byhigher education institutions, national education systems, andinternational agencies through which quality is maintainedand enhanced.

Quality assurance is effective when it refers to the very coreof the higher education activity and when its results are madepublic.

Quality assurance implies academic autonomy and isclosely dependent on academic management that is based onthe principles of efficacy, academic and scientific performance,as well as competitiveness.

Quality assurance can succeed only if it becomes inhertnt tothe institutional culture. Such a culture generates the necessarymotivation and ensures competence in implementing qualityassurance mechanisms.

2. BENCHMARKING QUALITY MANAGEMENT

Various higher education systems have developed policiesregarding quality criteria, quality assessment, qualityassurance, and quality management. The quality of highereducation can be defined in various ways as: excellence, "zerodefects", "goal adequacy", capacity for ongoing improvement,minimum standard, marketability, or competitiveness. Everyapproach has its contextual justification.

Quality starts by ensuring minimum standards. It extendsto the capacity of ongoing improvement and includes acompetitive dimension at the national and at internationallevels.

The European University Association considers that it isimportant to identify common benchmarks for qualitymanagement and assurance to contribute to the creation ofthe European Higher Education Area. These quality


benchmarks must focus on the multiple dimensions ofacademic activities, as follows:

a. academic autonomy as an instrument for improvedperformance and competitiveness;

b. explicit institutional mission and objectives ofinstitutions and programmes;

c. transparent and non-discriminatory access andrecruitment policies, the possibility of a second chance,and fair appeals policies;

d. curricular quality;e. academic staff quality;f. permanent feedback from the students and

responsiveness to their suggestions, proposals, andcritique;

g. flexible organization allowing credit transfer,interdisciplinarity, and studying within the framework ofvarious programmes or institutions;

h. quality of infrastructure and availability of adequateequipment;

i. resource allocation with the capacity of obtaining extra-budgetary resources, motivating academic staff, andinvesting in buildings and equipment;

j. accountability with regard to the use of human andmaterial resources and systematic auditing;

k. feedback from stakeholders and the possibility ofadapting degree programmes to labour-market needs;

1. international scientific competitiveness;m. internal quality assurance mechanisms;n. contribution to public debate and democracy;o. innovation potential in technical, scientific, cultural, and

artistic fields.

3. ACCREDITATION PRINCIPLESVarious countries have developed specific accreditationsystems. The European University Association considersaccreditation as one possible outcome of quality assuranceand defines it as a formal recognition of the fulfillment ofminimum, publicly stated standards referring to the quality of


a programme or an institution. Accreditation is the adequatemechanism for assuring minimum standards of educationand, in some cases, can be seen as the first step towardquality. It must be used, however, in combination with robustinstitutional quality review.

Periodic self-assessment of each institution or programme isan important step in quality assurance. Self-assessment carriesmore weight; however, [only] if it is accompanied by an externalassessment phase performed by independent assessmentagencies. In turn, ensuring an international dimension willcontribute to the quality of national assessments.

Communication among national systems is still poor, andthere is a quality information gap. It is necessary that differentnational systems accept a univocal significance of accreditation.

The European University Association considers that thetime has come to take steps towards making [the] accreditationstandards of various European countries compatible with oneanother, through bilateral or multilateral agreements. Atpresent, however, there is no need to develop a singleEuropean accreditation system, but it is timely to think aboutcriteria and mechanisms to validate the accreditationprocedures applied in Europe.

Specifically, the European University Association regardsthe following principles as central to accreditation procedures.They must be:

a. geared at quality enhancement which means that theprocess will focus on the internal quality controlmechanisms in the institution and insure that these areused for strategic planning;

b. preserve institutional diversity and autonomy as well asfoster innovation by evaluating institutions against theirmissions and strategic plans;

c. assure public accountability by (1) including stakeholdersin the process, (ii) communicating the results to thepublic, and (iii) being independent of governments,interest groups, and higher education institutions;


d. consist of a self-evaluation and an external assessmentwith a clear emphasis on self-evaluation as a formativestep in institutional planning;

e. have guidelines that are transparent to the highereducation institutions and the public;

f. set up a procedure that makes clear distinctions betweenconditions for accreditation and recommendations forimprovement;

g. have a specified and fair appeals procedure;h. be re-assessed on a cyclical basis in terms of the adequacy

of an agency's resources and its impact on institutions.The European University Association encourages institutions

to ensure the internal review of their programmes andsupports initiatives in Europe to promote defined andappropriate mechanisms for the accreditation of institutions.

4. EUROPEAN AND INTERNATIONAL CO-OPERATIONIn the context of globalization and internationalization, qualityassessment implies, more than ever, comparing approachesand results, as well as learning from good practice. It isnecessary and beneficial to extend international co-operationamong institutions in view of implementing quality assessmentand assurance mechanisms, improving the assessment ofacademic programmes, sharing assessment methods, andexchanging experience.

The European University Association encourages thenetworking of institutions in matters of comparingorganizations and results, ensuring co-operation in designingand improving quality assessment methods, comparing anddeveloping quality assessment systems, making publicexamples of good practice, and sharing experience (e.g., on theintroduction of new degrees or ECTS) these are effectivemeans to consolidate the quality of higher educationprogrammes. To this end, the European University Associationwill co-operate closely with ENQA (European Network forQuality Assurance), ESIB, OECD, UNESCO-CEPES, and otherinternational organizations and institutions concerned with thequality of higher education.


The European University Association supports thedevelopment of a database regarding higher educationsystems, dissemination of information on innovation, andelaboration of comparative studies of systems and institutions.

5. CONCLUSION

In the present context of changes brought about byglobalization and the internationalization of the academicsector, the European University Association and its memberssupport steps taken by institutions, in partnerships withgovernments, towards:

a) operating changes towards the expansion of theEuropean dimension of higher education and thecreation of the European Higher Education Area;

b) curricular reform;c) improving academic management, including the capacity

for internal quality management;d) developing universities as teaching, learning, research,

and service-providing units;e) expanding and consolidating scientific research in

universities;f) adopting compatible mechanisms for quality assessment;g) achieving convergent education systems by rendering

them comparable and compatible, based on commondenominators with a European dimension.

(Approved by the EUA Council, Dubrovnik, 27 September 2001<http: / /www.unige.ch /eua>)

4.7. ACCREDITATION AND RECOGNITION: SOMEINTERNATIONAL INITIATIVES BY THE ENGINEERINGPROFESSION

4.7.1. The European Federation of National EngineeringAssociations (FEAN1) <http: / / www.feani.org>

The European Federation of National Engineering Associationswas established in the early 1950s and today brings togethermore than eighty national engineering associations from


twenty-seven European countries. Through these associations,FEANI represents the interests of approximately two millionengineers in Europe.

The objectives of FEANI areto affirm the professional identity of the engineers ofEurope;

by ensuring that professional qualifications of engineersof the member countries are acknowledged in Europeand worldwide;by asserting the status and responsibility of engineersin society;by safeguarding and promoting the professional interestof engineers and by facilitating their free movementwithin Europe and worldwide;

to strive for a single voice for the engineering professionof Europe whilst acknowledging its diversity;

by developing a working co-operation with otherinternational organizations concerned with engineeringmatters;in representing the engineers of Europe in internationalorganizations and other decision-making bodies.

The Federation established the designation, EurIng, toencourage the continuous improvement of the quality ofengineers by setting, monitoring, and reviewing standards andto facilitate recognition and mobility.

FEANI is different from the Washington Accord (see below)in that not all of its member organizations have responsibilityfor national accreditation or quality assurance of engineeringprogrammes.

4.7.2. The Washington Accord <http: / / www.washington.accorcLorg>

The Washington Accord is a multinational agreement signed in1989 which:


recognizes the substantial equivalency of accreditationsystems of organizations holding signatory status and theengineering education of programmes accredited by them;establishes that graduates of programmes accredited bythe accreditation organizations of each member nationare prepared to practice engineering at the entry level.

The member nations are: Australia, Canada, Hong Kong,Ireland, New Zealand, South Africa, the United Kingdom, theUnited States, and Japan (provisional status). The terms of theAccord state that the accreditation system, for which eachsignatory is responsible, shall be subject to a comprehensivereview and report by representatives of the other signatories, atintervals of not more than six years. The review involves sitevisits. It is widely held as an example of good practice in themutual recognition of agencies, and the Accord is nowdeveloping a number of quality assurance instruments such asdraft principles for the signatories when they are operatingoutside their home territories.

To date, European attempts to become signatories to theAccord have not been successful perhaps in part because thereare no real European counterparts which meet the criteria tobecoming signatories other than in the United Kingdom andIreland.

4.8. WHAT DO QUALITY ASSURANCE NETWORKS DO?THE EUROPEAN NETWORK FOR QUALITY ASSURANCEIN HIGHER EDUCATION (ENQHA)

The European Network for Quality Assurance in HigherEducation (ENQA) was officially established in March 2000 topromote European Co-operation in the field of qualityassessment among all the different actors operating in thisarena. The idea for ENQA originated from a European PilotProject for Evaluating Quality in Higher Education that clearlydemonstrated the importance of sharing information anddeveloping experience in quality assurance. This concept wasofficially given momentum by the European Council

164


Recommendation of September 1998 (98/561/EC) onEuropean co-operation in quality assurance in highereducation as well as by the Bologna Declaration.

The basic task of ENQA is to share experiences on aEuropean level about good practices and methods ofevaluation. This task is important as information about goodpractices and new methods and innovations should beavailable to everyone. The Network tries to disseminateinformation as much as it can through free reports andpublications, newsletters, and the ENQA Website(<http: / /www.enqa.net>) which is updated on a regular basis.ENQA has, so far, published four reports. All the publicationsare available on the Website as well as in paper copies.

ENQA is also a European education network funded by theEuropean Commission and helps to bring evaluation expertiseto ministerial staff and to the evaluation experts of evaluationagencies.

ENQA arranges workshops of its own and participates inothers to promote contacts between European experts. Themembers of the Network Steering Group travel frequently tovarious European and international conferences as guestspeakers.

The membership of ENQA is open to quality assuranceagencies, public authorities, and quality associations that workin one or more subject fields and that are external to highereducation institutions. The Secretariat is often contacted byorganizations willing to join the Network. While the organizationwelcomes new applicants, the most important issue is indeedtransparency and delivering information. The network hasdetailed membership criteria (see its Website) which includeevidence of the independence of the member institutions.

The annual General Assembly is the decision-making bodyfor the network and includes representatives of all theEuropean Ministries of Education and member organizations.The Network has a ten-member Steering Group that preparesissues for the General Assembly and plans current and futureactivities for the Network.


The European Commission pays a contribution annually tothe activities of ENQA, but apart from that, the memberorganizations pay an annual membership fee.

The priority themes for ENQA for 2001-2002 are as follows:the quality assurance and assessment of new forms ofdelivery;the outcomes and follow-up of external assessment;the use of outcomes of assessment by other stakeholders;mutual recognition of the work of other qualityassessment agencies;the European dimension in quality assurance;accreditation, European standards, and equivalence' ofprogrammes.

The themes of upcoming ENQA seminars, which are open tomembers and non-members and provide opportunities for stafffrom agencies and higher education institutions across Europeto interact and exchange views, include:

benchmarking between agencies;staff development.

The ENQA Secretariat comprises two persons and iscurrently located in the Finnish Higher Education EvaluationCouncil (FINHEEC), in Helsinki, Finland. Further details canbe obtained from the Website or by contacting the Secretary,

Kauko HamalainenENQA SecretariatP.O. Box 1425Annankatu 34-36AFIN-00101 HelsinkiFax: +358-9-1607-6911

6. OTHER NETWORKS

International Network for Quality Assurance Agencies forHigher Education (INQAAHE) <http://www.inqaahe.nl>.

Annex 5

The Design of Study Programmes:Establishing External Reference Points

for Quality and Standards

Quality assurance and accreditation agencies (QAAAs) are notinvolved in the design and delivery of programmes of study.This role has been that of either ministries and/or universities.However, quality assurance and accreditation agencies mayindirectly influence the design and delivery of programmesthrough the external reference points they establish for qualityand standards or through their evaluations and recommendationsfor action.

Here are some examples of current work on thedevelopment of external reference points such as genericdescriptors for qualifications (Bachelor's and Master's Degrees)and for subject specific descriptors. Examples are fromregional (Germany), national (UK), and international initiatives.They focus on establishing descriptors for BA/MAqualifications, sometimes when such qualifications are newawards and sometimes when they are existing ones but maynot have been well defined in the past. Recent initiatives inrelation to subject (or discipline)-specific descriptors are alsoevoked. This area is one the status of which is either "work-in-progress" or "being tested".

Advice is given as to where to go for further information.

173

167


5.1. THE DEVELOPMENT OF EXTERNAL REFERENCEPOINTS FOR QUALITY AND STANDARDS OF STUDYPROGRAMMES

5.1.1. Towards Shared Descriptors for Bachelor's andMaster's Degrees: An International Approach A Report fromthe Joint Quality Initiative GroupA Joint Quality Initiative group (JQI), established by theFlemish and Dutch Ministries of Education, has beenconsidering the development of descriptors for Bachelor's andMaster's Degrees (BaMAa descriptors) that might be sharedwithin Europe and be available for a variety of purposesdepending on particular national, regional, or institutionalcontexts and requirements. A group with members fromseveral national or regional quality assurance organizations(Belgium, Denmark, Germany, Ireland, the Netherlands,Norway, Spain, Sweden, and the United Kingdom), for bothuniversity and non-university higher education, has discussedthe diverse requirements for, and characteristics of, suchBaMa descriptors and has developed descriptors that may nowbe tested and shared.

Several other national and regional projects have been orare currently working to identify the characteristics associatedwith particular higher education qualifications, and developtaxonomies and frameworks that clarify the relationshipsamong qualifications. The work of the JQI has includeddetailed consideration of such projects and has additionallydrawn on the outcomes of discussions in Helsinki on commoncharacteristics of Bachelor's Degrees. The Helsinki discussionscharacterized Bachelor's Degrees by the extent of study (years[3 or 4] or ECTS credits [180 or 240]). The work of the JQIgroup has been concerned with identifying the academic' andother requirements that, as the outcomes of study,characterize and distinguish between Bachelor's Degrees andMaster's Degrees.

A survey was carried out amongst participants in the JQIproject in preparation for discussions on the possible form,content, and application of BaMa descriptors. Responses

168

DESIGN OF STUDY PROGRAMMES 175

indicated a variety of needs and potential uses for suchdescriptors and also the importance of having a sharedunderstanding of the terms used both within the descriptorsand to describe the context(s) in which they applied.

There was agreement that each descriptor should indicatean overarching summary of the outcomes of a wholeprogramme of study. The descriptor should be concerned withthe totality of study and with the student's abilities andattitudes that have resulted in the award of the qualification.The descriptor should not be limited solely to describing theoutcomes of units of assessment at the level of thequalification. Thus, the JQI group has developed a sharedqualification descriptor not a shared level descriptor. However,it was noted that within some national, regional, andinstitutional contexts, there might also be a requirement forthe local development of level descriptors.

The JQI group discussed the merits of seeking a singleshared descriptor for the Bachelor's Degree and similarly, onefor the Master's Degree rather than seeking a process todemonstrate "compatibility" between descriptors developed fornational, regional, or institutional purposes and that reflect thedetail of local contexts. In line with the spirit of the BolognaDeclaration, the JQI group concluded that it should seek asingle generic descriptor for all Bachelor's Degrees, andsimilarly, a single generic descriptor for all Master's Degrees.The development of these descriptors should not hinder anynational, regional, or local requirements for additionaldescriptors.

There are a wide variety of programmes leading toBachelor's awards, varying in content, delivery and process,and nomenclature. For example, a number of countriesdifferentiate between Professional Bachelor's and AcademicBachelor's awards (see ZEvA criteria, below, as an example).Similarly, a wide variety of programmes leading to differenttypes of Master's Degrees exists. It was agreed by the JQIgroup that the value of generic descriptors could be cross-referenced to more detailed programme profiles or specifications.


A programme specification or profile would identify theparticular components of the programme leading to thequalifications; for example, it might include pre-requisites forentry to the programme, details of the components, theirdelivery and assessment, and any requirements relating to theneeds of the regulated professions. The form and componentswithin the profile would reflect national, regional, orinstitutional contexts and be related to the needs andresponsibilities of those awarding the qualification oraccrediting the particular programme.

The JQI group agreed that the shared descriptors should bedescribed in a language and style that is "readable" by all whowould have an interest in them, in particular students, theirsponsors, employers, higher education academics and theirmanagers, and society at large.

The JQI group proposed the following as generic descriptorsthat might be useful as indicators or reference points for theabilities and qualities of holders of Bachelor's and Master'sDegrees awarded within the European higher education area.

Shared Descriptors for the Bachelor's and Master's DegreesBachelor's Degrees are awarded to students who:

have demonstrated knowledge and understanding in afield of study that builds upon and supersedes theirgeneral secondary education, and is typically at a levelthat, whilst supported by advanced textbooks, includessome aspects that will be informed by knowledge at theforefront of their field of study;can apply their knowledge and understanding in amanner that indicates a professional' approach to their

1 The word, "professional", is used in the descriptors in its broadest sense, relating to thoseattributes relevant to undertaking work or a vocation and that involves the application ofsome aspects of advanced learning. It is not used with regard to those specificrequirements relating to regulated professions. The latter may be identified with theprofile/specification.

170


work or vocation, and have competencies2 typicallydemonstrated through devising and sustaining argumentsand solving problems within their fields of study;have the ability to gather and interpret relevant data(usually within their fields of study) to inform judgmentsthat include reflection on relevant social, scientific, orethical issues;can communicate information, ideas, problems, andsolutions to both specialist and non-specialist audiences;have developed those learning skills that are necessaryfor them to continue to undertake further study with ahigh degree of autonomy.

Master's Degrees3 are awarded to students who:have demonstrated knowledge and understanding that isfounded upon and extends and/or enhances thattypically associated with Bachelor's level and thatprovides a basis or opportunity for originality indeveloping and/or applying ideas, often within aresearch4 context;can apply their knowledge and understanding andproblem solving abilities in new or unfamiliarenvironments within broader (or multidisciplinary)contexts related to their fields of study;have the ability to integrate knowledge, to handlecomplexity, and to formulate judgments with incompleteor limited information, but that include reflecting onsocial and ethical responsibilities linked to theapplication of their knowledge and judgments:

2 The word, "competency", is used in the descriptors in its broadest sense, allowing for ascale of abilities and skills. It is not used in the narrower sense identified solely on thebasis of a yes/no assessment.3 Some JQI representatives suggested that MBA programmes be specifically excluded;others consider that MBA programmes should reflect the attributes contained within theshared Master descriptors.4 "Research" is used to cover a wide variety of activities, with the context often related to afield of study; the term here is used to represent a careful study or investigation based on asystematic understanding and critical awareness of knowledge.


can communicate their conclusions, and the knowledgeand rationale underpinning these, to specialist and non-specialist audiences clearly and unambiguously;have the learning skills to allow them to continue tostudy in a manner that may be largely self-directed orautonomous.

Members of the JQI group have opened discussions aboutoptions of testing the BaMa descriptors in joint pilot studiesthat involve different approaches to quality assurance. Suchstudies will investigate the utility of the descriptors, and inparticular their form, components, and levels of expectation. Inaddition to contributing to transparency concerning the natureof Bachelor's and Master's Degree qualifications, it isanticipated that such transnational investigations will alsocontribute to enhancing the understanding and recognition ofthe various purposes and characteristics of different evaluationsystems.

Agencies and organizations which have participated in thediscussions and drafting of the shared BaMa descriptors include:

Belgium (FL)

DenmarkGermanyIrelandNetherlands

NorwaySpain

SwedenUnited Kingdom

For further details,j ointquality. com >.

:Ministry of Education; VLIR (University

: EVA: MKW Niedersachsen; ZevA Hannover: HETC; NQAI: Ministry of Education; Inspectorate;

HBO wad; VSNU;Trailblazer Committee.

: Network Norway CouncilAgenqua Catalunya; Consejo desUniversidades

: Hoskoleverket: QAA

visit the JQI Website on <http:/ /www.

5.2. THE DESIGN OF STUDY PROGRAMMES

An example of (DRAFT) Standards for BaMa courses whensuch qualifications are a new development is taken from an


accreditation and evaluation agency that participated in theJQI process.

5.2.1. The Central Evaluation and Accreditation Agency,Hannover, Germany (ZevA)ZevA is a common institution of all Lower Saxon universities. Itwas established in 1995, and as of 2000, it assumed anaccreditation as well as evaluative role. It is accredited by theAkkreditierungsrat and has participated in the Joint QualityInitiative group.

GENERAL STANDARDS FOR THE ACCREDITATION OF NEW DEGREECOURSES: BACHELOR'S DEGREE MASTER'S DEGREE CONTINUING

EDUCATION

CONTENTS

1. Guidelines2. Research/Application Orientation3. General key Qualifications4. Internationalization5. Co-operation Agreements/Teaching Imports6. Student/teacher Ratios7. Modularization8. Additional general Standards of the Central Evaluation and

Accreditation agency in Hanover

PREAMBLE

With this draft, the Central Evaluation and AccreditationAgency in Hanover (ZevA) is attempting to develop central andgeneral standards to be applied in the accreditation of newdegree courses. They are initially aimed at Bachelor's andMaster's Degree level courses, but can also be developed toother courses and degrees at a later stage, for instance, forPhD programmes.

The general standards of ZevA do not intend to replace anyframework examination regulations, nor are they to beunderstood as static markers. The traditions of the individual


subjects and of the disciplines at the institutions of highereducation, many of which vary greatly, make it necessary todeviate from certain standards here and there, for exampleconcerning the question of compulsory field trips, theproportion of foreign languages in the degree course, studyabroad, certain key qualifications, etc. What is important isthat the totality of the general standards permit a generalevaluation of the quality of teaching and studies against thebackground of the requirements laid down by the EducationMinisters' Conference (KMK), the Conference of Presidents andRectors of Universities and Other Higher EducationInstitutions (HRK), and the Accreditation Council (AR), andthat the trend is for at least these standards to be achieved.For this reason, the general standards of ZevA are also to beapplied within the context of these general guidelines. Someexamples of such application are the reference framework forBachelor's and Master's Degree courses of the AccreditationCouncil and the recommendations of the KMK and HRK formodularization which, in their turn, are being developedfurther.

Where degree courses deviate from these standards, thefaculties offering them ought to give their reasons.

Of course, the standards themselves must be continuallyre-examined, and they must be developed further. In order todo so, it is necessary to hold discussions with the institutionsof higher education about their concepts, and also with theministries of the Lander, the professions, the students, and thescientific and professional societies and associations. They arethus invited to participate actively in the discussions about thefurther development of general standards.

In the discussion about higher education policy, theprofessional qualifications represented by the degrees, andtherefore the relevance to practice, will be stressed as far asthe future Bachelor's and Master's Degree courses areconcerned. Thus the question of the compulsory proportion ofpractical, field trips, projects, etc., required within a Bachelor'sor Master's Degree course programme must still be answered.


In this study, ZevA has, for the time being, omitted aquantitative determination, but it will discuss the matter in itscommittees.

An additional task will consist of relating the alreadydeveloped standards for various disciplines with those drawnup here. In particular, the relationship between the cross-section qualifications and the specialist qualifications in thenarrower sense will play a part. Another question that is stillopen is whether it is not already necessary at the level ofgeneral standards to differentiate between the areas of Stage IIstudies, for instance between the humanities and the naturalsciences.

1. GUIDELINES

The basic guidelines along which the degree programmesought to be oriented if they are aiming at accreditation by ZevAare the following:

L Graduates must meet the expectations of the highereducation institution, the labour market, and society.The higher education degree awarded must be a reliableindicator that the relevant demands have been met.

ii. The examinations must reach a level and standardnecessary for the completion of the degree course andthe award of the academic degree in accordance withthe Diploma Supplement.

ill. The curriculum must be suitable for providing thenecessary qualifications and imparting the appropriateknowledge for the examinations.

iv. The resources necessary for meeting the standards mustbe available. The organization of the course of studies,the teaching, and the examinations must fulfillappropriate conditions.

v. The concepts on which the curriculum is based withregard to the qualifications to be obtained, and to theeducational goals determining the course offered, mustbe appropriate.


2. RESEARCH /APPLICATION ORIENTATION

3. GENERAL KEY QUALIFICATIONS (OF DIFFERENT TYPES DEPENDING ONTHE PROFILE OF THE DEGREE COURSE)

i. knowledge of foreign languages;ii. ability to communicate, ability to work within a team,

ability to integrate, intercultural competence;iii. presentation ability, ability to head negotiations, project

management;iv. ability to make use of modern information technologies,

knowledge of electronic learning courses on the market;v. basic economic and legal competencies, sustainability,

ethics of economics;vi. mastery of research standards, research ethics for the

BF (research oriented Bachelor's Degree) and MF(research oriented Master's Degree) courses.

GENERAL STANDARDS

In the case of Bachelor's Degree courses, greater emphasisshould be laid, in principle, on key qualifications (as long asthey are designed consecutively). In Bachelor's Degree courses,the key qualifications (ii) and (iv) are the most important. Incourses qualifying one for higher degrees, the keyqualifications (v), and (vi) are the most important.Knowledge of foreign languages is absolutely essential for alldegree courses.

4. INTERNATIONALIZATION (FOR DEGREE COURSES WITH FOREIGNORIENTATION)

L Study abroad (with registration at a foreign institution ofhigher education);

a total of two semesters as a rule; in the case ofconsecutive BF/MF;in each case, either one semester or a work placementabroad as a rule (under the supervision of the highereducation institutions): BA/MA

176


FOREIGN LANGUAGE CLASSES

on the basis of appropriate standards, confidentnegotiation ability, understanding of technical/specialistterminology, TOFEL, etc.

MODULES

at least two modules (=between 12 and 20 hrs/wk persemester) must also be offered at a foreign partnerinstitution of higher education.

5. CO-OPERATION AGREEMENTS/TEACHING IMPORTS

The development of new Bachelor's Degree courses and, aboveall, Master's Degree courses, is already showing that innovativeprogrammes frequently come into existence on the boundariesof classical subject canons. This phenomenon leadsincreasingly to so-called teaching imports from other teachingunits of the same faculty, from the student's own institution orfrom another higher education institution, and from researchand practical application. This phenomenon also raises thequestion of the minimum proportion of core competencies aresponsible faculty must have at its disposal for the degreecourse.

Minimum Curricular Core competence

70% 60%BF BA

MF

50%

MA

Teaching imports must be covered by binding agreements.Organizational measures must be taken as a reaction to theincreasing demands made on the coordination of teaching,studies, and examinations when there is a higher level ofteaching imports.

6. STUDENT /TEACHER RATIOS

The student/teacher ratios must take the individual profiles ofthe degree courses into account.


7. MODULARIZATION

Bachelor's and Master's Degree courses must be offered in amodularized form.

Modularization is the concentration of subject areas (inputorientation) on thematically and temporally rounded off,examinable units which are complete in themselves and whichare provided with credit points. Modularization is thus anorganizational principle that leads to a reorganization of thestructure of a course of studies. In the interests of enablingstudents to complete a course, and of student internationalmobility, only in exceptional cases should modules last morethan one semester.

Modularization demands mutual recognition amonginstitutions of higher education. Modularization must belinked consistently with a credit point system (in accordancewith ECTS) which takes into account the students' learningeffort, and with an examination system involving continuousassessment.

Modules must impart definable abilities (outputorientation). They can be comprised of various forms ofteaching and learning. The descriptions of modules mustinclude the effort required on the part of students and also thecredit points to be awarded.

The description of a module must contain the following as aminimum:

i. The content and qualification targets of the module;Teaching forms;Requirements for participation;

iv. The usability of the module;v. Requirements for the awarding of credit points;vi. Amount of work required, credit points, and grading;vii. How frequently the modules are offered;viii. Duration of the modules;ix. Teaching staff.

Eight (+/-two) hrs/wk per semester should be taken as aguideline for the quantitative extent of a module.

178


8. ADDITIONAL GENERAL STANDARDS OF THE CENTRAL EVALUATIONAND ACCREDITATION AGENCY IN HANOVER

Obligatory courses must be offered each semester if the degreecourse is planned to begin each semester; otherwise annually.

Each semester, the faculty (departmental adviser) mustpublish a timetable (with details of classrooms) and a fulldescription of the courses offered (module catalogue) for thestudents or provide it on the Internet.

The admission requirements as well as the selection criteriafor admission must be laid down in (state-approved)regulations.

Modularization and the use of ECTS on the basis ofdifferentiated and transparent criteria for the studentworkloads must be laid down in binding examinationregulations.

At least one introductory event should be held eachsemester when students are admitted. Each module for whichcredit points are awarded must be examined in the course ofthe semester (with the possibility of retaking the examinationat the beginning of the following semester).

The teaching staff must have a sufficient academicplurality. The profile of the course of study and the title of thedegree must be clearly coherent.


Research and Application Orientation Complementing theReport of the AR (Reference Framework for Bachelor's andMaster's Degree Courses):

Bachelor's Degree

Research

Teaching of

Basic scientific principles (50%)

Methods (20%) andknowledge of the subject (30%)As generalist research - orientedbasic training for starting aprofessional career

Oriented Ability to participate in research tasks

Broad generalist basic knowledgePlacement semester desiredSubject oriented dissertationAbility to obtain further qualificationin Master's programmeBF (research-oriented Bachelor'sDegreeTeaching of

Basic scientific principles (30%)

ApplicationOriented Comprehensive ability to participate

actively in research tasks

Methods (20%) andKnowledge of the subject (50%)

As generalist application-orientedbasic training for starting aprofessional careerAbility to take on responsible tasks inthe field of development, application,sales

Master's DegreeTeaching ofKnowledge of the subject andmethods (40%)Special knowledge (30%) andresearch abilities (30%)

For more highly qualified professionaltasks

Comprehensive ability to participateactively in research tasksTheoretical and analytical abilities

Ability to obtain further qualificationin PhD programmesMF (research-oriented Master'sDegree)Teaching ofKnowledge of the subject andmethods (20%)Special knowledge (30%) andresearch abilities (30%) for morehighly qualified professional tasks

Broad generalist specialist knowledge Theoretical and analytical abilitiesPlacement semester plannedApplication-oriented dissertationAbility to obtain further qualification Ability to obtain further qualificationin Master's programmes in PhD programmeBA (application-oriented Bachelor's MF (research oriented Master'sDegree) Degree)

Note: Dependent on the previous degree courses, Master's Degree courses can be designedas

Consecutive courses with regard to time and subject;Consecutive courses with regard to time, conversive courses with regard to subject, orcomplementary courses;Continuing education courses complementing a person's special field (usually whileworking) and taking place after a break following the first degree.

180 BEST COPY AVAILABLE


Relevant to Allthe Profiles

Bachelor's DegreesAdmission requirements:As a rule, general universityentry qualification(BF - research-orientedBachelor's Degree)As a rule, restricted universityentry qualification(BA - application-orientedBachelor's Degree)

Course length as a rule, six toeight semesters

Dissertation: 6 weeks to threemonths

Master's DegreesAdmission requirements:

Bachelor's Degree

In the case of continuing educationcourses:Two years of job experience

Course length: as a rule, two tofour semesters in the case ofcontinuing education courses:No restriction to course length

Dissertation: three to six months

In the case of continuing educationcourses: no dissertation required

RELEVANT TO ALL THE QUALIFICATIONS

Research orientation must be guaranteed by the scientificprofile and by research activities of the academic staff.

Application orientation must, in addition to the coursecontents and forms of study (placement semester, projects,application-orientated dissertations), be guaranteed by thescientific profile and by practice-oriented key areas as well asby development activities of the academic staff.

Information provided by ZevA as of 12 March 2002. Forfurther details contact ZevA <http://www.zeva.uni-hannover.de>.

5.3. TOWARDS SUBJECT SPECIFIC DESCRIPTORS FORQUALIFICATIONS: A NATIONAL APPROACH THE QUALITYASSURANCE AGENCY FOR HIGHER EDUCATION OF THEUNITED KINGDOM

1. Subject Benchmark StatementsSubject benchmark statements provide a means for theacademic community to describe the nature and characteristicsof programmes in a specific subject. They also representgeneral expectations about the standards for the award of


qualifications at a given level and articulate the attributes andcapabilities that those possessing such qualifications shouldbe able to demonstrate.

Subject benchmark statements are used for a variety ofpurposes. Primarily, they are an important external source ofreference for higher education institutions when newprogrammes are being designed and developed in a subjectarea. They provide general guidance for articulating thelearning outcomes associated with the programme but are nota specification of a detailed curriculum in the subject.Benchmark statements provide for variety and flexibility in thedesign of programmes and encourage innovation within anagreed overall framework.

Subject benchmark statements also provide support toinstitutions in pursuit of internal quality assurance. Theyenable the learning outcomes specified for a particularprogramme to be reviewed and evaluated against agreedgeneral expectations about standards.

Finally, subject benchmark statements may be one of anumber of external reference points that are drawn upon forthe purposes of external review. Reviewers, however, do notuse subject benchmark statements as a crude checklist forthese purposes. Rather, they are used in conjunction with therelevant programme specifications, that is, the internalevaluation documentation of the institution, in order to enablereviewers to come to a rounded judgment based on a broadrange of evidence.

The benchmarking of academic standards for this subjectarea has been undertaken by a group of subject specialistsdrawn from and acting on behalf of the subject community.The work of the group was facilitated by the Quality AssuranceAgency for Higher Education, which publishes and distributesthis statement and other statements developed by similarsubject specific groups.

In due course, but not before July 2005, the statement willbe revised to reflect developments in the subject and theexperience of institutions and others who are working with it.


The Agency will initiate revision and, in collaboration with thesubject community, will make arrangements for any necessarymodifications to the statement.

More than sixty subject benchmark statements have beenproduced in this manner. They are all published on the QAAweb-site <http://www.qaa.ac.uk/cmtwork/benchmark/index.htm>.

5.4. TUNING STRUCTURES IN HIGHER EDUCATION:LESSONS LEARNED IN HOW TO ACHIEVE RESULTS

Tuning Educational Structures in Europe is a project funded bythe European Commission and coordinated by the Universitiesof Deusto (Spain) and Groningen (The Netherlands). The pilotproject has involved 105 university departments across Europeand has covered seven subject areas (five mainstream projectparticipants and two "synergy groups"). The project differs fromthe Joint Quality Initiative in that it has a subject/disciplinefocus whereas the JQI is looking at the development of genericdescriptors for BA/MA qualifications.

The project is taking place in the context of the Bolognaprocess which has stimulated debate and action (in varyingdegrees) within signatory countries to the Bologna Declaration,on issues such as the re-structuring of qualifications,programme design, and quality assurance.

The aims of the Tuning project (see <http://www.relint.deusto.es/tuning> for further details) include:

opening up the debate with a range of stakeholders inhigher education (graduates, students, and employers)on the nature and importance of subject-specific andgeneral competencies;identifying and exchanging information on commonsubject-based reference points, curricula content,learning outcomes, and methods of teaching, learning,and assessment;improving European co-operation and collaboration inthe development of the quality, effectiveness, andtransparency of European higher education by


examining ECTS credits and other suitable devices toenhance progress.

There is no intention that Tuning should lead to a unified,prescriptive, or definitive European curricula or that it shouldlead to the creation of a rigid set of subject specifications whichmight restrict or direct the content, delivery, or nature ofhigher education in Europe or to damage local and nationalacademic autonomy.

The final results of the project will be published by the endof 2002. The intention of this note is to focus on one of theobjectives of the project "to elaborate a methodology foranalyzing common elements and areas of specificity anddiversity in five subject areas: Business and Management,Educational Sciences, Geology, History, and Mathematics".The methodology was designed on the basis of four lines ofanalysis of degree programmes with the aim of making them"more legible and transparent". Such an approach would meetone of the objectives of the Bologna process that calls for the"adoption of a system of easily readable and comparabledegrees". The four lines are:

Line 1: General and academic skills;Line 2: Knowledge, core curricula, and content;Line 3: ECTS as an accumulation system;Line 4: Methods of teaching and learning, assessment and

performance, and quality.The purpose of this note is not to summarize the findings in

respect of the objective but briefly to describe the steps thatthe participants took in order to work towards defining leveldescriptors for the first and second cycle studies in five subjectareas in terms of knowledge, core curricula, and content (Line2). The project managers observed that there had been fourphases of development:

Phase 1: InformingPhase 2: StormingPhase 3: NormingPhase 4: Performing

184


In Phase 1, within their subject groups, members informedeach other about the current situation in their institutionsconcerning the types of programmes of study being designedand of future perspectives. Various synergy groups that hadnot been involved in the "informing" session then added to thisinformation.

Phase 2 was characterized by "questioning everything andanything". Hot discussions and long hours were necessary to"get the steam out!" This attitude was also carried forward intothe plenary discussions of the project Steering Group.

In Phase 3, the subject groups identified what was common,diverse, and dynamic in their subject areas. They tried to finda common framework for those elements for which it wasuseful to have clear reference points. At the same time,differences were highlighted and tested to determine whetherthey were in fact useful divergences and as such anenrichment.

Finally, in Phase 4, the subject groups performed smoothly.Agreements were reached, ideas outlined, and everybody feltready to take the project forward. There were strict deadlinesto be met in the project, but all groups ended up able topresent their results in a proper form and to makerecommendations in regard to level descriptors in the fivesubject areas to a wider public at the final conference of theTuning project on 31 May 2002.

Success factors for this project to contribute to aspects ofprogramme design appear to have included:

clear definitions of project aims and objectives;a tightly defined time scale;a small incentive for involvement of departments interms of modest project funding;freedom for academics to exchange views on subject-related competencies in a developmental environment;cross-fertilization of ideas across the subject groupsthrough plenary discussions and with stakeholders;


dynamic subject group chairpersons who were notalways from the subject group concerned thus notbiased in terms of any particular approach to a subject;effective project management.

Similar projects have taken place at national level (e.g., inthe United Kingdom to develop subject benchmarkstatements).

Part III

References, Bibliographies,

and Web Resources

In addition to listing the references actually cited in the maintext of this handbook, this section also includes several topicaland thematic bibliographies as well as lists of Web resources.As an additional aid to the reader, some of the works listed areannotated. Given the overlapping of the topics and themescovered and the comprehensiveness of the references, a certainamount of repetition in the listing of resources was inevitable.

1. LIST OF REFERENCES

ADELMAN, C., A Parallel Post-Secondary Universe: theCertification System in Information Technology. WashingtonD.C.: United States Department of Education, Office ofEducational Research and Improvement, 2000.

Assessment of Higher Education in the Baltic States, 1996,<http://www.culture.coe.int>.

Bozena Kenig Predpisi v visokem golstvu [The Higher EducationAct]. Ljubljana: The Ministry. for Education and Sport,1996.

COUNCIL OF EUROPE LEGISLATIVE REFORM PROGRAMME FOR HIGHEREDUCATION, BALTIC STATES CO-OPERATION FOR QUALITYASSURANCE IN HIGHER EDUCATION. Report of the RegionalAdvisory Mission, Riga, 24-26 October 1994,<http:www//culture.coe.int>.

COUNCIL OF EUROPE LEGISLATIVE REFORM PROGRAMME FOR HIGHEREDUCATION, BALTIC STATES CO-OPERATION FOR QUALITYASSURANCE IN HIGHER EDUCATION. Report of the AdvisoryMission on Quality Assessment and Accreditation, Riga, 19-22 February 1995, <http://www.culture.coe.int>.

193

182


COUNCIL OF EUROPE LEGISLATIVE REFORM PROGRAMME FOR HIGHEREDUCATION, BALTIC STATES CO-OPERATION FOR QUALITYASSURANCE IN HIGHER EDUCATION. Report of the AdvisoryMission on Quality Assessment and Accreditation, Riga, 19-22 February 1995, <http: / /www.culture.coe.int>.

"The Czech Republic", in, Individual Country Reports, 2002<http: / /www.ceenetwork.hu>.

Elet es Irodalom [Life and Literature] 18 January 2002, et seq.[discussion of the current situation of higher education inHungary] .

EL-KHAWAS, E. "Quality Assurance in Higher Education: RecentProgress; Challenges Ahead", Paper presented at theUNESCO World Conference on Higher Education, Paris,1998.

ESTONIA, GOVERNMENT OF THE REPUBLIC. Higher EducationStandard. Regulation No. 190 of the Government of theRepublic. Tallinn, Ministry of Education, 2000 [unofficialEnglish translation, unpublished].

ESTONIA, MINISTRY OF EDUCATION. Law on Universities, Adoptedby the Parliament of the Republic of Estonia, 12 January1995. Tallinn: Ministry of Education, 1995.

THE EUROPEAN TRAINING FOUNDATION. Quality Assurance in HigherEducation: Final Report and Project Recommendations.Turin: The European Training Foundation, 1998.

FRANKE, S. "From Audit to Assessment: A National Perspectiveon an International Issue", Quality in Higher Education 8 1(2002).

HARVEY, L., and GREEN, D. "Defining Quality", Assessment andEvaluation in Higher Education 18 1 (1993).

"Higher Education Act", in, Pavel ZGAGA and Tatjana JuRKoviC,Higher Education in Slovenia. Ljubljana: Ministry ofEducation and Sport, 1995, Section III. Article 33. p. 21.

JONES, D. P. Different Perspectives on Information aboutEducational Quality: Implications for the Role ofAccreditation. CHEA Occasion Paper. Washington D. C.:Council for Higher Education Accreditation, 2002.

RESOURCES 195

KELLS, H. R. Self-Regulation in Higher Education: AMultinational Perspective on Collaborative Systems of QualityAssurance and Control. London: Jessica Kingsley, 1992.

KRISTOFFERSEN, Dorte, SURSOCK, Andree, and WESTERHEIJDEN,Don. Quality Assurance in Higher Education: Manual ofQuality Assurance: Procedures and Practices. Turin: TheEuropean Training Foundation, 1998.

LOMAS, L. "Does the Development of Mass EducationNecessarily Mean the End of Quality?" Quality in HigherEducation 8 1 (2002).

MATOLCSY, Janos. "A magyar felsOoktatas finanszirozasirendszerenek reformja" [The Reform of the FinancialStructure of Hungarian Higher Education], MagyarFelsooktatas 9 (2001): 19-20.

MATOLCSY, Janos. "MegkezdOdott a Finanszirozasi RendszerReformja projekt" [The Financial System Reform Project HasStarted], Magyar Felsooktatcis [Journal of the Ministry ofEducation]. 1 2 (2002): 22-24.

MIDDLEHURST, R. Quality Assurance Implications of New Formsof Higher Education. Helsinki: European Network for QualityAssurance, 2001 <http://www.enqa.net>.

MoKoSIN, Vladislav, ed. Higher Education in the Czech Republic.Prague: The Centre for Higher Education Studies, 1999.

"1993. evi LXXX. torveny a felsooktatasrOl" [The HungarianHigher Education Law LXXX/1993], Akademiai ertesft6 (15September 2000): 72-105.

PoWNYI, Istvan, and TimAR, Janos. Tudasgyar vagy papirgyor[Knowledge Factory or Paper Factory]. Budapest: IJjMandatum, 2001.

ROZSNYAI, Christina, "Changing Focus: Internal Quality Auditas an Element in External Quality Evaluation", TertiaryEducation and Management 7 (2001): 341-344.

SEDDON, J. The Case Against ISO 9000. 2nd ed. Dublin: OakTree Press, 2000.

SMEBY, J. C., and STENSAKER, B. "National Quality AssessmentSystems in the Nordic Countries: Developing a Balance


between External and Internal Needs", Higher EducationPolicy 12 (1999): 3-14.

Statistical Yearbook of Hungary. Budapest: Hungarian CentralStatistics Office, 2000.

"Students and Graduates in Slovenia, 1919-1993", in, PavelZGAGA and Tatjana JuRKoviC, eds. Higher Education inSlovenia. Ljubljana, Ministry of Education and Sport, 1995,Supplement 1, p. 112.

TOMUSK, Voledemar. "When East Meets West:Decontextualizing the Quality of East European HigherEducation", in, Quality in Higher Education 6 3 (2000): 175185.

VAN DAMME, D. 'Trends and Models in International QualityAssurance and Acreditation in Higher Education in Relationto Trade in Education Services", Paper presented at theOECD/US Forum on Trade in Educational Services, 23-24May 2002.

WOJCICKA, Maria, and CHMIELECKA, Ewa. 'The Polish System ofQuality Assurance: Results of a Top-Down and Bottom-UpInitiative (Research Findings)", Paper presented at theINQAAHE Conference March 19-22, 2001 in Bangalore,India <http://www.inqaahe/nl/parallelpapers.html>.

WOJCICKA, Maria, and CHWIROT, Stanislaw. "Quality Assurance:Poland Against the Background of Other Countries", in,Maria WOJCICKA and Jolanta URBANIKOWA, eds. ExternalQuality Assurance in Higher Education. Mechanisms,Procedures, Tools [Manuscript excerpts in English fromMaria WOJCICKA and Jolanta URBANIKOWA, eds. ExternalSystems of Quality Assurance in Higher Education (inPolish)]. Warsaw: Industrial Press, 2001, p. 7.

WOJCICKA, Maria, and URBANIKOWA, Jolanta, eds. ExternalSystems of Quality Assurance in Higher Education [inPolish]. Warsaw: Industrial Press, 2001.

THE WORLD BANK. Constructing Knowledge Societies: NewChallenges for Tertiary Education. Washington D.C.: TheWorld Bank, 2002.

RESOURCES 197

ZGAGA, Pavel, and JURKOVIc, Tatjana, eds. Higher Education inSlovenia. Ljubljana: The Ministry for Education and Sport,1995.

2. BIBLIOGRAPHIES

2.1. Central and Eastern Europe: Useful Publications on aCountry by Country Basis

ALBANIA

"Law for the High Civil Education in the Republic of Albania",No. 8461, 25/02/1999. Bilingual excerpt on the"Foundation of the Accreditation system for HigherEducation", Decision of the Council of Ministers No. 303,date 01/017/1999, in, Sistemi I akkreditimit ne arsimin elarte. Republika e shqiperise, agjensia e akkreditimit toarsimit to larte. Tirana, 2000.

BULGARIA

GEORGIEVA, Patricia. "New Endeavours for Higher EducationQuality Assurance: Results from the Pilot InstitutionalEvaluation in Bulgaria", Higher Education Management 12 2(2000): 97-115.

HUMAN RESOURCES DEVELOPMENT, CANADA. ComparativeAnalysis of Higher Education Systems in Central andEastern Europe. Ottawa: Canadian Bureau for InternationalEducation, 2000, pp. 3-40.

MARTINOV, Nicolay. 'The Challenge in Accreditation of HigherEducation" [in Bulgarian], Science: The Journal of the Unionof Bulgarian Scientist 11 4 (2001): 5-8.

MARI1NOV, Nicolay. 'The National Evaluation and AccreditationAgency" [in Bulgarian], in, Higher Education 2 (2001): p. 60-71

MAR'I1NOV, Nicolay. The Role of the National Evaluation andAccreditation Agency within Quality Assurance of BulgarianHigher Education [in Bulgarian]. Sofia: The Centre of HigherEducation at the Open Society Foundation, 2001.


'The Regulations of the Activity of the National Evaluation andAccreditation Agency", State Gazette 52 (27 June 2000): 2-6.

REPUBLIC OF BULGARIA. Laws for Education [Public EducationAct, Higher Education Act, Law on the Level of Schooling,The General Education Minimum and the Syllabus, and theRepublic of Bulgaria 38th National Assembly VocationalEducation and Training Act]. Sofia: European TrainingFoundation, Human Resource Development Centre, St.Kliment Ohridski University Press, 1999.

CROATIA


LELAS, S., and MENCER, Helena J. "Croatia: System of HigherEducation", in, Burton R. CLARK and Guy NEAVE, comps.Encyclopaedia of Higher Education. Oxford: Pergammon,1997.

CZECH REPUBLIC

THE ACCREDITATION COMMISSION. THE SECRETARIAT. . The Statuteof the Accreditation Commission and Act No. 111/1998 onHigher Education Institutions and on Modification andAmendment of Other Act The Enactment Connected with theAccreditation Commission. Prague: Ministry of Education,Youth, and Sports of the Czech Republic, 1998.

BENES, Josef, and SEBKOVA, Helena. Tertiary Education in theCzech Republic: Country Report. Prague: Centre for HigherEducation Studies, Ministry of Education, Youth, andSports, 2001.

MoKoSIN, Vladislav, ed. Higher Education in the Czech Republic.Prague: The Centre for Higher Education Studies, 1999.

ESTONIA

ESTONIA, GOVERNMENT OF THE REPUBLIC. The Statute of theHigher Education Quality Assessment Council. Government

RESOURCES 199

Republic. Tallinn: Ministry of Education, 2000 [UnofficialEnglish translation, unpublished].

ESTONIA, GOVERNMENT OF THE REPUBLIC. The Statute of theHigher Education Quality Assessment Council. Governmentdecree No. 179, April 1995 [Unofficial English translation,unpublished].

ESTONIA, MINISTRY OF EDUCATION. Law on Universities, Adoptedby the Parliament of the Republic of Estonia, 12 January1995. Tallinn: Ministry of Education, 1995.

OECD. "Estonian Education Policy", in, OECD Education PolicyAnalysis. Paris: OECD, 2001 <http: / /www.oecd.org>.

PHARE. Strategic Study on Legislation, Accreditation,Recognition, and Quality Assurance Applied to Open andDistance Learning in Central and Eastern Europe. Reportand Recommendations. Turin: European TrainingFoundation. 1999.

Tomusii, Voldemar, "External Quality Assurance in EstonianHigher Education: Its Glory, Take-off and Crash", Quality inHigher Education 3 2 (1997): 173-181.

VAHT, G., UDAM, M., and Kffr, K., eds. Higher Education inEstonia. Tallinn: Ministry of Education and the EstonianAcademic Recognition Information Centre, 2000.

HUNGARY

Elet es Irodalom [Life and Literature] 18 January 2002, et seq.[Discussion of the current situation of higher education inHungary].

GOVERNMENT OF HUNGARY. "A Kormany 199/2000 .(X1.29.)Korm. Rendelete a Magyar AldireditaciOs BiyottsagmilkOdeserol" [Government Decree 199/2000 (18November) on the Operation of the Hungarian AccreditationCommittee], Magyar KOzlony 116 (2000): 7251-7260 [inHungarian].



MATOLCSY, Janos. "Megkezdodott a Finanszirozasi RendszerReformja projekt" [The Financial System Reform Project HasStarted], Magyar FelsOoktatets [Journal of the Ministry ofEducation] 1 2 (2002): 22-24.

MEszARos, Gabor. "Killfoldi felsooktatasi intezmenyekMagyarorszagon" [Foreign Higher Education Institutions inHungary], Magyar FelsOoktatos [Journal of the Ministry ofEducation] 1 2 (2002): 16-18.

"1993. evi LXXX. torveny a felsooktatasrOl" [The HungarianHigher Education Law LXXX/1993], Akademiai ertesito (15September 2000): 72-105. [Main amendments 1996, 1999,2000. 1996, English version at <http://www.om.hu/english>.]

POLONYI, Istvan, and TimAR, Janos. Tudasgyar vagy papirgyar[Knowledge Factory or Paper Factory]. Budapest: I.Jj

Mandatum, 2001.ROSZNYAI, Christina. "Changing Focus: Internal Quality Audit

as an Element in External Quality Evaluation", TertiaryEducation and Management 7 (2001): 341-344.

LATVIA

DZELME, Juris. "Use of General and Specific Standards for theEvaluation in Higher Education", in, Proceedings of the1NQAAHE 2001 Conference on Quality Assurance in HigherEducation: Quality, Standards, and Recognition. Bangalore,India, 19-22 March 2001, <http: / /wvvw.inqaahe/nl/parallelpapers.html>.

Education Law Adopted by the Saeima on 29 October 1998[Amended 1999, 2000] <http://www.aiknc.lv>.

LATVIA. CABINET OF MINISTERS. Accreditation Regulations forHigher Educational Establishments, Regulations No. 370.Riga, 28 November 1995 [Amended 16 October 2001]<http://www.aiknc.lv>.

LATVIA, MINISTRY OF EDUCATION AND SCIENCE. Instruction for Self-Assessment of Higher Education Institutions and StudyProgrammes. Order No. 272 of 5 June 1996<http: / /www. aiknc. Iv> .

RESOURCES 201

LATVIA, MINISTRY OF EDUCATION AND SCIENCE. MethodologicalRecommendations for Assessment of Higher EducationInstitutions and Their Study Programmes. Order No. 272 of 5June 1996 <http://www.skvc.lv>.

LATVIA, MINISTRY OF EDUCATION AND SCIENCE. Recommendationsconcerning Activities of the Assessment Commission. OrderNo. 272 of 5 June 1996 <http://www.aiknc.lv>.

Law on Higher Educational Establishments Adopted by theSaeima on 2 November 1995 [Amended 26 December 20001<http://www.aiknc.lv>.

Law on Scientific Activity Adopted by the Supreme Council of theRepublic of Latvia [Amended 1996, 1998, 20011<http://wwvv.aiknc.lv>.

LITHUANIA

CIAS, Algirdas. "Quality Assessment in Smaller Countries:Problems and Lithuanian Approach", in, Higher EducationManagement 9 1 (March 1997): 43-48.

CIAS, Algirdas. "Subject Benchmarking and AcademicFreedom", in, Proceedings of the INQAAHE 2001 Conferenceon Quality Assurance. in Higher Education: Quality,Standards, and Recognition. Bangalore, India, 19-22 March2001 <http://www.inqaahe/nl/parallelpapers.html>.

DAUJOTIS, V. "On the Idea of the European SocialInsurance/Security Fund, An Enlarged Europe forResearchers", Proceedings of the Conference, Brussels, 27-28 June 2001, pp. 67-68.

Law of the Republic of Lithuania on Education. No. 1-1489,adopted on 25 June 1991 <http://wwvv.skvc.lt>.

Law of the Republic of Lithuania on Research and HigherEducation. Adopted on 21 February 1991 <http://www.skvc.lt>.

MINISTRY OF EDUCATION AND SCIENCE OF THE REPUBLIC OFLITHUANIA. Law of the Republic of Lithuania on HigherEducation. Vilnius: Ministry of Education and Science, 2000<http://www.skvc.lt>.

MINISTRY OF EDUCATION AND SCIENCE OF THE REPUBLIC OFLITHUANIA. Regulations of the Centre for Quality Assessmentin Higher Education Approved by Order No. 190, 10 March

195


1995. Amended by Order No. 378. Vilnius: Ministry ofEducation and Science, 2001 <http://www.sk-vc.lt.>.

MINISTRY OF EDUCATION AND SCIENCE OF THE REPUBLIC OFLITHUANIA. Rules of Quality Assessment for Institutions ofResearch and Higher Education. Approved by Order No.1055, 28 June 2001. Vilnius: Ministry of Education andScience, 2001 <http://www.skvc.lt>.

MOCKIENE, Birute. Higher Education Policy in Central andEastern Europe: Lithuania Country Report Education forthe Transition. Washington D.C.: CEP, 1997.

MOCKIENE, Birute. "Transnational Higher Education inLithuania", in, International Higher Education 24 (Summer2001): 7-9.

MACEDONIA

Development of the System of Academic Personnel QualityAssessment at Universities in the Republic of Macedonia.TEMPUS Compact-Measure Project No. 03118-97,conducted during 1997-1998.

System of Quality Assurance in the Higher Education in theRepublic of Macedonia TEMPUS Compact-Measure ProjectNo. 03612-97, conducted during 1998-1999.

POLAND

BRZEZI&SKA, A., and BRzEzifisiu, J., eds. Evaluation of Educationin Higher School [in Polish] . Poznan: Humaniora FoundationPress, 2000.


KELLS, H. R. Self-Study Processes: A Guide to Self-Evaluation inHigher Education [translated into Polish by MariaSklodowska]. Lublin: University Press, 2000.

WNUK-LIPINSKA, Elzbeta, and WOJCICKA, Maria, eds. QualityReview in Higher Education. Warsaw: TEMPUS, 1995.

WOJCICKA, Maria, ed. Education Quality in Higher Education [inPolish] . Warsaw: Industrial Press, 2001.

RESOURCES 203

WOJCICKA, Maria, and CHMIELECKA, Ewa. 'The Polish System ofQuality Assurance: Results of a Top-Down and Bottom-UpInitiative (Research Findings)", Paper presented at theINQAAHE Conference, 19-22 March 2001, Bangalore, India<http : / /www. inqaahe /nl/parallelpapers. html>.

WOJCICKA, Maria, and URBANKIKOWA, Jolanta, eds. ExternalSystems of Quality Assurance in Higher Education [inPolish]. Warsaw: Industrial Press, 2001.

ROMANIA

DIMA, Ana-Maria. "Romanian Higher Education Viewed from aNeo-Institutionalist Perspective", Higher Education in Europe23 3 (1998): 397-406.

GOVERNMENT OF ROMANIA. "Government Decision 535/1999Concerning the Authorization and Accreditation of Publicand Private Higher Education Institutions", Regia AutonomyMonitorul Oficial (1999) [in Romanian].

GOVERNMENT OF ROMANIA. "Government Ordinances for theAccreditation of Eight Private Universities", Monitorul Oficial677 (26 October 2001) [in Romanian] .

GOVERNMENT OF ROMANIA. "Higher Education Act 84/1995,Amended 1999 ", Regia Autonomy Monitorul Oficial (1999) [inRomanian] .

GOVERNMENT OF ROMANIA. "Law 88/1993 Concerning theAccreditation of Higher Education Institutions and theRecognition of Diplomas", Regia Autonomy Monitorul Oficial(1999) [in Romanian].

GOVERNMENT OF ROMANIA. "Law 151/1999", Regia AutonomyMonitorul Oficial (1999) [in Romanian].


MARGA, Andrei. University Reform Today. Cluj-Napoca: ClujUniversity Press, 2001.

204 C. CAMPBELL and C. ROZSAIYAI

THE RUSSIAN FEDERATION

"Hochschulpolitik in Russ land and Deutschland. Redebeitrageim Rahmen der Deutsch-Russischen HochschulbOrse",Berlin, 2-4 May 1999, Hochschulrektorenkonferenz: Beitragezur Hochschulpolitik 11/1999. Bonn: Hochschulrektoren-konferenz, 1999.

HUMAN RESOURCES DEVELOPMENT, CANADA. ComparativeAnalysis of Higher Educhtion Systems in Central andEastern Europe. Ottawa: Canadian Bureau for InternationalEducation, 2000, pp. 309-370.

MINISTRY OF EDUCATION OF THE RUSSIAN FEDERATION. Licensing,Accreditation, and Attestation Department. Moscow:National Accreditation Center, 2000.

NATIONAL ACCREDITATION CENTER. State Accreditation: Guidanceon Preparation of Accreditation. Moscow: NationalAccreditation Center, 2000.

RUSSIAN FEDERATION. Law on Education, 1992, amended 1996,"About Higher and Post-Graduate Professional Education",22.08.96 N 125-FL, and 1999, "About Education", 13.01.99N 12-FL. Moscow, 1999.

TOMUSK, Voldemar. "Developments in Russian HigherEducation: Legislative and Policy Reform Within Centraland Eastern European Context", in, Minerva 36 2 (1998):125-146.

SLOVAKIA

Higher Education Act. Ratified by the National Council of theSlovak Republic and that went into effect on 1 April 2002.[Draft version 28082001-V159, manuscript, 99 pp.]


URBAN, Miroslay. "Operational Developments in QualityAssessment and Institutional Accreditation in the SlovakRepublic", Higher Education in Europe 20 1-2 (1995): 66-71.

RESOURCES 205

SLOVENIA

Bozena Kenig Predpisi v visokem golstvu [The Higher EducationAct]. Ljubljana: The Ministry for Education and Sport,1997.


Visokogolski zavodi v Republiki Sloven [Higher EducationInstitutions in the Republic of slovenia]. Ljubljana: TheMinistry for Education and Sport, 1996.

ZGAGA, Pavel, and JuRKoviC, Tatjana, eds. Higher Education inSlovenia. Ljubljana: The Ministry of Education and Sport,1995.

UKRAINE

GLOBAL GATEWAY. Ukraine: Education and Research. Kiev:Gateway, 2002 <http://www.ukraine-gateway.org>.


NOACK, Christian, and PENTER, Tanja. "Hochschulen in derUkraine: Geschichte, Struktur, Perspektiven",Hochschulrektorenkonferenz: Beitrage zur Hochschulpolitik 2(2001).

2.2. GENERAL READINGS OR READINGS PERTINENT TO ALLTHE CENTRAL AND EASTERN EUROPEAN COUNTRIES'

BILLING, David, and TEMPLE, Paul. Intermediary Bodies inHigher Education in Eastern Europe (Forthcoming).

COUNCIL OF EUROPE LEGISLATIVE REFORM PROGRAMME FOR HIGHEREDUCATION,

I For references to other books and studies on higher education issues in the Central andEastern European Countries see <http://www.cepesso/publications>.


Advisory Mission, Riga, 24-26 October 1994<http: / /culture. coe . int>

COUNCIL OF EUROPE LEGISLATIVE REFORM PROGRAMME FOR HIGHEREDUCATION, BALTIC STATES CO-OPERATION FOR QUALITYASSURANCE IN HIGHER EDUCATION. Report of the AdvisoryMission on Quality Assessment and Accreditation, Riga, 19-22 February 1995 <http://culture.coe.int>

COUNCIL OF EUROPE LEGISLATIVE REFORM PROGRAMME FOR HIGHEREDUCATION, BALTIC STATES CO-OPERATION FOR QUALITYASSURANCE IN HIGHER EDUCATION. Statute on the InternationalCo-operation in Quality of Assessment of Higher Education inthe Baltic States, 1996 <http: / /culture. coe.int>

HUMAN RESOURCES DEVELOPMENT, CANADA. ComparativeAnalysis of Higher Education Systems in Central andEastern Europe. Ottawa: Canadian Bureau for InternationalEducation, 2000.

Ten Years After and Looking Ahead: A Review of theTransformation of Higher Education in Central and EasternEurope. Bucharest: UNESCO-CEPES, 2000.

Tomusx, Voldemar. "Conflict and Interaction in Central andEast European Higher Education: The Triangle of RedGiants, White Dwarfs, and Black Holes", in, TertiaryEducation and Management 3 3 (1997): 247-255.

Tomusx, Voldemar. "When East Meets West: Decontextualizingthe Quality of East European Higher Education", Quality inHigher Education 6 3 (2000): 175-185.

THE WORLD BANK. Hidden Challenges to Education Systems inTransition Economies. Washington D.C.: The World Bank,2000.

RESOURCES 207

2.3. THEMATIC BIBLIOGRAPHIES2

2.3.1. The Changing Context of Higher Education: Trends,Challenges, and ResponsesADAM, S. Transnational Education Report Brussels: CEURC,

2001. [This report maps the development of transnationaleducation (education which crosses national borders) withinthe European Union and makes several recommendationsconcerning the quality assurance of such provision.]

ADELMAN, C. A Parallel Postsecondary Universe: TheCertification System in Information Technology. WashingtonD.C.: United States Department of Education, Office ofEducational Research and Improvement, 2000.

CAMPBELL, C., VAN DER WENDE, M. C. International Initiativesand Trends in Quality Assurance for European HigherEducation. Helsinki: ENQA, 2000 <http: / /www.enqa.net>.

CLARK, B. R. Creating Entrepreneurial Universities:Organizational Pathways of Transformation. Oxford:Pergamon, 1998.

CURRIE, J., and SUBOTZKY, G. "Alternative Responses toGlobalization from European and South AfricanUniversities", in, N. STROMQUIST and K. MONKMAN.

Globalization and Education: Integration and ContestationAcross Cultures. New York: Rowman and LittlefieldPublishers Inc., 2000.

CVCP, and HEFCE. The Business of Borderless Education; UKPerspectives. London: Committee of Vice-Chancellors andPrincipals of the Universities of the United Kingdom, andthe Higher Education Funding Council for England, 2000.[A comprehensive account of the virtual and corporatedevelopments in learning which along with the Internetare changing the global landscape in the delivery of highereducation services. The report alerts British universitiesand colleges to the challenge in borderless learning posed

2 These lists of publications are illustrative of the issues of quality assurance in highereducation and related matters. In some cases, short summaries are included, but only forpurposes of information, and do not constitute any kind of recommendation.

2 0 1


by their overseas counterparts, particularly in NorthAmerica. It addresses the question of how Britishuniversities may be able to rise to and capitalize on thatchallenge in the area of quality assurance. There is also asection on developments in Europe.]

DAHRENDORF, R. Universities after Communism. Hamburg:Korber-Stiftung, 2001.

HAUG, G. and TAUCH, C. Trends in Learning Structures in HigherEducation. Helsinki: National Board of Education, 2001.[This report is a follow-up, prepared for the Salamanca andPrague Conferences of March and May 2001, respectively,on progress on the implementation of the Bologna process.See <http://www.oph.fi/publications/trends2>.]

HENKEL, M. Academic Identities and Policy Change in HigherEducation. London: Jessica Kinsley Publishers, 2000.

KNIGHT, J. Trade in Higher Education Services: The Implicationsof GATS. London: The Observatory on Borderless HigherEducation, 2002 <http://www.obhe.ac.uk>.

LARSEN, K., MORRIS, R., and MARTIN, J. Trade in EducationalServices: Trends and Emerging Issues Working Paper.Paris: CERI/OECD, 2001.

MAASSEN, P. Government Steering and the Academic Culture:The Intangibility of the Human Factor in Dutch and GermanUniversities. Maarsen: De Tijdstroom, 1996.

MALLEA, J. International Trade in Professional and EducationalServices: Implications for the Professions and HigherEducation. Paris: CERI/OECD, 1998.

OECD. The Response of Higher Education Institutions toRegional Needs. Paris: OECD Publications, 1999.

Scarr, P., ed. The Globalization of Higher Education.Buckingham: Open University Press, 1998.

SPORN, B. Adaptive University Structures: an Analysis ofAdaptation to Socio-Economic Environments of US andEuropean Universities. London: Jessica Kingsley, 1999.

TASK FORCE ON HIGHER EDUCATION AND SOCIETY. HigherEducation in Developing Countries: Peril and Promise.Heernden, Virginia: World Bank Publications, 2000. [The

RESOURCES .209

Task Force on Higher Education and Society(<http://www.tfhe.org>) was convened by the World Bankand UNESCO to bring together experts from thirteencountries for the purpose of exploring the future of highereducation in the developing world. Based on research andintensive discussion and hearings conducted over a two-year period, the Task Force has concluded that withoutmore and better higher education, developing countries willhave increasing difficulties in benefiting from the globalknowledge-based economy. The report articulates thecharacteristics of an effective higher education system.]

UNESCO. Higher Education in the Twenty-First Century: A NewCommitment: Final Report of the World Conference onScience. Paris: UNESCO, 1998.

THE WORLD BANK. Constructing Knowledge Societies: NewChallenges for Tertiary Education. Washington D.C.: TheWorld Bank, 2002.

2.3.2 Governance and Management in Higher EducationBARNE'IT, R. Realizing the University in an Age of Supercomplexity.

Buckingham: SRHE/Open University Press, 2000.BAUER, M., MARTON, S., ASKLING, B., and MARION, F.

Transforming Universities: Changing Patterns of Governance,Structure, and Learning in Swedish Higher Education.London: Jessica Kingsley Publishers, 1999. [This bookexamines the implications of reforms for a higher educationsystem (Sweden) on three levels: the state, the institution,and the individual.]

BRAUN, D., and FRANCOIS-XAVIER, M., eds. Towards a NewModel of Governance for Universities? A Comparative View.London: Jessica Kingsley, 1999.[This book draws on casestudies in the United Kingdom, The Netherlands, France,Germany, Italy, The United States, and Switzerland, anddemonstrates the varied impact of increased stateintervention in higher education. It provides a comparativeoverview of developments in the governance of universitiessince the 1980s and an assessment of whether and how


changes in state intervention have transformed the politicaland organizational management of universities.]

BLEIKLIE, I., HOSTAKER, R., and VABO, A. Policy and Practice inHigher Education: Reforming Norwegian Universities.London: Jessica Kingsley Publishers, 2000. [This bookdescribes the nature of change over the last thirty years ofgrowth in the Norwegian higher education system.]

FORD, P. Managing Change in Higher Education: A LearningEnvironment Architecture. Buckingham: Open UniversityPress, 1996. [This study provides a generic model whichcan be used by managers of any individual learninginstitution in order to develop appropriate learningenvironment architecture of their own. It provides a methodfor creating a framework within which strategies, businessprocesses, and supporting information systems can bedeveloped and changed to meet the objectives of aninstitution.]

KOGAN, M., and HANNEY, S. Reforming Higher Education.London: Jessica Kingsley, 1999.[The authors focus on theways in which the changing concepts of the nature of thestate and its role have had an impact on the development ofhigher education policy in the last thirty years. The authorsstudy the dynamics behind the shift from state-subsidizedindependence to ambiguous but increased dependence onand deference to state policies. The book looks at thechanges in the machinery of the state for policyimplementation and consequential changes in institutionalgovernance.]

OECD. Redefining Tertiary Education. Paris: Organization ofEconomic Co-operation and Development, 1998.

2.3.3. Income Generation and EntrepreneurshipGRAY, H., ed. Universities and the Creation of Wealth.

Buckingham: Open University Press, 1999. [This bookprovides a basic insight into what universities currently doin terms of the creation of wealth and into what kind oforganizations they might usefully become. It explores how

RESOURCES 211

universities benefit their local and regional communities,and what more they might do to thrive locally in the globaleconomy. ]

2.3.4. The Evaluation of Institutional PerformanceCAVE, M., HANNEY, S., HENKEL, M., and KOGAN, M. The Use of

Performance Indicators in Higher Education: The Challenge ofthe Quality Movement 3rd ed. London: Jessica Kingsley,1996. [This study provides a critical introduction to thebasic concepts and practical considerations of usingperformance indicators within universities. Writtenprimarily from a British perspective, with one chapterdevoted to a comparative view, the third edition has beenthoroughly revised to take account of the most recentdevelopments in the practice, literature, and potential useof performance indicators with particular reference todevelopments in the "new" universities and colleges ofhigher education. Surveys of performance indicatorsrelating to teaching, research, and administration arepresented, as well as a discussion on the various modes ofapplication within various levels of an institution.]

SMITH, H., ARMSTRONG, M., and BROWN, S. Benchmarking andThreshold Standards in Higher Education. London: KoganPage, 1999. [A wide ranging exploration of the issues andchallenges involved in benchmarking and providing acritical analysis of the problems and potential of a powerfulbut controversial development tool in the United Kingdom,the United States, New Zealand, and South Africa.]

THACKWRAY, B. Effective Evaluation of Training andDevelopment in Higher Education. London: Kogan Page,1997. [Emphasis on the need for the development of betterand more consistent practices with regard to evaluation.This book uses relevant examples and detailed case studiesfrom the United Kingdom and elsewhere with the aim ofproviding a comprehensive guide to the best practice and ademonstration of the many benefits to be gained from theeffective use of evaluation.]


2.3.5. Human Resource ManagementFARNHAM, D., ed. Managing Academic Staff in Changing

University Systems: International Trends and Comparisons.Buckingham: Open University Press, 1999. [This studyprovides a contemporary and international analysis of howacademic staff in universities are currently managed. Itreviews recent developments in higher education policy infifteen selected countries and examines their impacts on theacademic profession. While rates of change differ, thecontributors argue that the massification, marketization,and managerialization of higher education are universal,international phenomena.]

2.3.6. Quality Assurance and Management3ASHWORTH, A., and HARVEY, R. Assessing Quality in Further and

Higher Education. London: Jessica Kingsley, 1994. [Thisbook outlines the system of Total Quality Management(TQM) in further and higher education, with illustrations ofquantitative measurement for its evaluation. This handbookof techniques is intended for managers of universities andcolleges.]

BROWN, S., RACE, P., and SMITH, B. 500 Tips for QualityEnhancement in Universities and Colleges. London: KoganPage, 1997. [Aimed at staff in higher education institutions,this book sets out to provide helpful, practical, and realisticsuggestions for the measurement and improvement ofquality. It covers the full range of quality issues including:valuing students and staff; enhancing learning throughteaching and assessment; quality processes, feedback andevaluation; preparing for assessment visits.]

EL-KHAWAS, E. "Quality Assurance in Higher Education: RecentProgress, Challenges Ahead", Paper presented at the

3 Copies of the publications listed in this section that are marked with an asterisk may beavailable from the Centre for Higher Education Research and Information (CHERI) or theOpen University at <http://www.open.ac.uk/cheri>.

RESOURCES 213

UNESCO World Conference on Higher Education, Paris,1998.

EUROPEAN COMMISSION. Evaluation of European HigherEducation: A Status Report. Prepared for the, DG XXII by theCenter for Quality Assurance and Evaluation of HigherEducation, Denmark, in co-operation with Comite Nationald'Evaluation, France. Paris, 1998 <http://www.enqa.net/does .lasso?docname=statusreportl . html>.

KRISTOFFERSEN, Dorte, SURSOCK, Andree, and WESTERHEIJDEN,Don. Quality Assurance in Higher Education: Manual ofQuality Assurance: Procedures and Practices. Turin: TheEuropean Training Foundation, 1998.

THE EUROPEAN TRAINING FOUNDATION. Quality Assurance inHigher Education: A legislative Review and Needs Analysisof Developments in Central and Eastern Europe. Turin: TheEuropean Training Foundation, 1998.

*THE EUROPEAN TRAINING FOUNDATION. Quality Assurance in HigherEducation: Final Report and Project Recommendations. Turin:The European Training Foundation, 1998.

*THE EUROPEAN TRAINING FOUNDATION. The European Dimensionof Institutional Quality Management: The EuropeanUniversity. A Handbook on Institutional Approaches toStrategic Management, Quality Management, EuropeanPolicy, and Academic Recognition. Turin: The EuropeanTraining Foundation, 2000.

FRANKE, S. "From Audit to Assessment: A National Perspectiveon an International Issue", Quality in Higher Education 8 1(2002). [This article describes recent developments in thenational quality assurance system in Sweden with theintroduction of quality assessments of all higher educationthat leads to general and professional degrees.]

FREED, J. E., and KLUGMAN, M. Total Quality Management onCampus: Implementation, Experiences, and Observation.Phoenix: The Oryx Press, 1997. [This book analyses "TotalQuality Management" (TQM) in action at ten colleges anduniversities across the United States, exploring the need to


improve quality, lessons learned, and the experiences ofeach institution implementing the various quality principlesassociated with the TQM philosophy.]

GEDDIS, E. "A Perspective on Tensions between ExternalQuality Assurance and Institutional Quality Development: ACase Study", Quality in Higher Education 8 1 (2002).

HAMALAINEN, K., et al. Quality Assurance in the Nordic HigherEducation Accreditation-like Practices. Helsinki: ENQA,2001.

HARVEY, L., and GREEN, D. Defining Quality: Assessment andEvaluation in Higher Education 18 1 (1993).

KELLS, H. R. Self Regulation in Higher Education: aMultinational Perspective on Collaborative Systems of QualityAssurance and Control. London: Jessica Kingsley,1992.[Intended as a reference source and guide for policymakers, this book describes several major forms ofregulatory systems for higher education. Existing modelsand systems are examined in national contexts. Usefulappendices include excerpts from various national andregional systems.]

JONES, D. P. Different Perspectives on Information aboutEducational Quality: Implications for the Role ofAccreditation. Washington D.C.: Council for HigherEducation Accreditation, 2002.

LISTON, C. Managing Quality and Standards. Buckingham:Open University Press, 1999. [Quality management, theapplication of standards, and client service areas are asvitally important to higher education as they are in anyother enterprise or public service activity. This text providesa practical, common sense approach to using proceduresand information to demonstrate improved performance andto account for outcomes. The text confronts both the qualityjargon and the cynicism of many academics about the"quality police". It is full of practical examples andguidelines for action and is useful for managers inuniversities and colleges worldwide.]

RESOURCES 215

LOIVIAS, L. "Does the Development of Mass EducationNecessarily Mean the End of Quality? ", Quality in HigherEducation 8 1 (2002).

MIDDLEHURST, R Quality Assurance Implications of New Forms ofHigher Education. Helsinki: ENQA, 2001 <http: / /www.enqa.net>.

SEDDON, J. The Case Against ISO 9000. 2nd edition. Dublin:Oak Tree Press, 2000.

SMEBY, J. C., and STENSAKER, B. "National Quality AssessmentSystems in the Nordic Countries: Developing a BalanceBetween External and Internal Needs", Higher EducationPolicy 12 (1999): 3-14.

SMITH, D. "How Will I Know If There Is Quality?" Report onQuality Indicators and Quality Enhancement in Universities.Toronto: Council of Ontario Universities, 2000. [This reportproposes that quality indicators should be considered asonly a component of a broader, more powerful approach toquality enhancement. They reveal certain symptoms ofquality, but they do not show the causes of quality nor theconjunction of resources and incentives that producequality. This report concludes with an emphasis onfostering a competitive and collaborative environment foruniversities, within a university sector that is fundedreasonably, in line with competing university sectors.]

STRATI-1ERN, M. Audit Cultures: Anthropological Studies inAccountability, Ethics, and the Academy. London: Routledge,2000. [In this book, twelve social anthropologists fromacross Europe and the Commonwealth chart thecontroversial phenomenon of "accountability".]

TOMUSK, V. "When East Meets West: Decontextualizing theQuality of East European Higher Education", Quality inHigher Education 6 3 (2000): 175-185.

VAN DAMME, D. 'Trends and Models in International QualityAssurance and Accreditation in Higher Education inRelation to Trade in Education Services", Paper presentedat the OECD/US Forum on Trade in Educational Services,23-24 May 2002.

2 n


VROEIJENSTEIN, A. I. Improvement and Accountability:Navigating between Scylla and Charybdis. London: JessicaKingsley, 1994. [This study provides a guide for the designof systems of external quality assessment, drawing upon amodel developed in the Netherlands but recognizing that noone model will meet the needs of different systems of highereducation.]

WESTERHEIJDEN, D. F. "Ex oriente lux?' National and MultipleAccreditation in Europe after the Fall of the Wall and afterBologna", Quality in Higher Education 7 1 (2001): 65-75.

2.3.7. Open and Distance LearningHARRY, K. Higher Education Through Open and Distance

Learning, World Review of Distance Education and OpenLearning Series. London: Farmer Press, 1999. [Open anddistance learning has grown substantially in recent yearsacross all levels of education, disciplines and countries.This book looks at the state of the art in open and distancelearning and presents an educational culture in transition.The edited collection contains authoritative analyses of keyissues together with current accounts of practice in eachregion of the world.]

HOPE, A. "Quality Assurance", in, G. M. FARRELL, ed. TheChanging Face. of Virtual Education: The Commonwealth ofLearning. Vancouver, British Columbia, 2001 <http:/ /www. col. org/virtualed / >.

IHEP. Quality on the Line: Benchmarks for Success in Internet-Based Distance Education. Washington D.C.: Institute forHigher Education Policy, 2000 <http://wwvv.ihep.com/quality.pdf>.

LOANS, S. Distance Education and Accreditation. WashingtonD.0 . : ERIC, 2001 <http: / /wwweriche .org/ digests /2001-08. htm>.

OECD. E-learning: The Partnership Challenge. Pans:Organization for Economic Co-operation and Development,2001.

21q

RESOURCES 217

3. SOME USEFUL WEBSITES

Central and Eastern Europe Network:<http://www.ceenetwork.hu> [individual agency links(access from <http://www.ceenetwork.hu>) [information onCEE agencies also at <http://wwvv.inqaahe.nl>]European Network for Quality Assurance (ENQA):<http://www.enqa.net> [This site is a comprehensive onehaving links to all member organizations. All ENQApublications, including the newsletter, are available on thesite and can be downloaded free of charge. Also availableare details of forthcoming workshops and seminars that areopen to non-members as well as to members.]International Network of Quality Assurance Agencies ofHigher Education (INQAAHE): <http://www.inqaahe.nl>[This site has links to all member organizations. Thenetwork newsletter can be downloaded from the site free ofcharge.]Council for Higher Education Accreditation USA (CHEA):<http: / /www.chea.org> [This site has links to all CHEArecognized accreditation bodies in the United States as wellas a useful international database and glossary.]Akkreditierungsrat (Germany):<http://www.akkreditierungsrat.de> [This agency isapproximately the German equivalent of CHEA. It onlyaccredits programmes itself on special request otherwise itsmain role is to accredit regional or subject specificaccreditation agencies. It has links to other Germanaccreditation agencies.]

3.1. INTERGOVERNMENTAL ORGANIZATIONS

Council of Europe: <http://www.coe.int>European Commission:<http://www.europa.eu.int/comm./education/higher.hthal>OECD: <http: / /www.oecd.org>UNESCO: <http://www.unesco.org> and European Centrefor Higher Education: <http: / /www.cepes.ro>


World Conference on Higher Education:<http: / /www.unesco. org/education/educprog/wche. index.html>Global Forum for Quality Assurance, Accreditation, and theRecognition of Qualifications:<http: /www.unesco org/education/studyingabroad>World Bank: <http://vvww.worldbank.org>World Trade Organisation: <http://www.wto.org>

3.2. NON-GOVERNMENTAL ORGANIZATIONS

American Association for Higher Education (AAHE):<http: / /www.aahe.org>American Council for Education (ACE):<http: / /www. ace. org>Association of Commonwealth Universities (ACU):<http://www.acu.ac.uk>Center for Higher Education Research and InfotmationCHERI: <http://www.open.ac.uk/cheri> [See informationand new project "Universities and Their Role in theTransformation of Society ".]European Universities Association (EUA):<http: / /www.unige.ch/eua>International Association of Universities (IAU):<http: / /www.unesco.org/iau>National Union of Students in Europe (ESIB):<http://www.esib.org/>Universitas21: <http: / /www. universitas21.corn>Coimbra group: <http://www.coimbra-group.pt>

3.3. SUBJECT ACCREDITATION /RECOGNITION AGENCIESOPERATING ON AN INTERNATIONAL BASIS

3.3.1. Business and ManagementAssociation of MBAs (AMBA): <http://www.mba.org.uk>[Accredits MBA programmes in the UK and abroad]

21?

RESOURCES 219

American Assembly of Collegiate Schools of Business(AACSBInternationall: <http://www.aacsb.edu> [AccreditsBusiness, Management, and Accounting Schools in the USAand in eleven other countries]European Foundation for Management Development(EFMD): <http://www.efmd.be> [Manages the EQUISProgramme and accredits business schools and beyond.]Foundation for International Business AdministrationAccreditation (FIBAA): <http://www.mba-info,de> [Thisorganization is accredited by the Akkreditierunsgrat(Germany) and accredits programmes in Business andManagement in Austria, Germany, and Switzerland.]

3.3.2. EngineeringEuropean Federation of National Engineering Associations(FEANI): <http: / /www.feani.org>The Washington Accord: <http://www.washingtonaccord.org/>

3.4. SOME OTHER RESOURCES ON ASPECTS OF QUALITYAND/OR EDUCATION

Bized <http://catalogue.bized.ac.uk>European Foundation for Quality Management (EFQM)<http: / /www.efqm.org/new_website>Information on Malcolm Baldridge, TQM and EFQM:<http: / /www. qpronline. com/baldridge /index. html>International Organization for Standardization (ISO)<http://www.iso.ch/en>National Institute of Standards and Technology (USA)<http: / /www. quality. nist. gov />Observatory on Borderless Higher Education: <http://www.obhe.ac.uk>. [An international strategic informationservice tracking developments in on-line learning,transnational provision, and new providers (e.g., for-profituniversities). The service is designed to appeal to a wide rangeof organizations and individuals concerned with highereducation policy.]

The Authors

CAMPBELL, CarolynAssistant Director, Quality Assurance Agency for HigherEducation (QAA)Address: Southgate House, Southgate Street, Gloucester

GL1lUB, United KingdomTel: +44-1452557000Fax: +44-7801387053e-mail: [email protected]

ROZSNYAI, Christina.Programme Officer for Foreign Affairs, Hungarian AccreditationCommitteeAddress: Ajtosi D. sor 19-21, H-1146 Budapest, Hungary

Tel: +36-13440314Fax: +36-134403133e-mail: [email protected]

221

214

UNESCO-CEPES PUBLICATIONSThe UNESCO European Centre for Higher Education produces five

series of publications:the quarterly review, Higher Education in Europe, published in

three language versions: English, French, and Russian;the Studies on Higher Education, which present relatively

comprehensive reports on and analyses of major issues in highereducation;

the Papers on Higher Education, which present shorter studiesand occasional papers;

the Monographs on Higher Education, which present studies onnational systems of higher education according to a common outline;

the Studies on Science and Culture, that publish the researchfindings undertaken foremost by the UNESCO Chairs, which arecollaborating with UNESCO-CEPES, in subject areas other than highereducation.

HOW TO ORDERSubscriptions to the English version of Higher Education in Europe

must be placed directly with CARFAX: Carfax Publishing, Taylor &

Francis Ltd.; Customer Services Department; Ranking Road;

Basingstoke, Hants RG24 8PR, United Kingdom; E-mail:[email protected]. The French and Russian versions can be obtainedfree of charge through the UNESCO-CEPES Web page:

http://www.cepes.roFor the volumes in the other series, please check off the titles, that you

would like to purchase bearing in mind that volumes in the Studies...,the Monographs..., and the Studies on Science and Culture series cost$20.00 (USD) or 22,00 each. Each volume in the Papers... series costs$15.00 (USD) or 16.50.

Please fill in the following blanks:Your Name:Institution:Address:

Telephone: Telefax:

E-Mail:Please mail this information to:

The Senior EditorUNESCO-CEPES

39 Strada *tirbei-VodaRO-70732 Bucharest, Romania

We shall bill you for the material that you have ordered and includeinstructions as to payment.

Date:

Studies on Higher Education* Gains and Losses: Women and

Transition in Eastern and CentralEurope (English, 1994, 115 p.)

* Academic Freedom and UniversityAutonomy. Contributions to theInternational Conference, 5-7 May 1992,Sinaia (English, 1993, 309 p.)

* Higher Education Reform in Romania: AStudy (English, 1992, 143 p.)

* The Doctorate in the Europe Region(English, 1994, 225 p.)

* Standards and Diversity in ArchitecturalEducation (English, 1996, 353 p.)

* Ten Years After and Looking Ahead: AReview of the Transformations of HigherEducation in Central and EasternEurope (English, 2000, 410 p.)

* Transnational Education and the NewEconomy: Delivery and Quality (English,2001, 172 p.)

* Good Practice in Promoting GenderEquality in Higher Education inCentral and Eastern Europe (English.2001, 160 p.)

Papers on Higher Education* Assisting Higher Education in Central

and Eastern Europe: Activities ofInternational Organizations - A Survey(English, 1992, 31 p.)

* CEPES 20 Years of Service (English,1992, 40 p.)

* Academic Freedom and UniversityAutonomy: Proceedings of theInternational Conference, 5-7 May 1992,Sinaia (Eriglish, 1992, 52 p.)

* University Profiling and Identity (English,1994, 21 p.)

* Academic Freedom and UniversityAutonomy: Two Perspectives (English,1995, 85 p.)

* La Formation pratique: principes etquestionnement (French, 1995, 52 p.)

* Report on Higher Education in Bosniaand Herzegovina: HistoricalDevelopment, Present State, and NeedsAssessment (English, 1996, 127 p.)

* Mutual Recognition of Qualifications: TheRussian Federation and Other EuropeanCountries (English, 1997, 124 p.)

* The Europeanization of EuropeanUniversities: A View from the East(English, 1997, 140 p.)

216

* A European Agenda for Change for .

Higher Education in the XXIst Century(Changer l'enseignement superieur enEurope, un programme pour le XXIesiecle) (English and French, 1997, 166 p.)

* A European Agenda for Change for HigherEducation in the XXlst Century: TwentyCase Studies (English, 1998, 390 p.)

* Internationalization of Higher Education:An Institutional Perspective (English,2000, 97 p.)

* Quality Assurance in Higher Educationin the Russian Federation (English,2001, 126 p.)

* From Words to Action: Approach to aProgramme (English, 2002, 240 p.)

* Policy-Making, Strategic Planning. andManagement of Higher Education(English, 2002, 194 p.)

* Financial Management and InstitutionalRelationships with Civil Society (English,2002, 234 p.)

* Quality Assurance and the Developmentof Course Programmes (English, 2002,222 p.)

Monographs on Higher Education* Albania (1986), Austria (1987), Belarus

(1983, 1999), Bulgaria (1983, 2002),Estonia (1997), Finland (1988),Germany (1999), The GermanDemocratic Republic (1983), Hungary(1985, 1997), The Netherlands (1985,1988, 1989), Norway (1983, 1991),Poland (1987), Switzerland (1984),Turkey (1990), The Ukrainian SSR(1985), The USSR (1990), The UnitedKingdom (1996), The United States(1982).

Studies on Science and Culture* Bioetica in Romania: teme §i dileme

(Romanian, 1999, 126 p.)* Politics and Culture in Southeastern

Europe (English, 2001, 335 p.)* Sustainable Development: Theory and

Practice Regarding the Transition ofSocio-Economic Systems towardsSustainability (English, 2001, 306 p.)

* South East Europe - The AmbiguousDefinitions of a Space/L'Europe du Sud-Est - les definitions ambigues d'un espace(English and French, 2002, 214 p.)

October 2002

Printed by "TREY' Publishing & Printing CompanyBucharest, Romania

Tiparul executat sub comanda nr. 20 409

Compania Nationala a ImprimeriilorCORESI" S.A. Bucuresti

ROMANIA

Sistemul calitatii certificat SR EN ISO 9001

217

UNESCO-CEPES (European Centre for HigherEducation/Centre Europeen pour l'EnseignementSuperieur), a decentralized office of the UNESCOSecretariat, was established in September 1972 with aview to promoting co-operation in higher educationamong Member States of the Europe Region (thecountries of Europe, North America, and Israel). Inaddition to this major mission, UNESCO-CEPES alsoserves as a field office representing UNESCO inRomania.

Although the activities of UNESCO-CEPES are focused foremost onhigher education in the Europe Region, the Centre also maintainscontacts with relevant organizations and institutions in other regions,in conformity with the universal vocation of UNESCO.

Through its pan-European mission and specific competence andexperience in Central, South- East, and Eastern Europe, developedover thirty years of presence in the region, UNESCO-CEPES is, in itsown way, a unique institution that deals with higher education in theEurope Region by providing bridges for active international co-operation.

UNESCO-CEPES is focusing its activities along the following fourmain strands:

Policy and Reform of Higher Education;Inter-university Co-operation and Academic Mobility;Publications, Studies, and Information Services;Status of Teachers and Teaching/ Learning in the InformationSociety.

It also:provides consulting services;participates in the activities of other governmental and non-governmental organizations;serves as a link between UNESCO Headquarters and Romania inrelation to the activities of the Organization in Romania.

In order to respond to the need for topical actions in support ofthe processes of reform and development of higher education inCentral and Eastern Europe, including South-East Europe, UNESCO-CEPES has focused its current activities on the making andimplementation of higher education policy, legislative reforms ineducation, academic quality assurance and accreditation, and therecognition of academic and professional qualifications.

It is also concerned with new approaches to governance andinstitutional management, university autonomy and academicfreedom, the status of teachers and their training, university-industryrelations, the use of new information technologies, including theInternet/virtual university, and transnational education.

BEST COPY AVAILAF3L

U.S. Department of EducationOffice of Educational Research and Improvement (OERI)

National Library of Education (NLE)

Educational Resources Information Center (ERIC)

NOTICE

Reproduction Basis

EPICEdnaltmalliesatantigonmficaCerau

This document is covered by a signed "Reproduction Release (Blanket)"form (on file within the ERIC system), encompassing all or classes ofdocuments from its source organization and, therefore, does not require a"Specific Document" Release form.

This document is Federally-funded, or carries its own permission toreproduce, or is otherwise in the public domain and, therefore, may bereproduced by ERIC without a signed Reproduction Release form (either"Specific Document" or "Blanket").

EFF-089 (1/2003)