networking visit of see qa experts to the agency for quality assurance in higher education (aracis)...

Networking visit of SEE QA experts to the Agency for Quality Assurance in Higher

Education (ARACIS) and the Bologna Secretariat

BUCHAREST, ROMANIA

19 – 20 OCTOBER 2011

Romanian Agency for Quality Assurance in Higher Education

(ARACIS)

QUALITY ASSURANCE IN ROMANIAN EDUCATION

Quality assurance in Romanian Education

Provisions of government emergency ordinance (GOE) regarding quality assurance in education, no. 75/2005, approved by the law no. 87/2006, modified by GOE 75/2011 - main operational concepts for quality in education :

Quality assurance (internal)Evaluation and control (internal &

external) Improvement (internal)

Romanian Agency for Quality Assurance in Higher Education (ARACIS)

1993: Law no. 88/1993 on Evaluation and Accreditation of Higher Education Institutions and Recognition of Diplomas

National Council for Academic Evaluation and Accreditation (CNEAA - NCAAA): 1993 - 2006 evaluation and accreditation of institutions and study programs

The Romanian Agency for Quality Assurance in Higher Education (ARACIS) officially established by Government Emergency Ordinance No. 75 for Quality Assurance in Education of July 12th, 2005, modified and approved by Law 87/2006, Published in the Romanian Official Journal, Part I No. 334 of April 13th, 2006 – modified by government emergency ordinance 75/2011

ARACIS: Mission and objectives stated by Law ; also, ARACIS assumes and carries on the official rights and obligations of the former Romanian National Council for Academic Assessment and Accreditation (CNEAA) as well as its technical staff, archives and goods


Objectives - Quality assurance

- develop and regularly review the national reference standards and the performance indicators, based on best practices, aiming to evaluate quality of higher education- cooperate with other national institutions involved in the development and promotion of policies and action strategies, aiming to increase the quality of education in Romania- organize yearly consultations with the higher education institutions to put forth the priorities in the field of quality assurance- develop its own procedures for the external quality evaluation and make them available to the public- perform external quality evaluation of programs and the providing institutions, national or foreign education establishments, as well as for the inter - institutional evaluation of similar programs, on contractual basis


Objectives - Quality assurance:

- perform the quality evaluation in cases of certain programs and higher education establishments, upon request by the Ministry of Education, Research, Youth and Sports- publish the results of the external assessments- publish manuals, guidelines, papers summarizing the best practices in the quality assessment and assurance- develop regularly, every three years, system analyses on the quality of the higher education in Romania- cooperate with other similar foreign agencies in order to develop and implement effective measures to improve the quality of the higher education programs- develop the Code of Professional Ethics Code applicable to the ARACIS experts- publish yearly report on its own activity - develop reports aiming to assess its own operational quality, every three years, with a view to prepare itself for the external assessment by similar foreign agencies


AccreditationTwo successive stages:

a) provisional authorizing, as first step - gives the HE provider the right to initiate and carry out the educational process and to organize, as the case may be, the admittance examinations;b) accreditation - grants the institution the right, along with the others rights provided under let. a), to issue diplomas, Ministry of Education, Research, Youth and Sports certificates and other official documents papers recognized by the Ministry of Education, Research, Youth and Sports and to organize, as the case may be, graduation examinations at Bachelor, Master and Doctorate level.

Other tasks related to accreditation: a) develop on a regular basis the Methodology and the accreditation standards for different types of programs and HE providers, to be submitted to the Ministry of Education, Research, Youth and Sports and approved by the Government; b) to evaluate quality of higher education, based on the standards and Methodology, upon request or of its own motion, and to propose the authorization, respectively, the accreditation of the higher education providers and of their study domains and programs. Based on the ARACIS reports, the Ministry of Education, Research, Youth and Sports initiates the necessary legislative provisions for the establishment of higher education structures.


Status and Resources- autonomous institution, of national interest, a legal entity with its own income and expenditure budget- lead by a board of 17 members (15 academics and two students), with high academic and moral status, holding teaching positions in the higher education system. Every four years, the Council of ARACIS shall renew its composition by opening 10 positions to a new selection procedure. The members of the Council can sit for only two terms, 4 years each. Rectors and persons holding official positions with the Presidency, Government or Parliament cannot become or remain members of the Council of ARACIS.


Resources (human)- develops its own database of external

evaluators, based on a transparent methodology (more than 1000 Romanian academic experts who passed on-line examination, students, foreign evaluators and representatives of the labor market)

- consistency of evaluation: the Council set up 15 permanent specialty (subject) Commissions

- permanent staff (35 – increase forseen)


Resources (facilities) premises (rented)

furniture (purchased)

equipment (purchased)

one car


Independence

The Agency is independent of any political or other external influence

From the financial point of view, as stated by the Government Ordinance No 102/2006, ARACIS is funded as follows:

a) fees collected for covering the costs of external evaluation for authorization/ accreditation/periodic evaluation of the higher education establishments and their study domains or programs; b) contracts for external evaluation of quality in higher education, concluded with third parties, as the Ministry of Education, Research, Youth and Sports; c) grants obtained through participation in international programs, donations, sponsorship activities, as well as other sources duly established.


Mission statementThe mission statement of the Agency is specified in the Law 87/2006 In accordance with its mission and the legal provisions, ARACIS elaborated a Methodology for external evaluation, standards, standards of reference and list of performance indicators, which was approved by Government Decision No. 1418/2006 in which the mission specified is fully stated.

According to this “Methodology”, quality assurance in education makes reference to the following fields and criteria:

A. Institutional capacityB. Educational effectivenessC. Quality management


The definition and operation of the procedures of the Agency are stated in the “Methodology” are detailed in the “Guide of the Activities of Quality Evaluation of University Study Programs and Higher Education Institutions” developed and approved by the Council of the agency:

Part I -Authorization/accreditation of licence (bachelor) and master programs (Bologna “First” and “Second” cycles);

Part II - External quality evaluation for accreditation at institutional level;

Part III - External quality evaluation at institutional level at accredited institutions;

Part IV – External evaluation of Departments for Teacher formation and training in HE institutions;

Part V – External evaluation of departments and distance learning programs offered by universities.


A NEW METHODOLOGY!(PROBABLY THIS MONTH)

Because

QA is a continuous process

A new legal context

ARACIS already realized a system analyze for Romanian HE system


A new legal context The law on quality assurance, law no. 87/2006,

remains into force. The law establishes the general criteria for quality assurance: institutional capacity, educational effectiveness and quality management.

A new law on education has been passed; the law foresees a differentiation process among universities – there will be three non-hierarchical categories: education centred, education and research and respectively research intensive universities.


A new legal context

The law also foresees the multiplication of the evaluation processes and focuses:

Evaluation for the authorization/accreditation and periodical evaluation (every five years) of institutions, licence programmes, master domains and doctoral schools

Evaluation for the classification of universities and ranking of study programmes

The law also foresees a new financing methodology – the allocation of funds will be more competitive on the basis of the ranking of the study programmes.


ARACIS finished the following evaluation activities:

- external evaluation of quality for 80 public and private accredited institutions

- external evaluation of a number of 2220 study programs, at the request of the interested institutions, in view of accreditation or periodic quality evaluation;

- external evaluation of a number of 2950 master programs, at the request of the interested institutions, in view of accreditation.


Strong points of the Methodology

Quality Barometers : high degree of consensus among the major stakeholders - academic staff and students as regards the appropriateness of these procedures

The Methodology has been recognized at European level (ARACIS - full member of ENQA in June, 2009 and listed in EQAR in September, 2009)

A large number of universities and study programs have been evaluated


Strong points of the Methodology

ACADEMIS project: opportunity to broaden the area of institutional evaluation by

covering a large part of the higher education landscape

systematic data gathering on main stakeholders’ opinions and perceptions on quality; objective data on universities (benchmarking exercise); secondary analysis and reflection on the current methodology.

active participation of students in external institutional evaluation

framework for wide consultation with experts in universities in order to improve the existing methodology on an evidence and data based approach.


Weak points identified in the HE system

Low level of differentiation among universities with regard to mission, strategies and quality assurance procedures

High degree of homogeneity and institutional isomorphism in organisation

but

Objective reason: specialized universities (i.e. medical, military, technical etc.) are more homogeneous than comprehensive universities



High levels of formalism and ritualism in the internal quality assurance practices in institutions; the focus has been on drafting dossiers for accreditation and/or periodic evaluation

SERs of higher education institutions: are too "laudatory," instead of showing an understanding the concepts of QA and the activities themselves.



The internal quality assurance is mostly a technical bureaucratic activity - not embedded in the day-to-day routines and practices of the universities or study programmes

The involvement of students - achieved in the external evaluation but to a much lesser degree in the internal quality assurance



The quality assurance practices have not been internalized - largely decoupled from the teaching/learning and research current activities in the universities

Internal quality cultures have not yet emerged, but rather countre-cultures blaming the excessive bureaucratization of academic activities.

Many higher education institutions do not demonstrate the effectiveness of their own internal quality assurance processes


New approach to policy making The ACADEMIS project has facilitated the collection

and analysis of relevant objective and subjective data at systemic level. The policy making process now benefits from a broader range of experiences and evidence as regards quality assurance in Romanian higher education.

The elaboration of the new methodology is an evidence-based type of policy making involving numerous actors and stakeholders and will be the result of systematic reflection on the current methodology’s achievements and weaknesses.


Setting new QA principles Shift of focus: from quality control or monitoring

(external evaluation) to quality enhancement.

Balanced focus on the benchmarking approach and Compulsory normative minimal requirements (mostly for accreditation) and fitness-for-purpose approach (for periodical evaluation)

Benchmarking approach – meeting of threshold indicators’ level for accreditation purposes – inter-institutional comparisons – benchmarking will be a dynamic process based on the current performance of Romanian unis, but also observing European trends and benchmarks.


Setting new QA principles

Fitness for purpose – mainly a quality enhancement tool; universities and study programmes will be encouraged to gather systematic data and to set internal reference points – intra-institutional or intra-programme comparisons. Universities will be encouraged to reflect on the strenghts and weaknesses and to take remedial decisions on the basis of evidence.


Setting new QA principles Accepted definitions - ENQA:

Fitness for purpose equates quality with the fulfilment of a specification or stated outcomes:

Institutional choice between a range of indicators according the university mission and objectives

Fitness of purpose evaluates whether the quality-related intentions of an organisation are adequate.

Increased weight and importance to the internal mechanisms for quality assurance

Implementation of yearly internal quality assurance reports of the study programmes/institutions


Setting new QA principles

Balanced set of indicators focusing mostly on processes (internal management of quality at all levels, study programmes, faculties, institution) and outputs (learning outcomes, scientific production), but also inputs (staffing, teaching/learning environment)

Periodical evaluation will be mostly focused on the audit of the internal quality system (process) and quality enhancement. The institutions will be thus encouraged to assume full responsibility for assuring quality and also to involve stakeholders so as to assure the validity of their study programmes.


NEW METHODOLOGY FOR QA

The European Standards and Guidelines

Multi-stage procedure: internal evaluation followed by external evaluation and decision making – it has already been successfully established and will be maintained.

Important focus on internal evaluation

Bottom-up approach in setting indicators and procedures at institutional level.

Quality indicators in-built form within the programme and more focused on student learning experiences.


NEW METHODOLOGY FOR QA An inclusive and evidence based process of

defining the list and threshold level of indicators for accreditation of different types of universities and study programmes.

Quality is a contextual and cultural concept: there can be various ‘qualities’

Diversity will be assumed and encouraged: not just one path to quality assurance - institution specific paths encouraged

Involvement of stakeholders will be stimulated

networking visit of see qa experts to the agency for quality assurance in higher education (aracis)...

Documents

quality of education

quality of higher education

field of quality assurance

quality assessment

higher education programs

higher education aracis1993

ministry of education

goods romanian agency