peer learning event on assessment and demonstration of ... with case studies... · universities of...
TRANSCRIPT
This project has been funded with support from the European Commission. This publication (communication) reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein
Peer Learning Event
on Assessment and
Demonstration of
Achieved Learning Outcomes
The Hague 29-30 October 2015 26 October 2015
NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 2
Table of Contents
1 Preface 3
2 Programma (v.1.0) 4
3 Abstracts 8
4 Biodata of the presenters 14
5 List of participants 16
The organizing team wishes you a pleasant stay in The Hague and a fruitful conference.
NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 3
1 PREFACE
The Accreditation Organisation of the Netherlands and Flanders (NVAO), together with the
Dutch Ministry of Education, Culture and Science (OCW), is organizing the Peer Learning
Event on the Assessment and Demonstration of the Achievement of Learning Outcomes on
Thursday 29 - Friday 30 October 2015 from at 9.30 am in The Hague, The Netherlands.
The conference is part of the EU-financed project Facilitating Bologna Tools (FaBoTo) of the
Dutch Ministry of Education (OCW), the National Agency Erasmus+ and NVAO.
What is it about?
Using achieved learning outcomes in quality assurance and accreditation and assessing
and demonstrating these has been identified as a development that needs further
enhancement and training. The peer learning event provides a platform for sharing and
discussing good practices and eventually formulate a set of practical guidelines, which will
later on be made available for a wide audience.
Themes and questions
– Do you know of any good practices or case studies from HEI when it comes to describing
intended learning outcomes in a clear and measurable way, both at course unit and at
programme level so that learning outcomes are understandable and verifiable for all
target groups concerned?
– Do you know of any HEI serving as examples of good practice when it comes to
demonstrating convincingly the achievement of the intended learning outcomes to the
stakeholders?
– Do you know of any good practices or case studies from QA when it comes to evaluating
intended learning outcomes and the demonstration of their achievement?
– Are there interesting case studies on the involvement of internal/external stakeholders in
the formulation of intended learning outcomes or on how HEI organize this involvement?
– Do you know of any training programmes HEI have access to in order to facilitate the
proper formulation and assessment of intended learning outcomes?
– Do you know of any convincing case studies of methods programme committees apply to
monitor the appropriate link between learning outcomes, curriculum design and learning
activities, and assessment methods and criteria?
– Are there interesting case studies of properly defined expectations that QA have with
respect to the use of intended learning outcomes and their assessment and
demonstration of achievement?
– Do you know of any convincing case studies of methods programme committees apply to
monitor the appropriate link between learning outcomes, curriculum design and learning
activities, and assessment methods and criteria?
– Are there interesting case studies of properly defined expectations that QA have with
respect to the use of intended learning outcomes and their assessment and
demonstration of achievement?
Participants
The Peer Learning Event has participants mainly from Higher Education Institutions (HEI)
and Quality Assurance Agencies (QA). The event is set up in response to interest for this
theme from a number of countries, who have set up delegations. Besides these, a large
number of independent participants have registered.
NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 4
2 PROGRAMME (v.1.0)
Times and titles of presentations are provisional and can change.
Thursday 29 October 2015
9.00-9.30 Reception of guests at the venue: the Ministry of Education,
Culture and Science (OCW)
9.30-9.45 1. Welcome address by Lucien Bollaert, member of the board of
the NVAO
9.45-10.30 2. General introduction to the topic of the event
Fiona Crozier, Head of International, QAA, UK
Dr. Kath Hodgson MBE, Director of Learning and Teaching Support,
Leeds University, UK
3. Two Case Studies from HEI’s related to the achievement of intended learning
outcomes
2x20 min presentation and joint discussion of 20 minutes chaired by Fiona Crozier.
10.30-10.50 3.A Dr. Daan Andriessen (HU University of Applied Sciences
Utrecht)
Title: Developing a Protocol for the Assessment of Achieved
Learning Outcomes for the Netherlands Association of
Universities of Applied Sciences [Vereniging Hogescholen]
10.50-11.10 3.B Andy Gibbs BSC (Hons), MSC, PGCE, RN
Senior Lecturer International, School of Nursing and Community
Health, School of Health and Life Sciences, Glasgow Caledonian
University
Title: Differing Approaches to Programme Learning Outcomes
11.10-11.30 BREAK
11.30-11.50 3.C Questions and discussion on the two case studies
Do these produce a set of general principles or good practices?
Chair: Fiona Crozier
NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 5
4. Two Case Studies from QA’s; The use of achieved learning outcomes in external
quality assurance
2x20 min presentation and joint discussion of 20 minutes
Chaired by Kath Hodgson
11.50-12.10 4.A Karin Järplid Linde, Head of Department of Quality Assurance
– Swedish Higher Education Authority (UKÄ)
Title: External quality assurance of higher education in Sweden
12.10-12.30 4.B Dietlinde Kastelliz - Head of Department Audit & Consulting -
Agency for Quality Assurance and Accreditation Austria (AQ
Austria)
Title: Learning Outcomes as Subject Matter of External Quality
Assurance
12.30-12.50 4.C Questions and discussion on the two case studies
Do these produce a set of general principles or good practices?
Chair: Kath Hodgson
12.50-14.00 LUNCH
14.00-16.00
5. Breakouts in two rounds of three workshops: Presentations of
good practices and discussion (follow the color codes)
- Each workshop starts with short presentations of good practices by
participants (10-15 minutes) followed by discussion or assignments
Desired outcomes:
- Could the presented practices be used by participants in their own
context?
- Are there useful alternatives not discussed before?
- Which elements constitute a good practice approach?
14.00-15.00 5.A Round 1 Workshops
Workshop 1 (Room 15.003, 15th
Floor)
Daan Andriessen (HU University of Applied Sciences Utrecht)
- Pilot Guidelines from Dutch Universities of Applied Sciences
Workshop 2 (Room Barcelona, 16th
Floor)
Johan Alvfors (Swedish National Union of Students)
- TBA
Oliver Vettori (Vienna University of Economics and Business)
- Meaningful measurements? - Between the poles of rigor and
relevance
Workshop 3 (Room Lissabon, 16th
Floor)
Petra Bulthuis & Anke Thijsen (Saxion University of Applied Sciences)
- The AuCom Model - Guidelines for researching, defining and
demonstrating achieved learning outcomes
NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 6
15.00-15.20 BREAK
15.20-16.20
5.B Round 2 workshops
Workshop 1 (Room Barcelona, 16th
Floor)
Paul Garré (Odisee)
- EQF-levels as a corner stone for institutional quality assurance
approaches
Workshop 2 (Room Lissabon, 16th
Floor)
Luc Van de Poele (Ghent University)
- Systematic Data Gathering of Intended and Experienced Learning
Goals
Workshop 3 (Room 15.003, 15th
Floor)
Robert Coelen (Stenden University of Applied Sciences)
- Workplace Acceptance of Program Learning Outcomes
Kees Kouwenaar (VU Free University Amsterdam)
- Required learning achievements at the entrance of the master’s
programme: Mastermind Europe (www.mastermindeurope.eu )
16.20-17.00 6. Synthesis from the six groups, discussion chaired by Axel
Aerden, NVAO
17.00-18.00
Free time
18.00-22.00 DINNER IN THE HAGUE, at Madurodam
Busses will be waiting at Central Station (instructions will be
announced)
NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 7
Friday 30th October
9.00-9.30 1. Opening by Mark Frederiks, NVAO International Affairs
Introduction to a possible outline of guidelines
9.30-10.00 2. Plenary discussion on an outline of guidelines
10.00-11.00 3. Breakout in two workgroups with a mix of HEI and QA
representatives, chaired by Els van der Werf (Bologna expert, the
Netherlands) and Oliver Vettori (Vienna University of Economics
and Business)
- how can the presented and identified good practices be included in the
guidelines?
- what elements can be used to formulate guidelines?
- how should they be described as guidelines?
11.00-11.20 BREAK
11.20-12.00
4. Plenary discussion and synthesis from workgroups on main
conclusions regarding guidelines, chaired by Axel Aerden, NVAO
12.00- 12.30 5. Concluding talk by Lucien Bollaert, NVAO Board member
Summing up the results
12.30- 13.30 End of the programme and LUNCH
NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 8
3 ABSTRACTS
1. General introduction to Achieved Learning Outcomes by Fiona Crozier and Kath
Hodgson
Our presentation aims to set out two sides of the „learning outcomes coin‟; i.e. transparency
or straitjacket (to put it very simply). We ask whether quality assurance agencies and
academics represent those two sides of the coin in their approach to learning outcomes and
whether, in fact, both sides of the coin impact positively on all stakeholders. What about the
impact on the views of students, their parents, employers and professional, regulatory
bodies? Do learning outcomes and their assessment as currently used improve the
students‟ overall experience?
We will look at the importance and impact of learning outcomes from these perspectives –
we have a lot of questions…and not many answers! We hope that our questions will
encourage debate and discussion and we look forward to hearing what the audience has to
say.
2. Case studies from Higher Education Institutions
2.1 Daan Andriessen, HU University of Applied Sciences Utrecht
Developing a Protocol for the Assessment of Achieved Learning Outcomes
In response to a crisis of public confidence in Dutch Professional Education a group of
experts developed a Protocol for systematic quality assurance and external validation of
final examination in professional education. This protocol is based on a model for the
assessment of achieved learning outcomes. The Netherlands Association of Universities of
Applied Sciences conducts a pilot with the protocol to gather information about the
effectiveness and the conditions for successful implementation.
Prof. dr. Daan Andriessen is professor in the methodology of applied research at the HU
University of Applied Sciences Utrecht and chairman of the Expertgroup Protocol.
2.2. Andy Gibbs, Glasgow Caledonian University
Differing approaches to Programme Learning Outcomes
This session outlines the background to the role of Learning Outcomes in European Higher
Education Reform and discusses module learning outcomes, programme learning outcomes
and specifications. Ways in which intended learning outcomes of a Higher Education
programme and the means by which the outcomes are achieved and demonstrated are
defined. The concept of constructive alignment is discussed and exemplar material is
provided to assist colleagues in higher education to implement constructive alignment in
their programmes. Differing approaches to formulating degree programme profiles are
highlighted and potential inconsistencies in demonstrating achievement of intended
programme learning outcomes are flagged for discussion. The implications of a harmonised
rather than a standardised approach are considered in the context of employers and HEI
providers.
NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 9
3. Case studies from Quality Assurance Agencies
3.1 Karin Järplid Linde, Swedish Higher Education Authority
External quality assurance of higher education in Sweden
The first generation of quality assurance in higher education in Sweden was developed in
the beginning of the 1990‟s. However, the Swedish National Agency for Higher Education
(HSV), later Swedish Higher Education Authority (UKÄ), was not established until 1995. The
evaluation model at the time was based on the concept of quality audit and focused on the
enhancement of internal quality assurance processes at the higher education institutions.
From 2001 and onwards, programme and subject evaluation became the predominant form
of external quality assurance. In addition, thematic evaluations were conducted. In 2010 the
Government commissioned UKÄ to develop a new model for the evaluation of first and
second-cycle programmes, focusing on student attainment of intended learning outcomes
specified in the national qualification descriptors.
In contrast to previous evaluation models, this cycle almost exclusively focused on output,
and very little on pre-requisites and processes. A peer review based model was used where
self-evaluations submitted by the institutions, interviews with representatives from the
institutions and students were taken into account. However, the student‟s independent
projects became the most important factor in the experts‟ assessment of the programmes.
The assessment of independent projects was seen as a way of confirming that the students
had achieved the intended learning outcomes laid down in the qualification descriptors and
as a measure of assuring the quality of a study programme. The approach was somewhat
controversial.
Between 2011 – 2014 about 2100 programmes were evaluated according to this output
oriented evaluation model. For programmes where shortcomings were identified, a follow –
up is done after one year. UKÄ has the power to revoke the entitlement to award
qualifications, if shortcomings persist after that year. In these follow – ups, many examples
of quality enhancement activities have been shown, such as: review of courses and
programmes, curricula reviews, strengthened link between education and research, review
of internal quality assurance procedures. According to a Danish evaluation of this quality
assurance model , the evaluation system has had many effects on the institutions‟ internal
quality assurance systems. There has been an increased awareness of the national
qualification descriptions and due to stakeholders involved in developing action plans,
cooperation within universities has also increased. You could argue that the model has been
sharp as institutions were compelled to take immediate actions, for the benefit of the
students. However, the evaluation model has been criticized for having too much focus on
the students‟ output rather than on the quality of programmes.
In 2016 a new model for external quality assurance will be launched in line with the
European Standards and Guidelines (ESG). Apart from the appraisal of entitlement to award
qualifications, it is foreseen to include both institutional audits and programme evaluations
as well as thematic evaluations. The focus on output will be kept, taking the internal quality
assurance procedures at the higher education institutions into account. Thus the model will
include parts from all former quality assurance approaches, balancing the national legal
requirements as wells as international agreements. We foresee a model where the role of
NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 10
controlling that quality is being maintained is balanced with being a driver for quality
enhancement of higher education.
3.2. Dietlinde Kastelliz, AQ Austria
Learning outcomes as subject matter of external quality assurance
Learning outcomes have gained considerable significance as one of the action lines in the
Bologna Process. The ECTS users‟ guide determines learning outcomes as follows:
“Statements of what a learner knows, understands and is able to do on completion of a
learning process. The achievement of learning outcomes has to be assessed through
procedures based on clear and transparent criteria. […]”
The Austrian Agency for Quality Assurance and Accreditation (AQ Austria) is a national
body with competences in all sectors of higher education. The agency‟s responsibilities
include external quality assurance procedures, such as institutional and study programme
accreditation or audit of internal quality management systems at higher education
institutions, as well as information and consulting services, and system-wide analyses. With
this broad field of responsibilities in mind, the agency takes a holistic view on external
quality assurance.
In the Austrian higher education system, two types of external quality assurance are
predominant: one is the ex ante accreditation of study programmes, the other is the audit of
the internal quality management system of an institution.
Learning outcomes in the ex ante accreditation of study programmes are considered in at
least four criteria which refer et.al. to their definition and the scientific and professional
requirements of the respective qualification objectives, the content of the programme, the
didactics, students‟ workload, and assessment. The actual check whether the intended
learning outcomes have turned into achieved learning outcomes by the individual student is
not part of a review process by AQ Austria.
In the audit, a higher education institution has to demonstrate that its internal quality
management system is functional and efficient. This includes mechanisms and processes
for the development and further development of curricula. Therefore, the object of the
review are not the defined learning outcomes of each study programme, but the way the
institution ensures their explication and deals with the evaluation of a full study programme
and single units within it.
AQ Austria adheres to the principle that autonomous higher education institutions are in
charge of quality assurance and quality development with respect to degree programmes,
teaching, research and development, and organisation. Taking this idea of autonomy
together with the legal stipulations in the country, learning outcomes as subject matter of
external quality assurance in Austria are considered on the level of study programmes (as
opposed to the individual student) and on the process level, whereas the responsibility for
the control and assessment of their achievement stays with the individual higher education
institution.
NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 11
5. Presentations of good practices
5A.1 Daan Andriessen: Pilot Guidelines from Dutch Universities of Applied Sciences
5A.2 Oliver Vettori: - Meaningful measurements? – Between the poles of rigor and
relevance
The assessment of achieved learning outcomes remains a theoretical and practical
challenge in several aspects: It requires a carefully balanced assessment design that is
methodologically sound, yet also feasible; It needs to provide detailed information on the
performance of individual learners, but on the same time is expected to feed into institutional
processes of curriculum development and stakeholder information; and it should cover a
broad variety of competences and skills on various levels. In simple words: The
measurements should be accurate but also create meaning - demanding a tightrope walk
between rigor and relevance. This presentation will not only delve into this issue, but also
relate it to the practical case study of implementing a quality-assurance driven learning
outcome assessment process at an Austrian public university. The decisions taken in the
context of this example will provide a starting point for actively discussing the question of
priorities in assessing achieved learning outcomes.
5A.3 Johan Alvfors
5A.4 Petra Bulthuis & Anke Thijsen: The AuCom Model - Guidelines for researching,
defining and demonstrating achieved learning outcomes
The responsibility of our institutions is to ensure that our students are prepared for and can
succeed in their future work environment. The ways that we teach and the types of
assignments, projects and internships must be relevant to todays‟ world: high quality and be
developed and assessed at a level that is appropriate to the programme.
Although we have national and international qualification frameworks for determining the
different levels, they are rather abstract. It is not immediately clear what the level of a
student who graduates from the University of Applied Sciences is.
That‟s the reason why Saxion University of Applied Sciences has developed a model, the
AuCom Model, which can help formulate the level of programmes and of assignments and
projects.
In the presentation we will explain why and how the AuCom Model was developed and how
the model can be used at your own institution. To experience this model we do a practical
assignment to allow everyone to put the model into practice. By seeing how the model
works, it may generate a more practical dialogue.
5B.1: Paul Garré: EQF-levels as a corner stone for institutional quality assurance
approaches
Examples from a specific professional bachelor programme will be given to illustrate some
crucial elements of the institutional quality assurance approach. All programmes of the
university college use the same competence framework, defining in rather general terms
four categories of competence achievement (basic, growth, integration, expert). These
categories relate to developing knowledge and skills, degrees of autonomy and
NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 12
responsibility, growing complexity and novelty of contextual elements. The third level
(integration) corresponds with the attainment of EQF6. Each programme has formulated
domainspecific key learning objectives, adjusted well to this third integrative level, and thus
corresponding with EQF6. Furthermore, each course unit of the degree programme is
related to one or more of these learning objectives (specifying the expected level of
achievement). Course units from the first year of the degree programme assess their
students at level 1 or 2 of the competence framework. Exams and papers in the third (final)
year will have to be assessed at the third level (EQF6). Students are informed about these
expectations via online available ECTS information guides. Once in a six yearly quality
cycle, assessment procedures of all degree programmes and course units are thoroughly
screened by a panel of external peers and representatives of the labour market. Teachers
as well as programme responsibles receive feedback reports about the quality of their
assessment practices (in terms of transparancy, relevance, level …). Moreover, these
reports contain suggestions for improvement and appraise good practices. Again, the
attainment of EQF6 is an explicit criterium for reflection and feedback during this peer
review. Additionally, after one and a half year of work experience, alumni receive a
questionnaire containing ten crucial competences (selected from the intended learning
outcomes). They are asked if these competences (in terms of intended learning outcomes)
are still considered as relevant taking into account their first steps in the labour market and if
so, whether they had achieved these competences at the expected level when entering their
first job. This precious feedback is used to adapt the degree programmes and learning
objectives where necessary.
Points for discussion
Collateral damage of extreme emphasis on learning outcomes driven evaluation.
Does quality assurance based on learning outcomes leaves room for innovation?
What could be our role as a teacher, as a dean, as a quality agency, as a legislator to find
the right balance?
5B.2: Luc van de Poele: Systematic Data Gathering of Intended and Experienced
Learning Goals
Ghent University Quality Assurance has always put great emphasis on learning outcomes
as the basis for curriculum development and monitoring. In a research-intensive university,
the level of specification and orientation on particular professions is permanently under
discussion. Explicit learning outcomes, or competences, as the term is systematically used
at Ghent University, are described at the level of each program, and at the level of each
course. Their coherence is monitored by the Ghent University Competence Model, which
has been in use/in place since 2005. Staff is permanently being trained in formulating
good/coherent? competences. And this, in turn, is systematically monitored, university-wide
as well as ad hoc. Quality assurance at Ghent University partially relies on student surveys,
in which the students are asked to give feedback on a set of general outcomes. The results
of these two-yearly questionnaires are openly reported in a comparable format, and are
what drives the quality assurance. The surveys have become even more important now
that the external review panels on the level of the study programs have been suspended by
decree, giving the Flemish higher education institutions an opportunity to experiment with an
internal quality assurance system. At Ghent University, our programs complement the
university-wide monitoring instruments with more specific data gathering among students
and alumni about the program competences. Since 2015, both program and course
competences are part of a new competence management system, where teaching staff
NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 13
indicate to which program competences their courses contribute, with which course
competences they work in order to achieve the program goals, and which teaching
techniques they use. Ultimately, the system also captures which program competences are
being evaluated and what evaluation techniques are being used. The system is still in pilot
phase and some issues have yet to be resolved.
Points for discussion:
Collateral damage of extreme emphasis on learning outcomes-driven evaluation.
Does quality assurance based on learning outcomes leave room for innovation?
5B.3: Kees Kouwenaar: Required learning achievements at the entrance of the
master’s programme: Mastermind Europe
5B.4: Robert Coelen: Workplace Acceptance of Program Learning Outcomes
Between conceiving the program learning outcomes (PLOs) and making them achieved
outcomes through appropriate testing, there is a verification step in professional education
that involves the work field. This presentation discusses an innovative way of aligning the
PLOs with the work place needs. At the same time this approach creates input for testing
that further enhances the proximity of work place reality with class room practice and
preparation. The approach is at variance with suggested practice. The benefits and pitfalls
will be discussed.
NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 14
4 BIODATA OF THE PRESENTERS
Fiona Crozier is currently Head of International at UK Quality Assurance Agency (QAA).
Before that she was Director of the Quality Promotion Unit at (since April 2013) at the
University College Cork, Ireland and responsible for the University's internal quality
assurance processes and system. Prior to that she was Assistant Director at the UK Quality
Assurance Agency for 13 years (responsible for policy development and for implementation
of external QA processes through institutional review) and was a board member and Vice-
President of European Association for Quality Assurance (ENQA) for five years of that time.
She has been involved in many European and international projects including chairing the
MAP-ESG project and as member of the drafting group that produced the latest version of
the ESG. She has spoken at numerous conferences, workshops and seminars. She also
worked at a UK university and for the Higher Education Funding Council for England and
Northern Ireland. She is currently a member of the Advisory Board of ACSUCYL and of the
Quality Assessment Committee of AQU Catalunya.
Dr Kath Hodgson MBE Following 20 years as a school teacher and completion of her PhD
in education management Dr Kath Hodgson began working at the University of Leeds as a
Planning Officer before moving into a learning and teaching role. She became head of the
Quality Assurance Unit and subsequently the University's Director of Learning and
Teaching, responsible for curriculum enhancement and support. She also undertook over 15
institutional reviews on behalf of the UK Quality Assurance Agency. In 2012 she was
awarded an MBE for services to Higher Education.
Dr Daan Andriessen is professor of the Methodology of Applied Research at HU University
of Applied Sciences Utrecht from 2013. His research deals with the continuing
professionalization of applied research in the higher professional education in the
Netherlands. The research group conducts research into best practices and techniques for
applied research. Another focus of their work is how students and teachers learn to carry
out research. Before this, Daan Andriessen was Professor of Intellectual Capital at
Hogeschool Inholland University of Applied Sciences. He was also a member of the Forum
for Applied Research and assumed responsibility for establishing a Code of Conduct for
Applied Research in higher professional education (HBO) in the Netherlands. In addition to
his work as a professor, Daan Andriessen is also a research coach at Onderzoekcoach.nl.
Through this website, he supervises Bachelor‟s, Master‟s and PhD students who are
designing and conducting research. Daan Andriessen gained his PhD in 2003 from
Nyenrode Business University with a thesis on the value of knowledge. Before he was
appointed as professor, he spent twelve years as an organizational consultant at
consultancy firm KPMG.
Andy Gibbs is Senior Lecturer at Glasgow Caledonian University responsible for
internationalisation of the curriculum. He is a UK European Higher Education Expert with a
particular focus on quality, mobility and curriculum development. He has completed a
number of related projects in Europe and established trans national education curricula in
SE Asia and China.
Karin Järplid Linde is Head of Department of Quality Assurance at the Swedish Higher
Education Authority. She has 13 years of experience in external quality assurance of higher
education, with different positions on a management level for the past five years. She has
NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 15
had several international engagements in external quality assurance in Europe and
represents the Swedish Higher Education Authority in various networks, such as ENQA.
She was also a member of the National Team of Bologna Experts 2007-2014.
Dietlinde Kastelliz is currently head of the department “Audit and Consulting” at the
Austrian Agency for Quality Assurance and Accreditation (AQ Austria) and was a project
coordinator at one of the agency‟s predecessors. Prior to that, she worked in the Austrian
Federal Ministry of Foreign Affairs. After graduation from the University of Graz, she acted in
different educational contexts as lecturer and office manager in Kyrgyzstan, the Ukraine and
Croatia for seven years, before coming back to work at the University of Graz. Dietlinde has
been involved in several projects related to quality management in higher education.
Oliver Vettori is Dean for Accreditations and Quality Management as well as Director of the
Department for Programme Management and Teaching & Learning Affairs at WU, the
Vienna University of Economics and Business, Austria. He obtained his PhD from the
University of Vienna in the field of Sociology and Organisation Theory and is still is also
liaised as a research associate with the Institute for Public Management and the Institute for
Organization Studies (both at WU). Oliver Vettori is the initiator and coordinator of the
Network for Quality Assurance and Quality Development of the Austrian Universities, a team
coordinator for the Institutional Evaluation Programme of the European University
Association (EUA) and has been frequently working as a reviewer , consultant and
evaluator, i.a. for ENQA, evalag, ZeVa, AQ-Austria and ASEAN-QA as well as for numerous
universities across Europe.
Johan Alvfors
Petra Bulthuis MSc is senior educational advisor at the School of Marketing & International
Management of Saxion University of Applied Sciences. Her job is to advise management
and teachers on improving the educational quality. She has broad experience as a teacher
and educational advisor at Institutes for MBO (vocational education) and Higher Education.
She develops tools and guides on assessment, study career guidance, curriculum design
and other subjects.
Anke Thijssen MSc is senior policy advisor at the department of Quality Assurance of
Saxion University of Applied Sciences. Her particular focus now is on quality assurance
issues, such as development and implementation of quality assurance protocols,
coordination of accreditation for Saxion and coaching schools in the preparation for internal
and external audits. Since this year she is a member of the Dutch Bologna Expert Group.
Prior to this job she worked as a policy officer at the economic unit of one of the precursors
of the MBO Raad, the Dutch association of VET Colleges, and at the University of Twente
as a researcher on curriculum development.
Paul Garré
Luc van de Poele
Kees Kouwenaar
Robert Coelen
NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 16
5 LIST OF PARTICIPANTS
Gabriele Abermann
Salzburg University of Applied Sciences
Austria
Axel Aerden
NVAO
Belgium
Johan Alvfors
Swedish National Union of Students (SFS)
Sweden
Christine Anderton
Nyenrode New Business School
The Netherlands
Dr. Daan Andriessen
HU University of Applied Sciences Utrecht
The Netherlands
Dr. Katrien Bardoel
EP-Nuffic
The Netherlands
Hélène Bartelds MA
Erasmus Universiteit Rotterdam
The Netherlands
Bartlomiej Banaszak
Ministry of Education
Poland
Rianne Bechet-Tjoonk
The Hague University
The Netherlands
Isabel Belmonte
Agency for Quality Assurance in the
Galician University System (ACSUG)
Spain
Ilze Birzniece
Riga Technical University
Latvia
Audbjorg Bjornsdottir
University of Akureyri
Iceland
Professor Emita Blagdan
Agency for Science and Higher Education
Croatia
Rihards Blese
Higher Education Quality Agency (AIKA)
Latvia
Lucien Bollaert
NVAO
Belgium
Hege Brodahl
Norwegian Agency for Quality Assurance in
Education
Norway
Elisabeth Brunner-Sobanski
University of Applied Sciences bfi Vienna
Austria
Petra Bulthuis MSc
Saxion University of Applied Sciences
The Netherlands
Bente Ringlund Bunæs
The Norwegian Association of Higher
Education Institutions Address
Norway
Dr. Robert Coelen
Stenden University of Applied Sciences
Leeuwarden
The Netherlands
Anna Collard
NOKUT
Norway
Perry Coppiëns
NUFFIC/OCW
The Netherlands
NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 17
Fiona Crozier
The Quality Assurance Agency for Higher
Education, UK
UK
Steven David
NVAO
The Netherlands
Dr. Thomas de Bruijn
NVAO
The Netherlands
Erick de Jong
UNESCO-IHE
The Netherlands
Nicole de Leeuw
NHTV Breda University of Applied Sciences
The Netherlands
Yvonne de Vries
Tilburg University
The Netherlands
Lieselotte de Zutter
NVAO
Belgium
Gisli Fannberg
University of Iceland
Iceland
Berta Fernandez Alvarez
UNESCO-IHE
The Netherlands
Dr. Mark Frederiks
NVAO/ECA
The Netherlands
Ingrid Furfjord
The Norwegian Police University College
Norway
Paul Garré
Departement Onderwijs Vlaanderen
Belgium
Drs. Guðrún Geirsdóttir
University of Iceland
Iceland
Andy Gibbs BSC (Hons), MSC, PGCE, RN
School of Health and Life Sciences,
Glasgow Caledonian University
UK
Anna Gover
European University Association
EU
Inge Greving
Hanzehogeschool Groningen
The Netherlands
Vidar Gynnild
Norwegian University of Science and
Technology
Norway
Varduhi Gyulazyan
ANQA
Armenia
Dr. Jon Haakstad
Norwegian Agency for Quality Assurance in
Education (NOKUT)
Norway
Professor Hans Hellsten
Lund University, Faculty of Performing Arts
in Malmö
Sweden
Dr. Kath Hodgson MBE
Leeds University
UK
Ellen Jagtman
NUFFIC/OCW
The Netherlands
NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 18
Karin Järplid Linde
Swedish Higher Education Authority
Sweden
Hallur Jonasson
The Icelandic Centre for Research
Iceland
Mag. Dietlinde Kastelliz MA
Agentur für Qualitätssicherung und
Akkreditierung Austria
Austria
Maria-Regina Kecht Mag.
Austrian Private University Conference
Austria
Åsa Kettis
Uppsala University
Sweden
Monique Knoester
NVAO
The Netherlands
Eveline Kok
VU Free University Amsterdam
The Netherlands
Jonna Korhonen
European University Association
EU
Kees Kouwenaar
VU Free University Amsterdam
The Netherlands
Professor Berta Leeb
Private Pädagogische Hochschule der
Diözese Linz
Austria
Drs. Patrick Leushuis
Ministry of Education, Culture and Science
The Netherlands
Dr. Eric Lindesjöö
Linköping University
Sweden
Sandra Louwersheimer
NVAO
The Netherlands
Mariusz Maciejczak
Warsaw University of Life Sciences
Poland
Marta Mallo Rey
Agency for Quality Assurance in the
Galician University System (ACSUG)
Spain
Maciej Markowski
Polish Accreditation Committee
Poland
Professor Vladimir Mrša
University of Zagreb, Faculty of Food and
Biotechnology
Croatia
Dr. Andreas Neuhold
Ministry of Science, Research and
Economy
Austria
Professor Anna Elisabet Olafsdottir
Bifrost University
Iceland
Joy Plokker
NUFFIC/OCW
The Netherlands
Henri Ponds
NVAO
The Netherlands
Dr. Alba Prieto-Gonzalez
European Union (EACEA)
NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 19
Maria Próchnicka
Jagellonian University, Krakow
Poland
Ana Proroković
replaced by: Leonardo Marušić
University of Zadar, Department of
Psychology
Croatia
Dr. Dagmar Provijn
NVAO
Belgium
Christina Raab
University of Innsbruck
Austria
Rebekka Silvía Ragnarsdóttir
Icelandic Erasmus+ National Agency -
Icelandic Centre for Research
Iceland
Dr. Andrejs Rauhvargers
Quality Assurance Agency
Latvia
Gudrun Salmhofer
University of Graz
Austria
Harald Scheuthle
Evaluationsagentur Baden-Württemberg
Germany
Drs. Anne Klaas Schilder
NUFFIC/OCW
The Netherlands
Tineke Sessink
Platfom Life Long Learning
The Netherlands
Sigurdur Sigurdsson
The Icelandic Centre for Research
Iceland
Gerard Sijben MSc
Fontys University of Applied Science
The Netherlands
Professor Paweł Stępień
University of Warsaw
Poland
Britta Stoelinga
Rotterdam Business School
The Netherlands
Una Strand Vidarsdottir
Ministry of Education, Science and Culture
Iceland
Susann Stroemsvaag
The National Union of Students in Norway
Norway
Prof.dr. Ana Tecilazić Goršić
Ministry of Science, Education and Sports
Croatia
Anke Thijsen MSc
Saxion University of Applied Sciences
The Netherlands
Professor Ruben Topchyan
ANQA
Armenia
Jeff van As
NUFFIC/OCW
The Netherlands
Lineke van Bruggen
NVAO
The Netherlands
Luc Van de Poele
Ghent University
Belgium
Tim van den Brink
NUFFIC/OCW
IJda van den Hout
NVAO
NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 20
The Netherlands The Netherlands
Leonard van der Hout
NUFFIC/OCW
The Netherlands
Jolien van der Vegt
NUFFIC/OCW
The Netherlands
Dr. Els van der Werf
NUFFIC/OCW
The Netherlands
Annemiek Verkuijl
International Business School/ Amsterdam
University of Applied Science
The Netherlands
Oliver Vettori
WU -Vienna University of Economics and
Business
Austria
Professor Annick Vlaminckx
Karel de Grote-Hogeschool
Belgium
Dr. Martina Vukasovic
European Union (EACEA)
Belgium
Michèle Wera
NVAO
The Netherlands
Agata Wroczyńska
University of Warsaw
Poland
Drs. Sarmite Zeire
Riga Stradins University
Latvia
NVAO | Peer Learning Event on Achieved Learning Outcomes | The Hague 29-30 October 2015 page 21
Accreditation Organisation of the Netherlands and Flanders (NVAO)
Parkstraat 28
P.O Box 85498 | 2508 CD DEN HAAG
T 31 70 312 23 00
www.nvao.net