maintaining a quality culture in doctoral education...maintaining a quality culture in doctoral...

Maintaining a quality culture in doctoral educationat research-intensive universities

University of Amsterdam - Universitat de Barcelona - University of Cambridge - University of Edinburgh

University of Freiburg - Université de Genève - Universität Heidelberg - University of Helsinki

Universiteit Leiden - KU Leuven - Imperial College London - University College London - Lund University

University of Milan - Ludwig-Maximilians-Universität München - University of Oxford

Pierre & Marie Curie University - Université Paris-Sud - University of Strasbourg

Utrecht University - University of Zurich

ADVICE PAPERno.19 - March 2016

LEAGUE OF EUROPEAN RESEARCH UNIVERSITIES

Acknowledgements

LERU is grateful to the lead authors of the paper:

• David Bogle, Professor of Chemical Engineering, Pro-Vice-Provost, Head of the Doctoral School, University College London, and Chair of LERU’s Doctoral Studies Community

• Jacqui Shykoff, Senior Research Scientist CNRS, Director of the Doctoral School in Plant Sciences at the Université Paris-Sud and steering group member of LERU’s Doctoral Studies Community

• Isolde von Bülow, Dr. rer. nat. of Molecular Biology, Head of the GraduateCenterLMU and steering group member of LERU’s Doctoral Studies Community

The lead authors wish to thank all members of the LERU Doctoral Studies Community and other LERU colleagues for

their help in developing the ideas in the paper and collecting good practice examples, as well as the Policy Committee, the Vice-Rectors Research and the Rectors’ Assembly for their valuable feedback on the paper.

Dr Katrien Maes, Chief Policy Officer of LERU, has supported the development of the paper.

Contents

3 Executive summary5 Recommendations6 Introduction7 Step 1: Expectations and principles12 Step 2: Scrutiny processes14 Step 3: Measurement15 Step 4: Feedback mechanisms and quality enhancement17 Conclusions and recommendations18 References20 Appendix A: Quality culture in practice at LERU universities36 Appendix B: Characteristics and skills sets of doctoral graduates

About LERULERU was founded in 2002 as an association of research-intensive universities sharing the values of high-quality teaching in an environment of internationally competitive research. The League is committed to: education through an awareness of the frontiers of human understanding; the creation of new knowledge through basic research, which is the ultimate source of innovation in society; the promotion of research across a broad front, which creates a unique capacity to recon-figure activities in response to new opportunities and problems. The purpose of the League is to advocate these values, to influence policy in Europe and to develop best practice through mutual exchange of experience.

3

EXECUTIVE SUMMARY

The training of doctoral graduates is at the heart of the mission of research-intensive universities (RIUs). Doctoral programmes within LERU aim to train the next generation of researchers to the highest skill levels in order to launch creative, critical and autonomous intellectual risk takers who will push back the frontiers of research. In addition, the modern doctorate needs to provide excellent training for roles beyond research and higher education, preparing doctoral graduates for a variety of careers that require deep rigorous analysis in public, charitable and private sectors.

How can universities ensure that these objectives will be achieved? They do this by ensuring that they maintain doctoral training embedded in a strong research culture and through Quality Assurance (QA) processes which scrutinise and enhance this culture and the activities.

A high quality research culture encourages doctoral candidates and their supervisors to pursue challenging questions with creativity and rigour. The culture must support all to work with integrity - to discuss and explore the pressures on researchers, the standards that are expected, and the wider ramifications on society of the research work. There must be a clear sense that research work is valued and supported as well as properly resourced with appropriate facilities. This requires excellent researchers supported by appropriate resources, but most importantly by strong leadership signalling the importance of research to the mission of the university.

To be satisfied that this culture is present and being maintained we need robust quality assurance processes. Much of this paper focusses on how this is accomplished at LERU universities. Quality assurance involves the following elements:1. Defining of expectations 2. Setting up scrutiny processes to explore whether expectations are met3. Measuring key quality indicators4. Providing feedback mechanisms to facilitate both correction and enhancement of the system

Within these elements QA in doctoral education can be considered at several different levels. At each level doctoral education should be considered together with the overall research environment.

A first level addresses QA of structural and administrative aspects of doctoral education as implemented within a pro-gramme, department, institute or faculty. There should be clear and easily navigated structures and administrative requirements (for recruitment, admission, examination, etc.) to ensure that suitable people are recruited and they are properly supported throughout their programme. Doctoral candidates must be able to find what they need efficiently to enable them to focus on the research and training programme. This is also particularly important in order to facili-tate the mobility of doctoral candidates within Europe, both to capitalise on the complementarity of training opportu-nities and to strengthen the research ties among European partners. Procedural quality assurance is not specific to the doctorate, and is probably best managed at the university level.

The second level involves the quality of each doctoral research training programme, which may be either an individual or a structured programme within a cohort. It is important that doctoral trainees be integrated into challenging and stimulating research environments and mentored by suitably qualified supervisors who can devote an appropriate amount of time and investment to their training. It is useful that the expectations and general training plan be explicitly defined as early in the doctoral experience as possible, for example by establishing a written statement that outlines the research area together with training goals, activities and expectations. The plan should be subjected to regular scrutiny to assess progress, measure quality of the output, the satisfaction of the doctoral candidate and the supervisory team, and revised as necessary. Responsibility for this level of quality assurance can be engaged in smaller scale structures, for example departments or doctoral schools, with a degree of independent oversight at institutional level.

A third level of QA involves assessing and enhancing the quality of the output. For accountability to society and for potential future employers it is important that the doctorates fulfil particular standards and criteria appropriate for research training. A doctorate from a LERU university should prepare its graduates to be cutting-edge thinkers ready to confront future challenges in a broad set of contexts and roles. Each doctoral candidate must have developed critical

Maintaining a quality culture in doctoral education at research-intensive universities

4


thinking and analysis and be working with integrity, either through formal training courses or through supervised research. This should be a natural part of a candidate’s exposure to an excellent research environment in her or his disciplinary specialty, with an emphasis on adaptability. Furthermore, it is important to continually revise and upgrade training programmes to meet changing challenges and demands of the employment market and of society.

LERU universities have a diversity of approaches to confronting the challenge of maintaining a quality culture and of specific quality assessment and quality enhancement processes of their doctoral training. The report is structured to give examples of processes used to fulfil each of the four elements for Quality Assurance given above. We do not propose a single solution to this challenge, nor do we have an answer for all contingencies. We do provide some ideas and examples from LERU universities on how we develop and review our quality culture and where quality assessment procedures could be usefully implemented. Many individual examples are representative of similar practices at other LERU members and other research-intensive universities. Finally, on the basis of the insight gained from analysing quality culture processes at LERU universities, we propose a number of recommendations for universities, policyma-kers and funders, which are listed hereafter.

5

Advice pAper - No. 19, MArch 2016

Recommendations

We recommend that universities:

1. Ensure that funded doctoral programmes are embedded in a strong research environment supporting candidates to work with rigour and integrity. There should be strong university leadership signaling the importance of re-search and research training to the mission of the university.

2. Have a doctoral education quality assurance system in place which considers the entire quality assurance cycle in a virtuous circle, including the following crucial elements as a minimum: 1/ clearly stated expectations, 2/ transparent scrutiny processes, 3/ documented measurements, and 4/ effective channels for feedback and quality enhance-ment. QA should review programmes and departments (institutes) as well as individual doctoral outcomes with all processes involving independent parties.

3. Ensure that review processes consider the quality of the research environment as well as the quality of the outputs.

4. Ensure that processes are transparent and information easily accessible.

5. Take note of and make available good practice elements from within and outside of the university.

We recommend that policymakers and funders (e.g. governments, research councils, charities, companies):

6. Recognise the important role that quality assurance has in maintaining a high quality research culture.

7. Ensure that quality assurance processes of institutions where doctoral candidates are funded consider the quality of the research environment as well as the quality of research programme outputs.

8. Recognise that there are many ways of undertaking quality assurance of doctoral programmes.

9. Recognise the key role that the review cycle works in a virtuous circle, which includes as four essential elements - clearly stated expectations, - transparent scrutiny processes, - documented measurements, and - effective channels for feedback and quality enhancement, and which reviews programmes as well as individual doctoral outcomes. Reviews should involve parties not in-

volved in each specific programme.

10. Acknowledge, encourage and reward institutions that have good quality assurance systems in place.

6


Introduction

1. The training of doctoral graduates is at the heart of the mission of research-intensive universities (RIUs). Doctoral programmes at LERU universities aim to train researchers to the highest skill levels to become creative, critical and autonomous intellectual risk ta-kers in pushing the boundaries of frontier research. The modern doctorate is an excellent training for those who go into roles beyond research and education, in the public, charitable and private sectors, where deep rigorous analysis is required. The values and charac-teristics of doctoral education as agreed by LERU uni-versities are expressed in earlier LERU papers (LERU 2014a, 2010, 2007). The main purpose and principles of doctoral education described in these papers are re-produced in the shaded boxes below.

2. Other European organisations have published position papers on doctoral education in recent years (Coim-bra, 2007; EC, 2011; EUA, 2005, 2011; Orpheus et al., 2012), as also has the Group of Eight leading Australian research-intensive universities (Go8, 2013). RIUs’ com-mitment to doctoral education is included in the Hefei Statement, which defines the characteristics of RIUs and was signed by LERU and other networks of leading RIUs around the world in 2013 (LERU, 2013).

3. While direct responsibility for doctoral education in general, and for quality assurance (QA) in par-ticular, lies with universities overseen by national or devolved authorities, it is clearly also a European issue. Doctoral education is recognised as the third cycle in the Bologna Process and is a crucial part of the EU’s policy of achieving the European Research Area and of its funding programmes such as Hori-zon 2020 and Erasmus+. LERU has played an active role in advocating the importance of high-quality, research -rich and innovative doctoral education at the European level (LERU, 2014b). The EU can cer-tainly have a contributing and supportive role to en-hance a quality culture, by bringing actors together from across Europe and encouraging exchange of good practice, thus helping universities to develop and maintain good QA. This will help to foster the take-up and implementation of the Innovative Doc-toral Training principles promoted by the EC (2011).

4. A high quality research culture encourages research-ers, including doctoral candidates and their supervi-

sors, to pursue challenging questions with creativi-ty and rigour. The culture must support all to work with integrity - to discuss and explore the pressures on researchers, the standards that are expected, and the wider ramifications on society of the research work. There must be a clear sense that research work is valued and supported as well as properly resourced with ap-propriate facilities. This requires excellent researchers supported by appropriate resources, but most impor-tantly by strong leadership signalling the importance of research to the mission of the university.

5. Doctoral education should be undertaken within departments and faculties where there is a strong research culture. Most staff should be research-active and engaged in vigorous research programmes funded by competitive external sources as well as by the university. The Hefei Statement outlines the characteristics of research universities where “a research culture permeates all of its activities” and by “the pursuit of excellence across all its operations, cali-brated though informed, independent, dis inte rested assessments from peer organisations and individuals from outside the university; and a commitment to transparent, meritocratic systems for selecting faculty, staff and students, creating an internal environment that nurtures learning, creativity and discovery…” (LERU, 2013). Other LERU papers also outline the characteristics of RIUs and their research culture (LERU 2014b, 2008). With these characteristics it is possible to ensure that doctoral candidates are well supervised and have the right opportunities to pursue original research and gain the necessary research skills.

6. How do universities go about ensuring that they are achieving their objectives? They do this through Quality assurance (QA) processes. Quality assurance articulates and implements systems that engage in the following processes, as illustrated in Figure 1:1. Define the expectations and principles of the pro-

gramme2. Set up scrutiny processes to explore the achieve-

ment of these expectations3. Measure the achievement of key quality indicators4. Provide feedback mechanisms to inform both cor-

rection and enhancement of the system

7


1 See http://www.ref.ac.uk/

7. Much has been written on quality assurance of taught programmes (see for example ENQA, 2009) but there are fewer codes for doctoral education. The European University Association (EUA, 2015) produced a report on quality assurance of doctoral education, exploring current practices across Eu-rope. The UK’s Quality Assurance Agency specifies 18 indicators which need to be met (UKQAA, 2013). The French Government has published a charter for doctoral education (French Ministery for Edu-cation and Research, 2006, 1998). The Dutch Med-ical Schools have guidelines for PhDs in biomedical sciences (NFU, 2011). Since the doctorate is a train-ing in research, the quality of the research environ-ment will be an important part of the scrutiny. Many countries have systems which explore the quality of the research environment and sometimes include aspects of doctoral education, for example the UK’s Research Excellence Framework1.

8. There are some aspects of QA in all three cycles with-in higher education that are common. However, the doctoral cycle is recognised as being distinctly dif-ferent to the Bachelor’s and Master’s cycles and so the QA processes are likely to be quite different. A characteristic of RIUs is that all academic units are research-active and the research also informs the edu cation of Bachelor’s and Master’s students.

9. Following the earlier LERU papers on the principles and on good practice in contemporary doctoral educa-tion, the present paper focusses on the characteristics, mechanisms and other aspects of quality assurance (and quality enhancement) that are suitable for the special case of RIUs and are practiced by LERU mem-ber universities. We propose that there is no single model for quality assurance that will work for all situa-tions. Our goal is to present a number of QA mecha-nisms from the experience of LERU universities. On the basis of the characteristics and mechanisms iden-

tified, we formulate a number of recommendations for universities, policymakers and funders.

10. The report has been structured along the lines of the four elements listed above including examples of each of the three levels: procedures, individual train-ing programmes, and output standards. In each case there are elements that address the quality of the uni-versity and its management of doctoral education and those of individual doctoral candidates’ experience.

11. This report considers principles of each of these four elements, for departments, programmes and for in-dividual candidates, and gives brief outlines of some of the practices at LERU universities. Further exam-ples from LERU universities are given in Appendix A.

Step 1: Expectations and principles

12. Each university or national agency must set up the expectations applicable to all actors in doctoral edu-cation: doctoral candidates, supervisors, depart-ments, external partners and the central structures of the university. The expectations are usually ex-pressed alongside the lifecycle of the programmes: promotion and marketing, the application process and admissions, induction, quality of research en-vironment, supervision, progression monitoring, complaints procedures, examination, award and post-award relationship. LERU (2010) identified the following as characteristics of PhD graduates:

• Doctoral graduates are best known for their analy-tical power and technical expertise which they have learnt to apply rigorously. However, the range of skills they develop is much wider. This is often not even recognised by the graduates themselves, al-though the increasing focus on skills development is helping to overcome this.

• Research-intensive universities aim to produce doc-toral graduates with a broad range of skills. Intel-lectual and academic skills are developed to a much deeper extent than is done at the Bachelor’s or Mas-ter’s level, and doctoral graduates are trained to be more inquisitive and independent. In addition, per-sonal and professional management skills are deve-loped as part of the doctoral experience. Research degrees concentrate strongly on the transferable skills relevant to research and such skills are relevant

Figure 1. Quality assurance processes

Expectations and principles

Measurement

Scrunity processesFeedback mechanisms and

quality enhancement

8


not only for the research workplace but also for other places of employment. Where appropriate, doctoral programmes focus on their transferability to other domains in which a high level of creative thinking and critical analysis are needed. This broad range of skill sets outlined in the report is given in Appendix B and a description of the modern doctorate is given in Box 1.

13. The following section breaks the expectations for QA in doctoral education into: a) institutional expectations and oversight of the

culture and quality of doctoral educationb) recruitment, promotion, marketing and admissionc) research environment, supervision and research

integrityd) learning outcomes and assessment.

1a) Institutional expectations and oversight of the culture and quality of doctoral education

14. Doctoral education (as outlined in Box 2 ‘The pur-pose of a doctorate’) has more in common with the research activities within a university than with the taught programmes. However, it is still a formal part of the education process, so programmes usually have a governance structure for regulation and over-sight specifically for research degrees. This structure requires representation from academic units, those responsible for the research environment (libraries, research training provision, management of degrees, etc.), and from the candidate population to ensure that the regulations and administration meet the needs of all relevant parts of the university. The structure is re-

Box 1: A model of the modern doctorate (LERU, 2010)

Doctoral training should be based on the following principles:

• Doctoral researchers are the drivers of their professional development… In order to develop as autonomous researchers, doctoral candidates should be the drivers of their project. They

should take responsibility at a very early stage for the scope, direction and progress of their project, and this should allow them to make a demonstrably novel independent contribution to their subject of study. The degree of auto-nomy which they take on at different stages will vary between disciplines.

• While being immersed in a research-rich environment… Programmes in specific areas should be developed in the context of a strong research environment with critical

mass of researchers, equipment, and administrative and personal support.

• Where boundaries to other research fields are highly permeable… It is recognised that many of the significant advances are developed at the boundaries of disciplines. Researchers

must have the opportunity to be able to cross these boundaries according to the needs of their project. The environ-ment should provide access to these opportunities and support the candidate in exploring new avenues.

• And in which connections to the external world have a global outlook… Research is an international business. Doctoral programmes should encourage experience of the research world at least

through attendance and presentation at seminars and conferences in other countries and institutions. However, they should also seek to provide opportunities for candidates to spend longer periods away from their home institution (po-tentially outside their own country and some outside Europe (EC, 2009) in order to be exposed to fresh ideas, to different research cultures, and to have access to different facilities and techniques. This is an area where European higher educa-tion has significant experience and resources. This could be further encouraged as a key feature of European doctorates.

• And link to other sectors of society… Through conferences and other professional activities doctoral candidates should be making links with society

beyond academia to seek fresh ideas for their research, to develop ways of communicating their ideas and results, and their significance, to a wide variety of audiences, and to develop broader career perspectives.

• So that the skills the new doctors develop are highly valuable to the knowledge society. The doctoral training process should be seen as one of skills acquisition as well as developing experience and ex-

pertise in a particular field.

9


2 The figure has risen to 24% according to the EC’s 2014 Researchers’ Report (EC, 2014).

3 See http://ec.europa.eu/euraxess/index.cfm/rights/europeanCharter

4 The 1998 decree on the charter (in French) can be found online at http://www.education.gouv.fr/bo/1998/36/sup.htm together with the 2006

decree which specifies who should sign the charter. The UPMC charter can be found at http://ifd.upmc.fr/modules/resources/download/ifd/ver-

sionanglaise/CharteGBdefinitive.pdf (in English).

sponsible for ensuring that doctoral education meets the published expectations of the university.

15. Rights and obligations of doctoral candidates and supervisors (UPMC, Paris-Sud, Strasbourg)

Since 1998 the rights and obligations of all stake-holders in doctoral education at all French uni-versities are defined in the Charter of Doctorate, a contractual and regulatory document signed by the candidate, her/his supervisor(s), the head of the lab, the director of the doctoral schools, and the rector. At UPMC, this text was updated in 2007 and is based on the European Charter for Researchers3, and re-views all phases of the doctorate, from registration over follow-up to defence and publications. It also includes a paragraph on conflict resolution. The Charter of Doctorate describes the basis of the docto-ral policy at UPMC4.

16. A university-wide quality programme framework: heiDOCS (Universität Heidelberg)

The Universität Heidelberg has introduced a uni-versity-wide quality programme named heiDOCS in order to assure high quality standards in doctoral education and to continuously assess and enhance framework conditions in doctoral education. hei-DOCS is embedded in Heidelberg’s quality manage-

ment system heiQUALITY and aims to integrate all existing services and programmes related to docto-ral education within the university by drawing on a new and comprehensive analysis of the situation of doctoral education and of doctoral students’ needs. For this purpose, all doctoral students register for an online doctoral file, in which they provide data on their doctoral project as well as information on their individual situation. This is complemented by peri-odic surveys on additional aspects such as doctoral candidates’ special training needs. Heidelberg deci-sively opts for maintaining various models of docto-ral education, ranging from the individual doctorate to structured programmes. It emphasises providing conducive framework conditions for all its junior researchers regardless which model they pursue. To that end, the Graduate Academy functions as an um-brella structure for doctoral support services, lining up its services with the twelve Faculties. In addition, the Rectorate allocates further resources to the Fac-ulties to improve supporting measures and struc-tures if these demonstrably contribute to the Univer-sity’s eight quality standards in doctoral education. These standards are monitored by the Council for Graduate Studies, which was established in 2013 and brings together representatives of the Rectorate, Fac-ulties, Graduate Schools and doctoral candidates. At

Box 2: The purpose of a doctorate (LERU, 2010)

• Research degrees at doctoral level aim to take bright Master’s graduates with an excellent academic track record (sometimes with work experience) to become creative, critical, autonomous researchers. The evidence of success of the doctorate is a thesis which contains a significant original contribution to knowledge in the chosen field, with the arguments successfully defended by the candidate through questioning by experts. Detailed characteristics are described in LERU (2007).

• The process of doctoral study develops in the candidate a range of skills to a very advanced level. These skills relate not only to the research process itself, but also to a broader personal and professional training and development. The latter skills are often labelled as ‘generic’ or ‘transferable’, because they are valuable not only for the successful com-pletion of the doctorate, but also for career development after the doctorate in a wide range of professional sectors.

• By training inquisitive, independent doctoral graduates European universities fuel the objectives of the Lisbon Agenda for an advanced knowledge based economy. Since many doctoral graduates seek careers beyond the EU, either by returning to their country of origin or seeking career options abroad, Europe’s universities are proud to contribute to the advancement of knowledge in other parts of the developed and developing world. 16.9% of doc-toral students in the EU’s universities come from third countries (European Commission, 20072) and within LERU universities this ranges between 12% and 35%. The research efforts of the European Union also benefit from the choice of some non-European doctoral graduates to stay and work here.

10


19. Online application tool (LMU Munich) The GraduateCenterLMU provides an online applica-

tion tool for all LMU doctoral programmes (over 30) to support the recruitment of doctoral researchers. This guarantees equal quality standards for all appli-cation processes. The online accessibility of the tool increases international visibility and attractiveness. It includes standardised modules to collect a set of basic data, which is mandatory for all applicants. Additionally, customised modules are shaped to the local requirements of the specific programmes. Together with standardised communication rou-tines the online tool improves transparency from the initial filing of an application, over a pre-check concerning completeness and formal requirements to the final acceptance of an applicant. Evaluation and selection of the candidates is done by the respec-tive recruitment committee of each programme and the related discipline-specific administrative units. Since the tool offers not only online application but also online evaluation, it is very well suited for inter-institutional and international consortia, e.g. Euro-pean Training Networks.

1c) Research environment, supervision and integrity

20. Doctoral education must take place in a research-rich environment. There must be appropriate expertise, experienced supervisors and opportunities to train new supervisors, induction activities for new can-didates, good facilities, opportunities to develop international networks, and transparent monitor-ing processes to ensure steady progress. Below are examples showing how universities or programmes ensure that specific aspects of the research environ-ment are overseen. Supervision agreements are often used to define expectations for specific candidates and examples are given in the section on Scrutiny.

21. Ensuring a strong research environment (University College London)5

All academic departments at UCL are research active and have doctoral students. Supervisors for an indi-

the same time, the success of the measures under heiDOCS are assessed within heiQUALITY’s domain ‘young academics and research’.

1b) Recruitment, promotion, marketing and admission

17. Research-intensive universities seek to attract docto-ral candidates who are well qualified (almost always to Master’s level), have demonstrated some aptitude for research, and are strongly motivated. They recruit those who wish to work in areas where the university has expertise. It is important to have open recruitment with transparent selection criteria and processes. In-creasingly, universities are seeking to ensure that they are open and attractive to candidates from the full diversity of the population especially concerning gen-der, age and those from non-traditional backgrounds and ethnicities. This is one of the principles of the Eu-ropean Charter on Researchers. Many universities ad-vertise their positions on EURAXESS to ensure open and transparent recruiting. Given that research is an international and indeed global undertaking (cf. point 4 in Box 1), it is a characteristic of doctoral education at research-intensive universities that they are open to candidates from all around the world.

18. Advertising and recruitment (Leiden University/LUMC) As a rule, vacancies for PhD student projects are

advertised and filled in open competition following advertisements in internal or external media. In the Honours Programmes, highly talented Bachelor’s students are selected for special MSc/PhD or MD/PhD programmes. Formally, admission to the Leiden University Medical Centre (LUMC) Graduate School and registration at LUMC (as an employed PhD stu-dent or PhD student guest researcher) are two dif-ferent processes. The standard PhD admission rule is on the basis of a Master’s Degree granted by a Dutch research university. The Head of the Graduate School (Dean at the LUMC) may grant exemption to the standard admission rule. Diplomas, including originality checks, CV and language proficiencies are among the items considered for granting exemption.

5 Key Links:

Academic Manual: Recruitment of Research Students To New Academic or Research Units: http://www.ucl.ac.uk/academic-manual/part-4/

new_academic_units

Procedures and Regulations for Appointment of Supervisors, Academic Regulations for Research Degree Students, Section 1: http://www.ucl.

ac.uk/srs/academic-regulations

Information on Approval and Recording of Research Degree Supervisors, including the EROS system, is on the Doctoral School website: http://

www.grad.ucl.ac.uk/ucleros/index.html

UCL Professional Development Programme for Research Student Supervisors: http://www.ucl.ac.uk/calt/support/phd-supervisors

Academic Manual Part 5, Monitoring the Supervision of Research Students: http://www.ucl.ac.uk/academic-manual/part-5/pgr-supervision

UCL Fieldwork Approved Code of Practice: http://www.ucl.ac.uk/estates/safetynet/guidance/fieldwork/acop.pdf

UCL Research Ethics Committee: http://ucl.ac.uk/research-ethics-committee

11


vidual student may come from more than one depart-ment or faculty. Students may belong to a number of ‘communities’ – research group, department, centre for doctoral training, academic society etc., but every student is registered in only one department, which is ultimately responsible for the quality of their edu-cation. New departments (or those merged from other institutions) must demonstrate that they have the capability and capacity to provide an excellent environment before they are permitted to admit doc-toral students.

22. Any member of staff who wishes to supervise re-search students must be approved by the faculty. All new members of staff who wish to supervise must demonstrate to the faculty that they have a good re-search record and must attend a mandatory briefing session about UCL’s process, people involved in the governance and administration of doctoral educa-tion, and an introduction to good practice. Longer sessions involving discussion about issues arising in supervision are given regularly and well attended; they must be attended by probationary members of academic staff (in their first three years of an aca-demic appointment). UCL also provides training for new examiners of doctoral degrees. More details on these programmes are given in Appendix A.

23. There is a limit to the number of students an indi-vidual supervisor may supervise. There is a formula (1.0 as primary for a full-time student, 0.5 as sub-sidiary, 0.5 primary for part-time, etc.) allowing a maximum total of 6.0 with an absolute maximum of involvement with nine students. If a staff member demonstrates poor supervisory practice over an ex-tended period, their permission to supervise can be withdrawn through a four-stage review process.

24. Doctoral work is expected to be based at UCL. Stu-dents may take study leave to pursue their work else-where (for example archaeological digs, field work, exploring archives, or benefiting from specialist sci-entific equipment), but UCL’s procedures relating to risk assessment, health and safety and ethics must be followed.

25. Joint programme to create critical mass (University of Zurich)

The University of Zurich receives funding for the creation of interuniversity doctoral programmes. With these funds, co-operation between universities within Switzerland are fostered and strengthened. By

developing joint programmes or individual modules (e.g. PhD colloquia, summer schools), the cooper-ating partners widen the research environment of their PhD candidates, offering expertise from and exchange with junior and senior researchers from other universities and with international guests. For research areas with few junior researchers at one university, this funding scheme allows for reaching a critical mass to create a stimulating research envi-ronment.

26. Research integrity (University of Freiburg) Ensuring academic integrity is a matter of central

importance for the University of Freiburg. In October 2014, the University created a post for a Vice-Rector for research integrity, gender and diversity. In 2011, the University passed new regulations on safeguarding academic integrity that oblige the faculties and research centres to familiarise their students and junior researchers with the rules of good academic practice and warn them against academic misconduct. In addition, the University has a representative for academic self-regulation, who advises those bringing forward an alleged case of academic misconduct as well as those accused of academic misconduct. Moreover, there is an investigative commission for safeguarding academic integrity.

27. The framework for doctoral degree regulations rat-ified on 25 March 2015 requires the faculties to en-sure that every doctoral researcher is familiar with the rules of good academic practice. The Interna-tional Graduate Academy (IGA) holds workshops on good academic practice in German and in English each semester – the latter for doctoral candidates in the life and natural sciences as well as engineering fields. By 2016 the University expects to have trained at least two staff members to hold these workshops in-house. In the case of conflicts resulting from the relationship between doctoral candidates and their supervisors or work on the dissertation, the Univer-sity of Freiburg has introduced a central two-level procedure for investigating complaints, whose guid-ing principles are confidentiality, transparency, and fairness6. The procedure is free of charge and can be broken off at the request of the person seeking advice at any time. It is structured as follows: As a rule, su-pervisors and doctoral candidates seeking advice and support in (emerging) conflicts should begin by con-tacting the ombuds assistants or ombudspersons at the IGA. If the conflict can be resolved on the basis of discussion, the process is terminated. If not, at least

6 See http://www.frs.uni-freiburg.de/ombudsstelle-en

12


one of the ombudspersons is called in to moderate a discussion between the parties. At the second level, the ombudspersons attend the discussion between the parties and help them to find and implement a solution. If this discussion does not lead to an amica-ble resolution or if the process is terminated because the person seeking advice does not consent to have other persons contacted, the ombudspersons may make recommendations to the parties.

1d) Learning outcomes and assessment

28. The objective of the doctorate is to train “creative, critical, autonomous intellectual risk takers” (LERU, 2010). The key outcome is that candidates demon-strate the ability to generate a substantial, original ‘contribution to knowledge’. This is the common point of all doctoral degrees. Candidates must also demonstrate discipline-specific and generic research skills and other generic skills to a high degree of so-phistication. For example, they must be able to con-vey complex ideas to international experts, to their peers and often, increasingly, to general audiences.

29. The doctorate is examined by a panel of international experts. The panel must have experience of exam-ining doctorates involving new members where ap-propriate to gain experience. Panel sizes vary from country to country. In many countries the examina-tion takes place in public enabling the audience to see the scientific debate and scrutiny in action. In some the exam is held privately, enabling a more de-tailed discussion.

30. Learning outcomes and assessment (University Col-lege London)7

The learning outcomes against which students are assessed are as follows: the examiners confirm that they have satisfied themselves that the candidate, as evidenced by the thesis and the exam, can communi-cate with the scholarly community about his or her areas of expertise. The examiners report that they have satisfied themselves that the thesis: • is genuinely the work of the candidate, • forms a distinct and significant contribution to

knowledge of the subject, • affords evidence of originality: by the discovery of

new facts and/or by the exercise of independent critical power,

• is an integrated whole and presents a coherent ar-gument,

• gives a critical assessment of the relevant literature, • gives the method of research and its findings, • gives discussion of those findings and how they

advance the study of the subject, • demonstrates deep and synoptic understanding of the

field of study, including objectivity, autonomy and the capacity for judgement in a complex situation,

• is satisfactory as regards literary presentation, • includes a satisfactory bibliography and references, • demonstrates research skills relevant to the thesis, • is of a standard to merit publication in whole, in

part or in revised form.

31. The defence is held in private by two examiners, one external to the University and one internal. The ex-aminers must be independent of the candidate and have had no involvement in the project or collaborate with the supervisor or candidate to ensure there is no conflict of interest. The supervisor may be present but only if permitted to be there by the candidate.

Step 2: Scrutiny processes

32. Institutions have a range of processes to scrutinise the effectiveness of the procedures in place and the environment in which they take place. The scrutiny processes are there to ensure that the expectations for doctoral education of the university are being delivered consistently and effectively. There should be periodic independent scrutiny a) of the depart-ment or faculty and its processes and b) of individu-al candidates and their supervisory team.

33. Since doctoral education needs to take place in a strong research environment, it is common that review processes consider doctoral training along-side the research environment. Sometimes the re-views consider all the activities of a unit together, particularly where the taught programmes are re-search-based, but practice does vary.

7 Key Links:

Academic Regulations for Research Degrees - Section 2 Programme of Study: http://www.ucl.ac.uk/srs/academic-regulations/res-deg/rd-sec-2

Guidance for Students, Staff and Examiners: http://www.ucl.ac.uk/srs/academic-regulations/res-deg/guidance

Research Student Surveys: http://www.grad.ucl.ac.uk/survey

Departmental Staff-Student Consultative Committees: http://www.ucl.ac.uk/academic-manual/part-7/sscc

13


2a) Scrutiny of universities and departments

34. Precept and periodic reviews (Imperial College Lon-don)

Imperial College assures the quality and standards of its research degree programmes by carrying out ‘Pre-cept Reviews’ and ‘Periodic Reviews’ of its Depart-ments8. Precept Reviews are an internal peer review exercise. The College’s Research Degree Precepts allow departments to have a degree of latitude in implementing and reflecting on their own research environments, but also ensure that there is consis-tency of standards in doctoral education across the College. The Precepts are aligned with the QAA’s UK Quality Code for Higher Education9. Each depart-ment is reviewed every three years.

35. Periodic Reviews take place every five to six years. They include a one-day visit to the Department where a pan-el of reviewers, including two external assessors, meet relevant students and staff, and report their findings.

36. Reports from the Precept Reviews and Periodic Reviews are considered by the Postgraduate Re-search Quality Committee (PRQC), which makes recommendations and highlights instances of good practice for dissemination across the College. The findings of the PRQC are reported to the College Senate for endorsement.

37. A quality system for doctoral studies (Universität Heidelberg)

Under the umbrella of its quality management sys-tem heiQUALITY, the Universität Heidelberg is cur-rently establishing a quality assurance system for doctoral education based on eight university-wide quality standards. These standards cover transparen-cy of admission and assessment processes, tailored recruitment and marketing measures for attracting highly qualified young researchers, optimal super-vision during the doctoral phase, as well as oppor-tunities for early scientific independence and career development. The quality process is based on data centrally collected through the new online doctoral file and through surveys. On this basis, the twelve faculties develop strategic plans on how to meet the quality standards, which are assessed by a commit-tee consisting of representatives of the Council for Graduate Studies, the Graduate Academy and the Senate. The Rectorate takes the decision to allocate central funding to develop supporting measures and structures within the faculties.

2b) Scrutiny of candidate progress

38. The individual study plan - A tool for planning, monitoring and progression of the doctoral candi-date’s graduate education (Lund University)

In Sweden it is mandatory according to the Higher Education Ordinance act to ensure that an individual study plan is made for each doctoral candidate. This plan shall contain the undertakings made by the candidate and the higher education institution and a timetable for the doctoral candidate’s studies. The plan shall be adopted after consultation with the doctoral candidate and his or her supervisors. Lund University has adopted regulations for how the individual curriculum for each doctoral candidate should be outlined. The individual study plan at LU must include the following:• the undertakings made by the student and the uni-

versity and a timetable for the student’s four-year programme,

• the subject and the general syllabus for the inten-ded degree,

• details of the planned funding of the doctoral can-didate’s studies,

• details of how the doctoral candidate’s supervi-sion is to be organised,

• what other resources are available to the doctoral candidate (laboratory, resources for traveling to a conference, for example),

• details of the planned compulsory and elective courses and other examined components that are to be included in the programme,

• details of the doctoral candidate’s participation in in-ternational activities; doctoral candidates are encour-aged to initiate international activities/contacts,

• information on any licentiate degree and/or mid-way review included in the programme,

• other information necessary for the efficient pur-suit of studies,

• details of any departmental duties and their scope and relation to the programme,

• how any costly components will be funded, taking into consideration that the doctoral candidate shall be able to complete his or her studies without incur-ring any unnecessary or unreasonable expenses,

• whether credits can be obtained for any training in teaching and learning in higher education – teach-ing students at lower levels.

39. The individual study plan is reviewed regularly - at least once a year - and amended as needed after con-sultation with the doctoral candidate and his or her

8 See https://www.imperial.ac.uk/about/governance/academic-governance/

9 See http://www.qaa.ac.uk/en

14


supervisors. A project is now aiming at implement-ing an electronic tool for the study plans, common for the University as a whole.

40. Teaching and Supervision Agreement (Utrecht Uni-versity)

The Teaching and Supervison Agreement (TSA) in-cludes the following elements at a minimum:

Supervision: The TSA sets out who the PhD candi-date’s principal and day-to-day supervisors are and the nature and extent of the supervision. If applica-ble, it also sets out agreements on reporting require-ments and evaluations.

Maximum teaching requirement: Full-time PhD candidates may be expected to carry out teaching work. The TSA sets out the maximum percentage of working hours that may be allocated to teaching.

Study programme: The TSA sets out agreements on the number and nature of programmes/courses to be followed. It also indicates whether these are required or optional.

Academic community: doctoral candidates need to have the opportunity to regularly attend internal and external academic events (seminars, conferences, etc). The TSA sets out agreements on this.

Career development: Given that many PhD candi-dates find employment outside of academia, it is important to spend time during the programme on career development through training courses and in-dividual support. The TSA sets out specific activities related to career development support.

41. Graduate Supervision System (University of Oxford) The Graduate Supervision System (GSS) is a web-

based tool which enables students and supervisors to submit termly supervision reports on graduate students’ progress. Directors of Graduate Studies in departments and faculties can use these reports to ensure that students are progressing satisfactorily and receiving suitable supervision to meet course requirements. They can also submit comments on reports. Students’ college advisors and administrators may view the reports to assist with college pastoral care and support. Reporting is mandatory for supervisors but voluntary for students, who are encouraged to contribute to the termly reporting cycle by making a self-assessment report about their own academic progress in the two-week period preceding the supervisors’ reporting period. The GSS includes a number of useful features, including automated email reminders to users about reporting periods and reports submitted, easy retrieval of previous terms’ reports, capacity for saving draft reports for later editing before

submission, and access from any where any time with support for industry-standard browsers and internet connections. The GSS also allows supervisors or students to flag a report if they have concerns about academic progress. This flag is communicated to the Director of Graduate Studies and college advisor for follow-up.

42. Termly reporting by supervisors has long been a pol-icy requirement at Oxford. The move to a web-based system supports quality assurance and enables in-stitutional oversight of the process as well as local monitoring. The system also enhances previous ar-rangements by providing students with the oppor-tunity for reflection, identification of training needs and feedback on the support they are receiving.

Step 3: Measurement

43. To assist in the scrutiny of the QA processes it is nec-essary to have a number of measurements to help judge the success of a programme and to benchmark against other institutions. National agencies also find common measures useful to judge the overall effectiveness of the doctoral education within their higher education system.

44. What is success and what should be the measures? Rigour, quality of research output, value of doctors to society, cost effectiveness, optimal development of human potential are some of the success crite-ria. Suitable measures will vary depending on na-tional needs but common measures for judging the effectiveness of programmes are: average times to completion, failure rates, destinations of doctoral graduates, quality and number of research outputs, average number of candidates per supervisor. For each candidate the measures also include: scientific productivity (reports, data and papers), timeliness of reporting, time to completion, external presenta-tions, and amount of generic skills training.

45. Indicators of success (Pierre & Marie Curie University, Paris)

UPMC has implemented indicators for doctoral edu-cation within departments, which mostly follow the request of the national evaluation agency. The indi-cators include time to completion, employment rate of doctoral graduates, average number of candidates per supervisor, as well as volumetric indicators, such as the average number of doctorates defended in the

15


last three years and the number of active supervisors that have supervised a doctorate in the last four years.

46. In addition, UPMC evaluates newly graduated doc-

torate holders’ satisfaction, both in relation to their doctoral education and to their present job, and whether their present job meets their expectations. The employment and satisfaction survey is now carried out at regional level and at different higher education institutions, thus giving a precise view of the employment of doctorate holders. The ques-tionnaires are sent out every year for three years fol-lowing graduation. A major outcome of this survey is the confirmation of previous surveys, proving that satisfaction in the employment after the doctorate is strongly correlated with the development of a profes-sional career plan prior to the defence. Moreover, it has been found that the development of a profession-al career plan prior to the defence strongly reduces the unemployment rate three years after graduation (from 3% to 1%).

47. Faculty strategic plans and indicators of success (University College London)

Faculties report annually to the Doctoral Training Strategy Committee on their strategy. To promote careful self-analysis the Doctoral School assembles a basket of indicators on the doctoral training en-vironment, which are considered by the faculties to inform their future plans. The strategies are reviewed to ensure they are in line with UCL’s research strate-gy and doctoral training expectations. Indicators in-clude: staff/student ratio, four-year submission rates

(seven-year for part-time candidates), candidates’ satis faction rating with their supervision and research environment, application and acceptance data, research student log usage, formal complaints, and career destination information. Submission rates are scrutinised annually by the Research Degrees Committee prompting actions where rates are not satisfactory.

48. Quality measures (Universitat de Barcelona) The research work of the doctoral candidate is subject

to the academic quality and QA system established by the UB Doctoral School. Quality is measured through the satisfaction of all parties involved (researchers in training, academics and administrative staff ) and the effects of all administrative procedures: access and admission to a doctoral programme, the defence and the publication of the thesis, and the issue of the title. QA processes consider information, rights and obligations, opinions, suggestions and participation rules, as well as obtaining the necessary data to de-velop relevant indicators.

49. To analyse the level of satisfaction of the doctoral candidates, two types of surveys are performed: a yearly one included in the monitoring report, and another one after submission of the thesis. These are reviewed regularly by the Quality Agency of UB and the Doctoral School.

50. To analyse the degree of satisfaction of tutors and su-pervisors, a survey is conducted every five years. Ques-tions cover the training offered by UB for doctoral students, the administrative procedures and the meas-ures needed to ensure the improvement of the devel-opment of the doctoral programme. The five-year survey is first evaluated by the director of UB Doctoral School, who issues a report to the Steering Committee of the School. Each programme coordinator receives the corresponding reports and is prompted by the Doctoral School to introduce improvement measures.

Step 4: Feedback mechanisms and quality enhancement

51. Once activities and processes have been independent-ly scrutinised, both qualitatively and using available measurements, mechanisms should be in place to feedback key messages. This should result in plans for action and subsequent reviews to ensure improve-ments are made. These messages are made to the can-didates themselves but also to supervisors on their ap-proach to a particular project and in general, as well as to departments to ensure that ways of enhancing the research culture are identified and acted upon, and that processes are efficient and effective. Processes should be in place to review and enhance the culture and framework of research across a university if it wishes to be a research-intensive organisation.

52. Supervisor training, surveys and discussion (KU Leuven)

To improve the quality of supervision, new principal investigators (PIs) are given an introductory course composed of three parts: 1) regulations and proce-dures, scientific integrity and the Doctoral School, 2) management and leadership, and 3) HR skills for recruitment and supervision in professional deve-lopment of doctoral students.

53. Besides the evaluation of the scientific development of the doctoral student, both advisor and student are asked at several moments during the doctorate to fill out a short survey about the PhD-advisor relation. If problems are identified, the ombudsperson will con-

16


tact the student proactively. If a supervisor recurrent-ly has problems with his/her students, he/she will be invited to discuss a plan to remediate the problems.

54. Functioning of the departments (including their doctoral training and research environment) in Biomedical Sciences is evaluated at several levels. Heads of departments meet on a monthly basis with the Vice-Rector to discuss current affairs, including those related to doctoral candidates. The Doctoral School is also represented. In this way any issues arising from the case of a particular candidate re-flecting a general issue are addressed at a higher level. Departmental boards also discuss these issues. They require the inclusion of a representative from doctoral candidates and postdocs, which makes it a good platform to discuss issues of all sorts.

55. Annual registration and feedback meeting (Univer-sity of Strasbourg)

In France, the official duration of a PhD is three years. At Strasbourg, registration of PhD students is done through a website and can be done without any direct contact between the student and the head of the research institute and/or the head of the PhD School (Ecole doctorale). To promote personal inter-action and to ensure that the head of the institute is aware of the PhD work going on under his or her general responsibility, several feedback meetings are organised by the Ecole Doctorale in Earth Sciences and Environment, as described below.

56. At the first registration (first year), the head of the PhD school meets each student for 20-30 minutes, explaining the roles of the supervisor, the head of the institute and of the PhD school.

57. By the end of the first year, the student has to give an oral presentation of his/her achievements in front of a jury, composed of representatives of each institute connected to the Doctoral School; it does not include the supervisor. The jury may give recommendations to both the student and the PhD supervisor. This event is mandatory for the next registration.

58. After the second year, the student has to give anoth-er presentation in front of a jury. The objective is to anticipate sufficiently in advance a potential need for an extra fourth year of training, giving the supervi-sor time to look for funding (only funded students are authorised to register). This presentation is also mandatory for the next registration. The jury has the same composition as the previous one.

59. At the fourth registration, the student is interviewed by the head of the doctoral school. It is clearly ex-plained to the student that this will be his/her last registration.

60. The doctoral candidates’ voice (Universität Heidel-berg)

Doctoral candidates in Germany form a diverse group with different status and rights, pertaining to students, research associates with contract or with-out contract. Up till now, their representation within the Universität Heidelberg has been minimal and difficult. As a status group, they have been represent-ed in the Council for Graduate Studies at Heidelberg with four seats since 2013. With the set-up of a Doc-toral Council in autumn 2015, doctoral candidates have a voice within the University – an important step to overall quality enhancement through insti-tutionalised participation. Feedback mechanisms are currently being established to hear doctoral can-didates on issues debated in the Council for Gradu-ate Studies, on the development of faculties’ degree regulations, as well as on issues of quality assurance within the quality management process.

17


61. Doctoral education must take place where there is a high-quality research culture. To be sure this culture is maintained, robust and rigorous quality assurance processes are needed. Our analysis of quality assur-ance at LERU universities shows that a variety of comprehensive doctoral QA processes are in place. We have examined these processes in four dimen-sions, as steps in a virtuous circle, going from de-fining expectations, to setting up scrutiny processes that explore whether expectations are met, to meas-uring key quality indicators, and to providing feed-back mechanisms that facilitate both correction and enhancement of the system.

62. QA at LERU universities shows a diversity of ap-proaches, responding to different institutional and national/regional contexts. The different examples show the extent to which doctoral QA serves to fulfil structural and administrative requirements, to en-hance the quality of the doctoral research training programme (individual or structured), and to assess the quality of the output.

63. To achieve and maintain a rich research culture, the environment must be independently scrutinised to ensure it meets its stated objectives. Independence can usually be achieved by reviewers from within the same university, but with some external representa-tion to ensure that international standards are main-tained and that the expectations of national agencies are met. QA processes should be easily accessible and transparent.

64. We recommend that universities wishing to train excellent doctoral candidates, should have a QA sys-tem in place which takes into account these points. Our recommendations to universities are listed at the beginning of this paper. We hope that the examples from LERU universities included in this paper help to demonstrate and disseminate good practice among universities in Europe and beyond.

65. While universities are our main target audience for this paper, we wish this paper to signal to others, in particular policy makers and research funders, that quality assurance plays an important role in main-taining a high quality culture, that the quality of the research environment as well as the quality of the

Conclusions and recommendations

outputs is considered, that reviews of programmes and of individual doctoral outcomes should be re-viewed, with independent reviews in some situa-tions, and that institutions that have good QA sys-tems in place should be acknowledged, encouraged and rewarded.

66. While direct responsibility for doctoral QA lies with universities overseen by national or devolved autho-rities, it is clearly also a European issue, with doc-toral education recognised as the third cycle in the Bologna Process and as a crucial part of the EU’s ERA policy and funding programmes. Therefore, the EU can have a contributing and supportive role, by bringing actors together from across Europe and en-couraging exchange of good practice, thus helping universities to develop and maintain good QA. This will help to foster the take-up and implementation of the Innovative Doctoral Training principles pro-moted by the EU. The EU should not aim to harmo-nise or standardise, let alone regulate or prescribe doctoral QA, neither in its policy making nor in its funding programmes. It is clear from the examples given in this report that there are varied practices that successfully achieve high quality doctoral education within a vigorous research culture and these must not be stifled.

18


Bologna Follow-up Group (BFUG). 2008. Introduction to third cycle (doctoral) studies as part of the tuning process. http://www.unideusto.org/tuningeu/tuning-3rd-cycle/introduction.html

Coimbra Group. 2007. Doctoral process position paper. http://www.coimbra-group.eu/DOCUMENTS/Doctoral%20Programmes%20CG%20Position%20Paper.pdf

EC (European Commission). 2014. Researchers’ report 2014. http://ec.europa.eu/euraxess/pdf/research_policies/Researchers%20Report%202014_FINAL%20REPORT.pdf

EC (European Commission). 2011. Principles for Innovative Doctoral Training. http://ec.europa.eu/euraxess/pdf/research_policies/Principles_for_Innovative_Doctoral_Training.pdf

EC (European Commission). 2009. Preparing Europe for a new Renaissance: A strategic view of the European Research Area. First Report of the European Research Area Board.

https://ec.europa.eu/research/erab/pdf/erab-first-annual-report-06102009_en.pdf

ENQA (European Association for Quality Assurance in Higher Education). 2009. Standards and Guidelines for Quality Assurance in the EHEA (European Higher Education Area).

http://www.enqa.eu/wp-content/uploads/2013/06/ESG_3edition-2.pdf

EUA (European Universities Association). 2013. Quality assurance in doctoral education – results of the ARDE (Accountable Research Environments for Doctoral Education) project.

http://www.eua.be/Libraries/publications-homepage-list/EUA_ARDE_Publication

EUA (European Universities Association). 2011. Salzburg II recommendations: European universities’ achievements since 2005 in implementing the Salzburg Principles.

http://www.eua.be/cde/publications.aspx

EUA (European Universities Association). 2005. Bologna Seminar on “Doctoral Programmes for the European Knowledge Society” (Salz-burg Principles).

http://www.eua.be/eua/jsp/en/upload/Salzburg_Conclusions.1108990538850.pdf

French Ministery for Education and Research. 2006. Arrêté du 7 août 2006 relatif à la formation doctorale. Journal Officiel de la République Française, texte 22 sur 104.

French Ministery for Education and Research. 1998. Sanction des études - Charte des thèses. Bulletin Officiel de l’Education Nationale N°36 du 1er octobre 1998.

http://www.education.gouv.fr/bo/1998/36/sup.htm

Group of Eight (Go8). 2013. The changing PhD – Discussion paper. https://go8.edu.au/sites/default/files/docs/the-changing-phd_final.pdf

LERU (League of European Research Universities). 2014a. Good practice elements in doctoral training. http://www.leru.org/files/publications/LERU_AP_15_Good_practice_elements_in_doctoral_training_2014.pdf

LERU (League of European Research Universities). 2014b. An ERA of change: LERU briefing paper for the EU institutions. http://www.leru.org/files/publications/BP_ERAOFCHANGE_FINAL.pdf

References

19

advice paper - no. 19, March 2016

LERU (League of European Research Universities). 2013. Hefei statement on the ten characteristics of contemporary research universities announced by AAU, LERU, Go8 and C9.

http://www.leru.org/files/publications/Hefei_statement.pdf

LERU (League of European Research Universities). 2010. Doctoral degrees beyond 2010: Training talented researchers for society. http://www.leru.org/files/publications/LERU_Doctoral_degrees_beyond_2010.pdf

LERU (League of European Research Universities). 2008. What are universities for? http://www.leru.org/files/publications/What_are_universities_for_September_2008.pdf

LERU (League of European Research Universities). 2007. Doctoral studies in Europe: Excellence in researcher training. http://www.leru.org/files/general/%E2%80%A2Doctoral%20Studies%20in%20Europe%20Excellence%20in%20

Researcher%20Training%20(May%202007).pdf

NFU (Nederlandse Federatie van Universitair Medische Centra). 2011. NFU guidelines for PhD tracks in biomedical sciences in the Netherlands.

http://www.nfu.nl/img/pdf/11.14200_NFU_Guidelines_for_PhD_tracks_-_nov_2011.pdf

Orpheus, AMSE, WFME. 2012. Standards for PhD Education in Biomedicine and Health Sciences in Europe. http://www.zmmk.uni-koeln.de/e1241/e3287/e3770/e3771/ORPHEUS-AMSE-WFME-standards-for-PhD-education.pdf

UK Quality Assurance Agency. 2013. The UK Quality Code for Higher Education, Part B, Chapter B11 Research Degrees. http://www.qaa.ac.uk/assuring-standards-and-quality/the-quality-code/quality-code-part-b

VSNU (Vereniging van Samenwerkende Nederlandse Universiteiten). 2004. Hora est! - Reforming the Research Training System. http://www.vsnu.nl/files/documenten/Domeinen/Onderzoek/2013%20PHD%20conference/Positionpaper_Hora_

est!.pdf

20


Appendix A: Quality culture in practice at LERU universities

Step 1: Expectations and principles

1a) Institutional oversight of the quality of doctoral education

A legal framework (University of Freiburg)

New regulations for doctoral examination procedures through the new State Higher Education Act of Baden-Wurttem-berg (LHG) were brought in on 1 April 2014. These include:1. the decision to accept a doctoral researcher to be made by a committee2. an obligation to draft dissertation supervision agreements with the following minimum content:

· schedules for regular advising sessions and progress reports tailored to the dissertation project and the life situation of the doctoral candidate, including provisions for updating them as needed

· individualised study programme· statement obligating both parties to observe the rules of good academic practice· regulations for resolving disputes· timeframe for grading the dissertation upon its submission

3. registration of all doctoral candidates after they have signed a supervision agreement4. inclusion of so-called external doctoral researchers5. establishment of doctoral researcher conventions6. appointment of ombudspersons for doctoral researchers and dissertation supervisors

A University framework for doctoral degree regulations was ratified on 25th of March 2015, including:1. possibility for highly qualified graduates (top 5%) of three-year Bachelor’s programmes to enter the doctoral pro-

gramme (fast track)2. right to supervise dissertation projects for non-habilitated leaders of junior research groups under certain conditions3. provision that the doctoral degree regulations of the faculties stipulate periods of time in which they control

whether the dissertation project can be pursued or not4. conditions for cumulative dissertations: (1) several related academic papers (2) combined to form a single work.

These papers (3) must address a common research question and (4) must have been published or be accepted for publication in internationally recognised reviewed journals; (5) the doctoral candidate must have made a funda-mental contribution to at least one of the papers; (6) none of the submitted papers may be the topic of another dissertation in an ongoing or completed doctoral examination procedure; (7) the dissertation must include an in-depth introduction including a critical assessment of the research topics and the most important findings from the publications in the context of the academic literature on the topic and, if applicable, (8) recognition of the doctoral candidate’s own contribution to the research as well as the contributions made by other authors of the individual publications

5. sworn statement attesting that the dissertation is the result of independent research6. faculties are responsible for ensuring that doctoral candidates are familiar with the rules of good academic practice7. possibility for joint doctoral degrees

A national system for accreditation and assessment (University of Milan)

Italian doctoral courses have been recently reformed by Decree of the Ministry of Education, University and Research n. 45 of February 8, 2013. Particularly, the Ministerial Decree has introduced a system for accreditation of doctoral courses.The accreditation system consists of an initial approval subject to the existence of specific requirements as reported below, and of periodic assessment to ensure requirements laid out by the initial accreditation are carried out:- a doctoral board consisting of academic staff with international research record in the disciplinary areas of the

doctoral course;- a minimum number of scholarships to be awarded for each doctoral course;- appropriate and stable financing ensuring sustainability towards research carried out in the doctoral course within

21


which doctoral students are enrolled;- availability of relevant qualified administrative and research facilities;- doctoral training, also jointly conducted with other doctoral courses, which can either be disciplinary or interdis-

ciplinary, specialised or transversal.

The regulations issued in 2013 introduced a further novelty in the national HE system. An assessment system was es-tablished according to which each university is assigned specific funds by the Ministry for sustaining doctoral courses. Ministerial assessment, carried out by the national evaluation agency ANVUR, takes into account the following criteria:- quality of research carried out by the members of the doctoral board;- level of internationalisation of the doctoral course;- level of cooperation with enterprises and impact of the doctoral course on the socio-economic system;- attractiveness of the doctoral course;- services, infrastructures and financial resources available to the doctoral course as well as to doctoral students, also

as a result of a merger or federation of higher education institutions;- employability of PhD graduates.

Assessment carried out in accordance with specific methodologies set out by ANVUR will produce a national ranking of doctoral courses in disciplinary areas based on certain indicators of the above criteria. Within the Italian accred-itation and assessment systems, each university’s assessment unit plays an effective role consisting in monitoring doctoral activities jointly with ANVUR.

Bearing in mind national rules on accreditation and assessment, the University of Milan has established its own crite-ria to assess its academic staff ’s eligibility to supervise in doctoral programmes. Specifically, the University has defined thresholds for research carried out within three years preceding a targeted year. Then, as for resources either deriving from the Ministery or from own resources, the University assigns funds to doctoral courses through assessment pro-cedures which take into account efficient and effective results produced by each doctoral course.

Managing research degrees (University College London)

Doctoral education is overseen by the Research Degrees Committee (RDC) which has academic representation from all Faculties, relevant members of the administration, and student representation. Doctoral education is delivered in line with a set of Regulations which state formal requirements for the award of doctoral degrees. UCL’s expectations and entitlements of students and expectations of supervisors (primary and subsidiary) and graduate tutors are set out in the Code of Practice for Research Degrees distributed to all new students as they arrive and to staff. There is also a set of guidelines for good supervisory practice.

The Doctoral School maintains a website which contains all information on doctoral education or links to key sites as a first port of call on doctoral education at UCL for staff, students, and people outside UCL10.

Curricula and their preparation (University of Helsinki)

Key actors in the planning of doctoral education at the Faculty of Arts include the vice-dean in charge of postgraduate education, the directors of the doctoral programmes under the Faculty’s responsibility, department heads, discipline coordinators and professors as well as the administrative staff of postgraduate affairs (the faculty head of research af-fairs, the planning officer and academic affairs coordinator for postgraduate studies as well as the coordinator for the

10 Key Links:

Academic Manual Part 7 - Academic quality review, monitoring and feedback framework (including IQR Procedures): http://www.ucl.ac.uk/aca-

demic-manual/part-7

UCL Quality Management and Enhancement: http://www.ucl.ac.uk/srs/governance-and-committees/qme

Quality Assurance Agency for Higher Education ‘UK Quality Code for Higher Education Chapter B11 Research Degrees: www.qaa.ac.uk/en/publi-

cations/documents/quality-code-B11.pdf

Code of Practice for Graduate Research Degrees: http://www.grad.ucl.ac.uk/codes/CoP_Research_1314.pdf

Graduate Admissions Policy: UCL Academic Manual: http://www.ucl.ac.uk/academic-manual/part-4/graduate-admissions

Graduate Student Recruitment and Admissions Good Practice: http://www.ucl.ac.uk/academic-manual/part-4/pg-recruitment-admissions

22


doctoral programmes). Key decision-making and preparatory organs are the Faculty Council and the Committee for Postgraduate Studies, department councils as well as the management groups of the doctoral programmes under the Faculty’s responsibility. The Ethics Committee and the Research Committee discuss postgraduate affairs, if necessary. As the operations of the doctoral programmes are still being launched, their role in the quality management of the planning of education is still being specified.

The vice-dean in charge of postgraduate education (the chair of the Committee for Postgraduate Studies) and the Com-mittee for Postgraduate Studies, discipline coordinators and department councils as well as the Faculty Council, which decides on degree requirements, participate in the design of the postgraduate curriculum. At the Faculty Office, the preparation of the curricula is the responsibility of the planning officer for postgraduate studies (the secretary of the Committee for Postgraduate Studies) and the faculty head of research affairs (the supervisor of postgraduate affairs). From the beginning of 2014, curriculum preparation will be implemented together with the management groups and coordinators of the doctoral programmes under the Faculty’s responsibility and the Doctoral School in Humanities and Social Sciences. The Faculty will develop the curriculum in conjunction with strategy work and on the basis of feedback received from evaluations of doctoral education.

The Faculty prepares the degree requirements of postgraduate studies with the general objectives of the degree in mind so that they are clear and flexible. In this way, the student can, together with his or her supervisor, create a meaningful degree entity in terms of the research topic and individual objectives. The Faculty revises the degree requirements every other year in conjunction with the updating of the degree requirements of first-cycle and second-cycle education. The Committee for Postgraduate Studies and the Faculty Office’s postgraduate affairs unit prepare guidelines for the revising of the degree requirements, including recommendations for preparing the descriptions of study modules and learning outcomes. The disciplines prepare the necessary corrections to the study modules and learning outcomes, and prepare descriptions of the content and methods of completion of studies. The Faculty Council approves the de-gree requirements at the proposals of department councils.

Oversight and change in doctoral education (LMU Munich)

Traditionally, at LMU Munich, governance of doctoral education is the responsibility of faculties and departments. However, since doctoral education is an important and crosscutting topic, the GraduateCenterLMU has been established as the central unit for all matters related to doctoral studies. With its coordinating function and services it supports various processes to optimise the conditions for all doctoral candidates at the university. Such a central, coordinating institution guarantees constant dialogue and information flow between all parties involved in doctoral education. Gen-eral regulations, guidelines and recommendations are developed in coordinated discussions with all stakeholders on a local and institutional level (bottom-up). The implementation is then steered centrally (top-down). The resulting com-munication loop propagates good practice within the institution and enhances overall quality of doctoral education.

LMU Munich has revised and published two documents on the organisation of doctoral studies in 2011: ‘Recommen-dations for the organisation of doctoral studies at LMU Munich’11 and ‘Recommendations for the arrangement of doctoral programmes at LMU’. The commitment for good scientific practice, supervision and handling of scientific misconduct (“Richtlinien der Ludwig-Maximilians-Universität München zur Selbstkontrolle in der Wissenschaft”) has been Uni-versity policy since 2002.

1b) Recruitment, promotion, marketing, admission

Candidate selection (Utrecht University)

Dutch law specifies admission requirements which apply to all PhD candidates (art. 7.18 WHW). Each of Utrecht’s seven Graduate Schools may establish additional general admission requirements which apply to all doctoral programmes within that Graduate School. Individual doctoral programmes can also set out specific admission requirements.

11 http://www.en.graduatecenter.uni-muenchen.de/phd_studies/supervision/recommendations.pdf

23


The Board for the Conferral of Doctoral Degrees checks whether the PhD candidate meets the general admission re-quirements (diplomas), and appoints the supervisor and co-supervisor(s) upon admission of the PhD candidate to the doctoral programme.

Recruitment processes (University of Zurich)

The Life Science Zurich Graduate School (LSZGS) organises a centralised recruitment process for its 16 PhD pro-grammes, thereby making use of synergies and increasing the international outreach of the school to attract highly qualified PhD candidates to Zurich. There are two application deadlines per year. The recruitment process consists of an online application, a paper-based pre-assessment and a three-day interview round in Zurich with final selection of candidates.

Currently, the LSZGS is undertaking to improve the final assessment phase (i.e. interviews) by looking in depth into the individual steps with the aim of defining strategies and (standardised) guidelines for the interviews with the candidates to enhance transparency, clarify the selection criteria and assure the validity of the final decisions.

Entry into structured programmes (University of Freiburg)

The University of Freiburg offers a very large and growing number of structured doctoral programmes in order to offer excellent research and training conditions for doctoral researchers. Now approximately one-third of all docto-ral researchers are writing their dissertations within structured programmes12. Applications for initial and continued funding for these projects, many of which are externally funded, are reviewed by external experts. They generally have a two-tiered selection procedure. The Spemann Graduate School of Biology and Medicine (SGBM) offers a good-practice example of such a procedure. The school goes through all written applications and adds the candidates it wishes to invite for an interview to a short list. Each candidate has one individual 30-minute interview in English with one of the three principal investigators mentioned in his or her application, as well as a tandem interview with two members of the selection committee (30 minutes). In addition, candidates present their previous research work (Diplom or Master’s) to the selection committee. This is a ten-minute presentation (without PowerPoint) followed by a ten-minute question round. Funding is awarded on the basis of the results of the interviews and the presentation to the selection committee.

Admission (University of Helsinki)

The groundwork for the high-quality implementation of postgraduate education is laid during the postgraduate ad-missions process, which the Faculty has been developing with perseverance. The process chart of postgraduate admis-sions has been updated in conjunction with the establishment of the doctoral programmes. Prospective students can apply for a postgraduate study place twice a year. The Faculty confirms the maximum annual intake of postgraduate students in conjunction with its target programme. In order to ensure the fluency of communications and studies the applicants must have sufficient discipline-specific language skills necessary for completing the doctoral dissertation and postgraduate studies as well as communication-level language skills in Finnish, Swedish or English. The evalua-tion of the applications is the duty of the doctoral programmes. In addition to the applications and the quality of the applicants’ research plans, the evaluations are made on the basis of statements from the discipline and the department and take into account the expertise of available supervision and the sufficiency of the discipline’s supervisory resour-ces. The decisions on admission are delegated by the Faculty Council to the Committee for Postgraduate Studies. The Faculty and the doctoral programmes together organise an orientation for the new postgraduate students so as to support the launching of studies and the integration of the students into the University.

Lifelong learning (University of Helsinki)

The degrees of Licentiate and Doctor of Philosophy at Helsinki can be completed in combination with other employment. Especially for language and other subject teachers pursuing postgraduate education alongside work

12 See http://www.iga.uni-freiburg.de/Promotionsprogramme

24


is a relevant form of continuing education and lifelong learning. Because the faculty may require the completion of supplementary studies from admitted postgraduate students, the need to update the scientific competence provided by a second-cycle degree does not in itself affect the applicant’s chances of being admitted. Work experience does not constitute an admissions criterion but can have a positive effect on the evaluation on the applicant’s ability to complete a doctorate and will effect induction processes to ensure that the new candidates are properly prepared. The flexible methods of completion of postgraduate studies make it possible to include prior learning in the degree. When the student submits his or her dissertation for the preliminary examination, the Faculty offers him or her the possibility to join its alumni activities, which provide especially those doctoral degree holders who will pursue a career outside the University a way of keeping up with the research and events in their own field. Making research seminars available not only to postgraduate students but also to postdoctoral researchers supports the lifelong learning of doctoral degree holders engaged in research work.

1c) Research environment, supervision and research integrity Training of supervisors and examiners (University College London)

UCL has three types of training events for supervisors. A mandatory briefing is given to all new academic staff (of all grades) who wish to supervise research students (in practice almost all new academic staff ). The two-hour briefing introduces key people in the management of doctoral education, outlines the main policies and procedures, introduces the Doctoral Skills Development Programme and the Research Student Log (the online project management tool which all research students must use), and gives some good practice from experienced supervisors. To be given permission to supervise staff must have attended this briefing and be judged to be research active through submitting a CV to the Faculty.

A one-day workshop introduces participants to the main issues faced by supervisors today. The workshop discusses the motivations behind research students, a session on ‘whose PhD?’, and explores the internal and external pressures on supervisors arising from both quality assurance and funding. The sessions allow discussion and sharing of good practice. It also involves discussing a number of case studies where matters have not gone well. It is particularly rel-evant to new supervisors. Some bespoke sessions are also run for Departments or groups of Departments to explore specific issues such as improving submission rates and improving motivation.

Sessions are also run on examining research doctorates. The session provides an overview of the main issues involved in examining research degrees successfully, together with an opportunity to consider case studies and discuss best prac-tice. There is a brief outline of the UCL regulations and processes, three experienced examiners share advice on how they proceed in the examination and how to deal with tricky issues, tabled regulations from other institutions are discussed, and finally three case studies are discussed in small groups. There is plenty of time to raise concerns and issues. British exams are closed to all but the candidate, the two examiners, and one supervisor if permitted by the candidate (practice varies across disciplines). The session does discuss public examination processes in different countries based on expe-rience of the experts. The session is particularly relevant to academics who have not examined previously.

Supervision (University of Helsinki)

In order to ensure the quality of the supervision of postgraduate students, the Faculty of Arts has developed common principles for the supervision of postgraduate studies that have been approved by the Committee for Postgraduate Studies (revised in 2011). They discuss• the stages, planning and overall supervision of postgraduate studies • the rights, obligations and roles of the postgraduate student and the supervisor as well as the distribution of res-

ponsibility for supervision• the good practices of a supervisory relationship and the starting points of good supervision as well as the integra-

tion of postgraduate students into the scientific community• possible problem situations and interventions for them.In addition, the principles include recommendations regarding the preparation and updating of the supervision plan. The Faculty reviews the principles about every five years. They are sent to all new postgraduate students with the letter

25


of acceptance. The template of the supervision plan approved by the Postgraduate Admissions Board in spring 2014 is available to enable doctoral programmes to edit it to meet specific needs which may vary between disciplines.

In order to ensure the quality of supervision, postgraduate admissions pay particular attention to situations in which the proposed supervisor already has more than ten active students in his or her supervision. In such cases, at minimum a second supervisor with less than ten active postgraduate students will be required. Limitations concerning the num-ber of students in supervision support the quality of supervision and the workload of supervisors. The Faculty advances the continuity of supervision with the requirement that at least one of the supervisors be part of the staff or at least a fixed-term employee hired for several years. In order to enhance the expertise of supervision, multidisciplinary disser-tation projects may apply for a second supervisor from outside the student’s own department.

Managing expectations: For the Faculty, the principles it has prepared for the supervision of postgraduate studies con-stitute one way of influencing the wellbeing of postgraduate students and supervisors. They also include guidelines for solving problem situations and information on whom to turn to if discussions between the supervisor and the student do not solve the situation. The job description of the faculty head of research administration includes providing advice in problem situations, and the postgraduate affairs staff support students in case of adversities related to the examina-tion process of the dissertation.

The wellbeing of postgraduate students is weakened by concern for funding and the inequality caused by different funding situations (affecting, for example, work facilities and teaching duties). The Faculty seeks to advance post-graduate students’ equal opportunities of participating in the competition for funding through the distribution of information and financial advice (information sessions). As regards office space, it has sought to solve the problem using office agreements concluded at departments and by increasing transparency (criteria). All postgraduate students belong to the Faculty’s email list and thus receive information on issues pertaining to postgraduate students regardless of their employment or office situation. The aim of the measures for integrating postgraduate students into the Facul-ty’s research community (including ensuring supervision and offering cooperative research seminars) is to advance the wellbeing of postgraduate students. The role of communality and individual supervision in the wellbeing of postgrad-uate students is all the more significant since they are not entitled to the services offered by the Finnish Student Health Service or the University’s counselling psychologists.

Teachers’ competence and occupational well-being: The teaching skills of applicants for teaching positions are evalu-ated by the Faculty’s Teaching Skills Committee. For the evaluation of teaching skills, the Faculty has together with the senior lecturers in university pedagogy and the Committee for Study Affairs prepared evaluation matrices for teaching qualifications and the demonstration of teaching skills. The Faculty’s research and support unit for learning and teach-ing supports the development of teacher competence by organising training in university pedagogy, seminars focusing on the supervision of theses, and discussions (the pedagogical café). These events offer teachers and supervisors the opportunity to receive peer support and to share supervision experiences and good practices. Especially in bigger dis-ciplines, supervisors support each other by, for instance, running research seminars in cooperation with a colleague. One aim of limiting the number of supervised students is to promote the wellbeing of teachers.

Professionalising research management (LMU Munich)

The GraduateCenterLMU promotes the development and implementation of new doctoral programmes at LMU Munich. This process is guided by the GraduateCenterLMU recommendations for the arrangement of doctoral programmes to ensure institutional quality standards. In addition, professors are supported in applying for nationally or internatio-nally funded doctoral programmes.

Because most doctoral programmes are interdisciplinary and interinstitutional, professionalisation in programme management and quality assurance is becoming increasingly important. The GraduateCenterLMU actively supports this by regularly inviting doctoral programme managers for networking activities, workshops and working groups. Thus a community of practice can develop on an institutional level.Since the number of structured doctoral programmes has significantly increased, conceptual ideas originally inherent to these programmes are more and more adopted to individual dissertation schemes. Team supervision, for example, has become common practice in many areas of the university.

26


To support the professionalisation of research management in general and to enlighten the importance of the pro-gramme coordinator’s role, the GraduateCenterLMU initiated a working group which discussed the activity profile of a doctoral programme coordinator at LMU Munich and jointly drafted a job description which makes the scope of activities and responsibilities more transparent13.

The online doctoral file - a tool for online registration, administration and communication for doctoral students (Universität Heidelberg)

As of the end of 2015, the Universität Heidelberg requires all doctoral candidates to register for the new heiDOCS online doctoral file. It portrays the progress of the registration and examination procedure as well as information on frame conditions and the situation of doctoral candidates throughout their doctoral education. In addition, the central university administration and quality management units, as well as the faculties’ offices can access the online doctoral file for administration and quality enhancement processes. Furthermore, it encompasses modules for networking and communication between the doctoral candidate and the Doctoral Council, the Graduate Academy and his/her respec-tive Faculty. Through the online doctoral file, doctoral students will receive information tailored to their status and needs, and be able to access all university services.

Links between research, development and innovation as well as artistic activities and education in the Humanities (University of Helsinki)

The core of postgraduate education consists of the student’s own research work. The starting point of postgraduate admissions is that postgraduate studies must be completed at a university and in a discipline able to provide sufficient supervision to the student on his or her research topic. This aims to enhance the integration of postgraduate students into the research activities of the discipline and the department as well as into the focus areas of research specified in the Humanities Faculty’s target programme. It is also the supervisor’s duty to ensure that the postgraduate student establishes contact with the research community. Traditionally, research in the humanities has been conducted by individuals, but in recent years, the Faculty has strongly focused on supporting cooperation between researchers and, in particular, the acquisition of research funding allocated to research groups. The increase in research cooperation and project funding also advances the integration of postgraduate students into the scientific community. The Faculty has sought to enhance postgraduate students’ opportunities to present their dissertation work and receive feedback at international conferences by allocating the majority of the Chancellor’s Travel Grants to doctoral students. Funding is granted primarily to those who have no chance of acquiring travel grants from elsewhere.

Research integrity and research ethics (University College London)

As part of its Research Governance Framework, UCL operates a Code of Conduct for Research setting out the general principles of conduct by which UCL expects all research to be carried out at or in the name of UCL. The UCL Code of Conduct in research covers five main areas: • professional and personal integrity of researchers • process of research design • publication process • leadership responsibilities • institutional responsibilities

The UCL Code of Conduct also usefully signposts researchers to the relevant associated UCL policies, for example the student IPR Policy, and Guidance for the Storage and Disposal of Data and Samples.

The other key element of UCL’s Research Governance Framework is UCL’s arrangements for investigating allegations of misconduct in academic research. Doctoral School Research Integrity web page for research students and supervi-sors at: http://www.grad.ucl.ac.uk/research-integrity/

13 See http://www.en.graduatecenter.uni-muenchen.de/about_us/range_of_services/coordinator/index.html

27


Standard Evaluation Protocol (Utrecht University)

The Standard Evaluation Protocol (SEP) serves as general minimum requirements for the quality of those aspects of the doctoral programme that are assessed as part of a SEP evaluation. This concerns:- objectives and organisational embedding- programme structure- supervision- study success figures- teaching resources (such as training, resources for conference visits)

Each graduate school provides a Teaching and Supervision Agreement (TSA) which PhD candidates and supervisors must complete. This sets outs agreements on issues such as courses to be taken, and supervision. This document can be used as the basis for progress meetings. For PhD candidates with an appointment at UU or University Medical Cen-tre Utrecht, the completion of this plan is linked to the appointment being effected by Human Resources.

Training and guidance plan for candidates (Leiden University)

The thesis advisors (minimum 2) and PhD candidate constitute the daily Supervising Group and develop, sign and upload into the Graduate School Management System (GSM) a Training and Guidance Plan (TGP) describing the Re-search (Project) and Educational plan. The TGP is made available to Dean, who may or may not comment leading to ‘’seen by Dean’’ and the ‘’ongoing’’ status in the Supervision section of the GSM. The candidate, thesis advisors and other supervisors have regular progress meetings: six and ten months after starting and from then on annually or more frequent as desired. The ten-month meeting should result in a go/no go decision. The candidate and thesis advisor prepare a report of each meeting and appraisal, sign and upload the documents (only visible to PhD student and the-sis advisor) into the GSM. Temporary leave (e.g. maternity leave) as well as premature ending can be registered with comments. In order to build up a portfolio the candidate keeps a record of his/her academic achievements, including courses taken and taught, in the milestones subsection. Thesis advisors approve the milestones. For mandatory PhD courses, an exemption by the Dean can be requested.

Compulsory registration (University of Freiburg)

Compulsory registration for all doctoral researchers at the University of Freiburg was introduced in 2010 via a resolution of the Rectorate. The central registration of doctoral researchers is important for the university’s quality assurance measures. Since April 2014, when the new State Higher Education Act came into effect, registration has also been required by law. By collecting further data, the university aims to make the procedure for earning a doctoral degree more transparent and tailor the services of the IGA more closely to the needs of the doctoral researchers in order to provide them even better support.

Support and raising concerns (University College London)

All students have at minimum one primary and one subsidiary supervisor. They may have more than one of each and may also have other informal supervisors who may come from outside UCL. If a student has concerns, they should first raise the issue(s) with their supervisory team. They may raise issues with their departmental graduate tutor if they feel they can-not discuss it with either supervisor or escalate to the faculty graduate tutor or the Doctoral School. UCL also has a student mediator who has the authority, on behalf of the Provost, to mediate, to act relatively informally and speedily, and propose practical solutions to help resolve matters. If a student feels that concerns are still not met, he or she can submit a formal grievance under UCL’s complaints procedure, which is then considered formally by a panel which takes evidence. If this is still not satisfactory, issues can be submitted to the national Office of the Independent Adjudicator.

Setting standards through summer schools and a seminar on plagiarism (University of Strasbourg)

PhD students in law, sciences and history of law very often work in isolation and tend to focus only on their particular subject. In order to broaden their perspectives and to break the isolated climate in which they may work, the docto-ral school organises a summer school funded by an ‘Initiative of Excellence’ grant by the University of Strasbourg. Professors from France and elsewhere give seminars promoting and facilitating discussion on specific themes. Two

28


workshops are also organised on writing. Not only do students receive ‘passive’ learning on what plagiarism is and its consequences in legal and disciplinary terms, they also have the possibility to test themselves by participating in workshops where they face real examples of plagiarism.

1d) Learning outcomes and assessment

Doctoral learning outcomes (Utrecht University)

Utrecht University has established learning outcomes for doctoral programmes based on the generic learning out-comes in a position paper by the Dutch Universities Association (VSNU, 2014). These learning outcomes, which are set out in the UU’s Doctoral Degree Regulations and apply to all PhD candidates at Utrecht University, are:• the PhD candidate has provided an original contribution to academic research which can stand up to the peer-

based quality review system used in the Netherlands;• the PhD candidate has demonstrated the ability to independently apply the academic methods used in the field of

study concerned for the development, interpretation and application of new knowledge;• the PhD candidate has acquired and worked with a substantial body of knowledge, which at the very least includes

the principles and methods of international academic practice and theoretical formulations, methodologies and study of the relevant field of study;

• the PhD candidate has the ability to design and implement a substantial project aimed at developing new knowledge;• the PhD candidate is capable of adequately expressing the findings and methodologies of the relevant specialism

and/or field of study;• the PhD candidate is capable of exercising the social responsibility connected with executing, applying and using

his or her research.

Examination processes (Leiden University)

The PhD candidate (formally) requests the thesis advisor(s) to initiate the PhD graduation process. The thesis advisor proposes to the Dean a thesis committee of at least four members. He/she indicates to what extent the proposed mem-bers have been involved in the research and/or the realisation of the dissertation. The thesis advisors cannot be mem-bers of the thesis committee. A list of papers to be published in the thesis (title, authors, journal, if published) is made available to the Dean. The Dean’s Office checks for 1) adherence to Doctoral Rules and Regulation and 2) independence of thesis committee members (e.g. no co-author of PhD student’s thesis manuscript papers and may suggest change in the membership). Then the Dean appoints the secretary and installs the thesis committee.

The secretary of the thesis committee requests the members to assess the thesis manuscript in terms of the PhD can-didate’s ability to perform independent research within six weeks. The secretary informs the PhD candidate about the decision of the thesis committee and if favourable requests the Dean to grant the PhD student admission to the public defence. The decision of the public defence admission advice is taken by majority vote. In case of an Honours degree three extra external referees are recruited. The thesis committee advice is the most important quality control step at LUMC/UL.

Then an opposition committee is assembled for the public defence. The public, oral defence itself is followed by a brief deliberation by the opposition committee, after which the PhD degree is bestowed on the candidate by the chair of the opposition committee (the Rector or replacement).

Methods used to assess learning (University of Helsinki)

The Faculty has developed the dissertation examination process and its guidelines with a long-term perspective. In order to ensure impartiality, the preliminary examiners and the opponent(s) must, in accordance with the standing regulations, be generally chosen outside the Faculty and preferably outside the University. They must not have pub-lished co-authored publications or worked in research projects with the student during his or her postgraduate studies. The head of department makes a proposal on the preliminary examiners, the opponent(s) and the internal examiner after consulting the supervisors of the dissertation. The quality of the process is ultimately under the responsibility of

29


the Faculty Council, which appoints the preliminary examiners, the opponent(s), the internal examiner as well as the custos14, and decides on the approval and the grade of the dissertation.

The guidelines of the preliminary examiners and the opponent(s) explain the criteria against which the dissertation must be evaluated. The Faculty has prepared grade descriptions to advance the consistency of grading. The internal examiner must be familiar with the Faculty’s evaluation practices, and it is he or she who ensures that the grading committee fol-lows the regulations of the University and the Faculty. The Faculty Council may, at its own discretion, and especially if the preliminary examiners disagree with each other, invite the internal examiner to act as an expert in deciding whether to grant a doctoral candidate permission to defend the dissertation in a public examination. Also the cancellation of the pro-cedure of preliminary examination because of negative statements is always noted by the Faculty Council, which super-vises the quality of the preliminary examiners’ statements in addition to the quality of dissertations. The language of the dissertation is reviewed in a separate language revision, unless the doctoral candidate has written the dissertation in his or her native language or for a specific language subject. The Faculty has included the decision to implement a plagiarism control system in the examination of dissertations in the standing regulations for its postgraduate degrees.

Step 2: Scrutiny processes

2a) Scrutiny of universities and departments

The internal quality review process (University College London)

In the United Kingdom each university is reviewed under the Quality Assurance Agency’ Higher Education Review every five years. This considers quality processes including those for research degrees and results in actions which must be followed up. The QAA’s Quality Code includes a specific chapter (chapter B11) on research degrees with 18 indicators against which universities are judged15. All universities are required to have an internal quality review process and the Higher Education Review judges its effectiveness.

At UCL each Department (or equivalent unit) is reviewed formally every six years under the Internal Quality Review process (IQR) by a panel consisting of three senior members of staff, a student, an external expert, and a secretary. The IQR process reviews all aspects of the Department’s educational provision (undergraduate and postgraduate including research degree provision) and explicitly considers the extent to which the Code of Practice for research degrees is being met and the quality of the research environment. The outcome lists necessary, advisable and desirable actions along with elements of good practice. Departments must respond to actions requested and these are followed up.

Faculties report annually on their strategy for doctoral education, which includes how they are strengthening the re-search environment and ensuring their ability to provide an excellent quality training environment.

The research quality of the University is reviewed externally every six years through the Research Excellence Frame-work16. This considers elements of doctoral education but only as evidence of research excellence, including numbers of candidates and doctoral degrees awarded.

The Standard Evaluation Protocol (Utrecht University)

The quality of research at Dutch universities is assessed once every six years. This happens during research visits by inter-national panels of experts and colleagues. The assessment criteria and procedures for this are described in the Standard Evaluation Protocol17. Recently, the SEP has been expanded to include the supervision of PhD candidates at universities.

14 A custos is a thesis supervisor, a professor in charge of the public examination, who ensures that everything goes according to the rules and practices

in the public examination.

15 http://www.qaa.ac.uk/assuring-standards-and-quality/the-quality-code/quality-code-part-b

16 http://www.ref.ac.uk/

17 https://www.knaw.nl/nl/actueel/publicaties/standard-evaluation-protocol-2015-2021/@@download/pdf_file/SEP%202015-2021.pdf

30


Experts visit research schools and graduate schools so that recommendations for further improvement can be obtained.

In addition to the national review system, Utrecht University has an internal quality assurance system for research and teaching. Its main underlying principles are that Deans are primarily responsible for the quality of teaching and research within their faculty and the system used to safeguard that quality, and that quality assurance is best achieved from within the research and teaching environs, with the delivery of quality assurance fitting, subject to certain pre-conditions, within the scope, nature and culture of the relevant organisational entity.

All Master’s and doctoral programmes are organised into Graduate Schools. Deans have assigned responsibility for the teaching and supervision of PhD candidates to the Board of Studies (BoS) of the Graduate Schools. In order to safe-guard the quality of doctoral programmes, each graduate school has a cyclical quality assurance system which enables the BoS to monitor and continually improve the quality of the programmes. The Executive Board has established min-imum requirements for the quality assurance system. Deans are required to report once a year to the Executive Board on the quality assurance in the Graduate School(s) within their faculty.

French doctoral school reviews (UPMC, Paris-Sud, Strasbourg)

All French doctoral schools are reviewed every fifth year by a national agency, the HCRES18. The Evaluation protocol fol-lows guidelines published on the website of the agency19. Most important is the self-evaluation by the doctoral schools.

At UPMC, the self-evaluation schema is common to all doctoral schools and is structured into seven sections: scientific policy, candidate recruitment, candidate follow-up, training policy, international opening, governance and internal life of the doctoral school, pooling of means and actions across doctoral schools.

The responses to the self-evaluation schema are analysed by the Institute of Doctoral Education (IFD, the graduate fac-ulty at UPMC) to get a picture of the actual state of doctoral education, and to extract best practices. This analysis is pre-sented to the IFD council, to which all doctoral schools belong, and is further discussed between the doctoral schools.

A multi-level system of quality assurance and control (University of Milan)

PhD programmes are assessed at three levels at the Università degli Studi di Milano: national level, university level and individual PhD programme level. The attention toward quality assessment is also motivated by the fact that resources for PhD student salaries are for the largest part distributed by the Ministry to each university, and thus need criteria to be distributed to the various PhD schools.

At national level, decree n. 45 issued in 2013 by the Ministry of Education, University and Research, has defined criteria for the assessment of the quality of PhD courses: a threshold number of students, the scientific quality of faculty mem-bers, and the provision of adequate research equipment. This quality assessment is carried out by a national evaluation agency (ANVUR) every three years. ANVUR has the power to close PhD programmes which do not meet the required standard level of quality.

The University of Milan has established an internal assessment body (Nucleo di valutazione) whose aim is to carry on periodic (every few years) quality assessment of departments and teaching activities, including PhD programmes. An in-depth evaluation of PhD programmes is currently underway, based on openness and accountability of selection procedures, satisfaction of PhD students, resources made available for the PhD programme, internationalisation, and scientific productivity of thesis advisors.

Besides these mechanisms, some PhD programmes have been voluntarily trying out additional quality assessment procedures. For example, the PhD programmes in Sociology, Political Science, Labour Studies and Physics collect

18 HCERES stands for “Haut Conseil de l’évaluation de la recherche et de l’enseignement supérieur” (formerly AERES “Agence d’évaluation de la recherche et de l’ensei-

gnement supérieur”).

19 http://www.education.gouv.fr/bo/1998/36/sup.htm

31


systematic evaluations of teaching activities and training satisfaction from first- and second-year students through an anonymous on-line questionnaire. Some of these PhD programmes have also been monitoring the occupational outcomes of their doctors. The PhD programmes in Philosophy and Physics have adopted procedures of accountability in the phase of screening and selection of PhD candidates, in which the admission interviews are organised as a work-shop and publicised to all members of the faculty. These additional procedures have been proposed by the government of the University of Milan for adoption by all PhD programmes.

Evaluation office (University of Zurich)

The Evaluation Office of the University of Zurich is mandated to organise and supervise evaluations for all parts of the university at all levels. The purpose of the evaluations is to assess, assure and improve the quality of the academic work associated with research, teaching and central services. Evaluations are carried out at regular intervals of six to eight years. The evaluation of academic units comprises the fields of research, teaching, promotion of junior researchers, services, leadership and administration as well as organisational structures. At the University of Zurich, a multi-level evaluation procedure (informed peer review) is applied, which consists of the following phases: self-evaluation, exter-nal evaluation, reporting, follow-up and monitoring.

Evaluation of doctoral programmes (LMU Munich)

The GraduateCenterLMU has developed an evaluation scheme for doctoral programmes which is tailor-made for inter-nal use to support steering. This evaluation scheme includes several consecutive modules which are adapted to the demands and requirements of the individual programme. It is comprehensive as it includes multiple perspectives (doc-toral researchers, PIs, coordinators) on the programme’s structures, processes, outcome and added value. The results are passed on to the programme in the sense of a SWOT analysis. The implementation and execution of the evaluation by the GraduateCenterLMU ensures an objective, quasi-external view and guarantees absolute confidentiality. The evalu-ation is conducted at the request of the doctoral programmes. Results are not only used to provide feedback to improve the evaluated programme but also for funding advice and the development of new services, recommendations, and for the development of doctoral education at LMU Munich in general.

2b) Scrutiny of candidate progress

A web-based management system for Quality Assurance (Leiden University)

Leiden University Medical Centre (LUMC) is in the process of implementing a web-based PhD student registration and monitoring system, which will subsequently be taken up by the whole university. It has the three sections (PhD admis-sion, supervision and graduation) and the (main) stakeholders are PhD students, thesis advisors, Deans and Graduate School offices. The system incorporates a workflow that is based on the Rules and Regulations of the UL Doctorate Board and guides the PhD candidate, thesis advisor and Dean in a user-friendly way through and across sections. It provides valuable help in keeping the Plan-Do-Check-Act cycle as it allows easy document retrieval, monitoring of (progress in) processes and extensive reports (including metadata) at any aggregation level in a complex matrix or-ganisation such as LUMC. The system is also linked to LUMC’s research publications database enabling bibliometric assessment of the Graduate School’s scientific production.

Scrutiny of the candidate’s work plan (KU Leuven)

Within six months after the start of a PhD, a provisional work plan has to be submitted and is evaluated by the Doctoral Committee. Within the first year, the candidate has to defend his/her project before a jury composed of the advisor(s) and two internal experts. The jury examines the candidate on all aspects of the project (background, techniques, etc.), giving a positive or negative evaluation. In case of a positive evaluation, suggestions can still be made for further im-provements (e.g. taking courses for presentations skills, statistics,...). In case of a negative evaluation, the student is given a second chance, during which a member of the Doctoral Committee is added to the jury. If the evaluation remains negative, the PhD is terminated.

32


In the second year a research seminar is given in the presence of the advisor(s) and internal experts. The jury is encour-aged to engage in scientific discussions and to make suggestions for improvements.

In the third year a final doctoral plan is submitted and defended before a jury composed of the advisor(s), internal experts and four members of the Doctoral Committee. The doctoral plan describes the studies that have been performed already and the remaining experiments to be performed in the last year. In case of a positive evaluation, the candidate is allowed to submit a thesis when the remaining experiments have been performed. The jury can also reject the plan or request additional studies or experiments. Depending on the extent of these revisions, a second presentation may be necessary.

In the final year a thesis is submitted and evaluated by the advisor(s), the internal experts and two external experts (one from outside KU Leuven but within Belgium and one international expert). The thesis can be rejected, major or minor revisions requested, or accepted. In case of rejection, the candidate is given one more chance to resubmit. Once accepted (after revisions), the candidate publically defends the thesis in the presence of the same jury that has evaluated the thesis.

Measuring progress (Utrecht University)

The (co-)supervisor holds an evaluation meeting with the PhD candidate at least once a year, to discuss the candidate’s progress. Evaluation of the supervision itself is part of this annual meeting. At the end of the first year of a full-time programme, the results of this meeting form the basis for deciding whether or not to continue with the doctoral pro-gramme (part-time programmes have modified requirements).

Step 3: Measurement

A monitoring database (University of Zurich)

In an effort to increase transparency and orientation for PhD candidates and supervisors, two large graduate schools at UZH are currently setting up or planning to use a database to monitor the dissertation process of each individual PhD candidate from admission to final graduation. All necessary steps and tasks to be done (e.g. signing the doctoral agreement, past and upcoming meetings of the PhD candidate with the thesis committee, open and fulfilled teaching duties or certificates) are available in the database. At every moment, the doctoral candidate (as well as the supervisor) knows exactly which steps are to be taken and also sees what he or she has already achieved. This database furthermore facilitates the quality assurance process for the coordinators of the graduate schools or doctoral programmes since all information is available for all PhD candidates of the school/programme, and actions to be taken (or reminders to be sent to professors or PhD candidates) are transparent and easy to determine.

Student monitoring through the online research student log and faculty strategic plans and indicators of success (University College London)

All research students must report at six monthly intervals through UCL’s online Research Student Log. This must be signed off by principal and subsidiary supervisors. Departmental graduate tutors monitor this to ensure smooth and steady progress following up where students (or supervisors) seem to be having problems. Usage statistics are repor-ted to Faculties and to RDC. The Log also contains a skills self-assessment section, which helps students determine their development needs and leads to the training opportunities available. The Log has been in operation since 2003 and mandatory since 2009.

Students are initially admitted as MPhil students. After between 9 to 18 months they must submit an upgrade report outlining their research idea/hypothesis, progress to date, and plans for the remainder of the PhD (a report on progress is also submitted by the principal supervisor). This is assessed orally by a panel chaired by the subsidiary supervisor (but not including the principal supervisor). If successful, the candidate’s registration is upgraded to PhD. If not, they may have a second attempt and if this is unsuccessful they may only submit for an MPhil once they have completed the project. Upgrade rates and average timings are also monitored and reported to Faculties.

33


Submission rates (within four years for full-time students) are reported to Faculties and to RDC annually, prompting actions where rates are not satisfactory.

Supervisor workload management and supervision quality indicators (University of Helsinki)

The Faculty of Arts has sought to ease the workload of the implementation of postgraduate education by limiting the intake of postgraduate students and the number of supervised students per supervisor. The Faculty has sought to use the official six-year monitoring of the postgraduate students’ study progress to improve the situation of such students by, for instance, having the students update their research proposal and, if necessary, appointing a new supervisor. Although several actors participate in the quality assurance of the implementation of postgraduate studies as such, it is possible that many of them remain distant from the student’s viewpoint. Consequently, one of the objectives of the doctoral programmes is to increase the communality of postgraduate education.

Thus far, the Faculty has only recommended the use of a supervision plan, but information on the extent of its use has been lacking. The fundamental aim of the supervision plan is to serve as a tool for the interaction between the supervisor and the student. As of the spring 2014 application period, the Faculty seeks to make the use of the supervision plan a common practice by sending the new postgraduate students and their supervisors a clear template they can use for its preparation.

The examination process is fairly heavy and long, which is why the shortening of its duration without compromis-ing its quality is the aim of the 2013–2016 strategic period. The Faculty has begun implementing these measures by switching to the electronic processing of dissertation manuscripts, by shortening the period given to the preliminary examiners for preparing their statement and by changing language revision practices. As a result of these measures, the duration of the process was shortened by two months in 2013 in comparison to the previous year.

The Faculty’s senior lecturer in university pedagogy has made analyses of the statements from preliminary examiners. The latest analysis was prepared at the beginning of 2014 to chart the most often repeated criticism. In addition, the faculty head of research administration, who serves at the Faculty Council as a presenting official, follows up on the statements regularly. The Faculty utilises the statements in the quality management of its postgraduate education, especially when updating the guidelines for students and supervisors. They have also been used for the preparation of evaluation criteria and grade descriptions for the assessment of doctoral dissertations. The grade descriptions seek to influence both the preparation of dissertations (learning outcomes) as well as promote the consistent quality of the grading of dissertations. The Faculty has launched discussion on whether it should adopt a three-level scale in the grading of dissertations (pass with distinction, pass, fail).

In addition to the grades, the key follow-up indicators are the number of Doctor of Philosophy degrees and the degree completion times. The number of degrees has clearly been on the increase in the 2000s, and since 2008 the Faculty has regularly exceeded its degree target, which for 2013 was raised from the earlier 49 to 52. However, the degree comple-tion times have not become much shorter, although the length of the completion times in 2010-2012 may also suggest that the six-year monitoring of the postgraduate students’ study progress in fact works and supports the completion of dissertations whose progress has been slow. Verifying this, however, would require combining the information on the six-year monitoring and the completed degrees.

Step 4: Feedback mechanisms and quality enhancement

Incentives to improve quality (University of Zurich)

The Graduate Campus (GRC) of the University of Zurich supports doctoral programmes, graduate schools and facul-ties in their efforts to realise pilot projects to assure and/or further develop quality at the doctoral level. As such, once a year GRC launches a call for proposals for funding for those responsible for doctoral studies (e.g. coordinators and di-rectors of programmes, schools or faculty heads). In the past two application rounds thirteen very promising projects

34


received funding and appreciation for activities such as organising workshops for supervisors, the development of a database to monitor the dissertation process as well as setting up a mentoring programme for PhD candidates. GRC is following the implementation of the projects and promotes the dissemination of the results and lessons-learned via its university-wide Network Doctoral Programmes.

Enhancing quality through incentives (Pierre & Marie Curie University)

At UPMC, quality indicators for the doctorate are used to develop the doctoral education policy. Firstly, besides the volu-metric indicators that constitute the base input variables in the algorithm that distributes doctoral contracts between the doctoral schools, performance indicators such as times to completion, and number of candidates per supervisor are used as modifiers, thus giving the doctoral schools strong incentives to comply with the rules set up by UPMC and monitored by the IFD council. More precisely, only deviation with respect to the mean value of these performance indicators are considered and translated into bonus or reduction of the number doctoral contracts awarded to each doctoral school. However, the maximum deviation due to performance indicators is limited to 20% of the basic distribution. The unem-ployment rate among just graduated doctoral candidates is also taken into account by further reducing accordingly the number of contracts.

Secondly, the results from the employment and satisfaction survey are used to check the relevance of the training offer from IFD, and trim it if necessary to the needs expressed by the doctorate holders. So far, recent PhD graduates support the training offer. Therefore, in order to further incentivise the doctoral schools to send their candidates to the train-ing, doctoral schools spending more than 50% of their budget on training receive a budget increase, at the expense of doctoral schools that spend less than 90% of their budget altogether.

Thirdly, to compel doctoral schools to satisfy the terms of the Charter of Doctorate, extensions beyond three years for individual doctorates are dependent on the availability of corresponding financing, and on the existence of a report from the mid-thesis evaluation by a follow-up committee. Thus, deviant practices have slowly been eliminated, and all UPMC doctoral schools now adhere to the principles of the upcoming French regulation on doctoral education.

Fourthly, the heads of the doctoral schools regularly meet to discuss key elements of doctoral education at UPMC, thus ensuring a feedback and harmonisation of best practices for doctoral education.

Quality reviews and student feedback and quality review outcomes (University College London)

All doctoral students are surveyed every two years about aspects of their experience: supervision, facilities, opportu-nities for networking and conferences, induction, awareness of quality processes and of graduate tutors, etc. The fin-dings are fed back to the Faculties and considered as part of the Internal Quality Review (IQR) process.

Every department has a staff-student consultative committee, which includes a research student. They consider issues relating to the environment for research student training. Departments are required to respond in writing to concerns demonstrat-ing actions they are taking. All departmental staff-student consultative committee minutes are considered at the institutional staff-student consultative committee, which can raise concerns with departments and inform the IQR process.

Each Internal Quality Review results in a set of necessary, advisable and desirable recommendations. The department must respond as to how they will address issues raised and must report on progress made one year later.

A voice for doctoral researchers: Doctoral researcher conventions (University of Freiburg)

ProDoc, an initiative of doctoral researchers that aims to establish a representative body for doctoral researchers at the University of Freiburg, has received support from the Rectorate since 2011, including a lounge as a meeting place for doctoral researchers20. The aim of the initiative is to discuss questions relevant to doctoral researchers, to network with other doctoral researchers at the University, to network with initiatives for doctoral researchers at other universities, etc. ProDoc organises regular informal get-togethers for doctoral researchers and holds monthly meetings. At regular

20 https://www.prodoc.uni-freiburg.de/

35


meetings with the vice-rector and the International Graduate Academy, ProDoc members have the opportunity to voice their opinion on specific matters of concern for doctoral candidates and give feedback.

In 2015 the University of Freiburg established faculty-based doctoral researcher conventions including all doctoral re-searchers of the faculty. The aim of these conventions is to give doctoral researchers a strong voice at the University. They discuss all questions concerning doctoral researchers and make recommendations to the university’s governing bodies. They are asked for their opinion on faculty-specific doctoral degree regulations, which are then attached to the documents submitted to the Senate for approval. They serve as advisory members on faculty councils on matters concerning doctoral degrees regulations. Each doctoral researcher convention delegates a member to a joint committee responsible for discuss-ing general subjects relevant to all doctoral researchers that serves as a contact to the Rectorate and other university bodies.

The International Graduate Academy (IGA) employs several academic assistants, all of whom are doctoral researchers. They support IGA’s academic coordinators, especially with the course programme for doctoral researchers, and give regular feedback on research conditions for doctoral researchers, on IGA’s services, and on services they would like to see offered in the future. The doctoral researchers provide continuous evaluation of and feedback on IGA services (advising, course programme and coaching).

The structured doctoral training programmes have their own elected doctoral representatives, who are almost always members of the steering committee and are thus able to give feedback, propose new measures and changes to estab-lished measures, and enhance the programme.

Student feedback and scrutiny of students’ learning and well-being (University of Helsinki)

The University’s feedback system includes both practices applied only inside the University as well as surveys con-ducted on a national scale with cooperation partners. Such national surveys include the National Bachelor’s Graduate Survey and the graduate employment survey conducted in the National Aarresaari Academic Career Services Network. The University’s own ‘Learn’ feedback survey on the learning and study environment is currently distributed to the en-tire student body. In addition, each faculty has field-specific course feedback procedures and other necessary methods of collecting feedback.

It is primarily the supervisor’s duty to follow up on the progress of a postgraduate student’s dissertation and studies, but the Faculty has sought to systemise the follow-up by developing feedback mechanisms. The first-year feedback has been initiated and a template for an annual report is under preparation in cooperation with doctoral programmes and the Com-mittee for Postgraduate Studies. In conjunction with the six-year monitoring, the Faculty seeks to identify the factors that delay progress and, for instance, updates student register entries and supervision arrangements, if necessary.

The Faculty is currently developing the role of postgraduate feedback in the quality work of education planning. In autumn 2013 it initiated a survey charting the progress of postgraduate students’ first year of studies and is currently preparing a degree-level feedback survey for postgraduate students submitting their dissertation for the preliminary examination. The follow-up of the progress of the first year of studies increases the possibilities of addressing prob-lems related to supervision, the setting of objectives and the flow of information at an early stage, and of developing the practices that support them. The aim of degree-level feedback is to better take into consideration the needs of stu-dents and the demands of working life in the development work of postgraduate education. The Faculty has received external feedback to support the planning of postgraduate education from the Constructive Partners Group and the international Scientific Advisory Board (SAB). International benchmarks are also obtained from the cooperation of the faculties of humanities at Nordic capital universities, who together have signed a MoU on postgraduate cooperation (including the training of supervisors).

As regards the training of research skills and general career skills, the Faculty has collected course-specific feedback and has been able to use it to develop its educational offerings to better meet student requirements.

36


Appendix B: Characteristics and skills sets of doctoral graduates

LERU (2010) identified the following as characteristics of PhD graduates:

Doctoral graduates are best known for their analytical power and technical expertise which they have learnt to apply rigorously. However, the range of skills that they develop is much wider. This is often not even recognised by the gradu-ates themselves, although the increasing focus on skills development is helping to overcome this.

Research-intensive universities aim to produce doctoral graduates with a broad range of skills. Intellectual and aca-demic skills are developed to a much deeper extent than is done at the Bachelor’s or Master’s level, and doctoral gradu-ates are trained to be more inquisitive and independent. In addition, personal and professional management skills are developed as part of the doctoral experience. Research degrees concentrate strongly on the transferable skills relevant to research and such skills are relevant not only for the research workplace but also for other places of employment. Where appropriate doctoral programmes focus on their transferability to other domains in which a high level of crea-tive thinking and critical analysis are needed.

This broad range of skill sets includes:

Intellectual skills, which comprise the ability to• think analytically and synthetically • be creative, inquisitive, and original• take intellectual risks• deploy specific technical research related tools and techniques

Academic and technical skills, which comprise the ability to• understand, test and advance complex theories or hypotheses and to deploy sophisticated concepts, methodolo-

gies and tools in the chosen subject to a very high level• be able to identify issues and translate them into questions amenable to scholarly enquiry• develop and demonstrate academic credibility and become recognised as a member of an international scholarly

community• successfully pursue original research in the chosen field• understand the workings of a specific, high-level, research-intensive environment• use critical judgment in an objective manner based on verifiable evidence• apply highest standards of rigour in the proof of ideas• manage a high degree of uncertainty both in method and in outcomes• transfer new knowledge to scholarly communities and communicate it to society• work according to ethical principles• work in an interdisciplinarity setting or on an interdisciplinary topic

Personal and professional management skills, which comprise the ability to• persist in achieving long terms goals• manage projects with uncertain outcomes in diverse settings and organisations• take a project through all its stages: from developing the original idea, to developing a plan, garnering the evi-

dence, and communicating the results and their significance • be self-motivated and autonomous• work to achieve results with minimum supervision• be flexible and adaptable in approaching complex and uncertain problems• communicate very complex concepts• network internationally • work in a team• speak and present effectively in public

37


The following skills are sometimes also developed:• the ability to lead other researchers• the ability to teach and train others• the ability to organise conferences and workshops

These skills should enable and enhance the doctoral worker/graduate in three complementary domains: • Competence: acquiring specific expertise, knowledge, technology and methodology to conduct and understand

research within a discipline and across disciplines; • Achievement: gaining personal effectiveness, time, project, and self-management, developing a problem solving

attitude and assuming a leadership role; • Relationship: developing a team work attitude, collaborating and communicating with specialists and non-spe-

cialists.

38

notes

LERU publicationsLERU publishes its views on research and higher education in several types of publications, including position papers, advice papers, briefing papers and notes.

Advice papers provide targeted, practical and detailed analyses of research and higher education matters. They antici-pate developing or respond to ongoing issues of concern across a broad area of policy matters or research topics. Advice papers usually provide concrete recommendations for action to certain stakeholders at European, national or other levels.

LERU publications are freely available in print and online at www.leru.org.

All LERU publications, unless otherwise stated, are licensed under a Creative Commons Attribution 4.0 International License

advice paper - no. 19, March 2016

University of Amsterdam

Universitat de Barcelona

University of Cambridge

University of Edinburgh

University of Freiburg

Université de Genève

Universität Heidelberg

University of Helsinki

Universiteit Leiden

KU Leuven

Imperial College London

University College London

Lund University

University of Milan

Ludwig-Maximilians-Universität München

University of Oxford

Pierre & Marie Curie University

Université Paris-Sud

University of Strasbourg

Utrecht University

University of Zurich

LERU Office

Minderbroedersstraat 8B-3000 Leuven Belgium

tel +32 16 32 99 71 [email protected] www.leru.org

@LERUnews

maintaining a quality culture in doctoral education...maintaining a quality culture in doctoral...

Documents