1General introduction

R1

R2

R3

R4

R5

R6

R7

R8

R9

R10

R11

R12

R13

R14

R15

R16

R17

R18

R19

R20

R21

R22

R23

R24

R25

R26

R27

R28

R29

R30

R31

R32

R33

R34

Chapter 1

8

R1

R2

R3

R4

R5

R6

R7

R8

R9

R10

R11

R12

R13

R14

R15

R16

R17

R18

R19

R20

R21

R22

R23

R24

R25

R26

R27

R28

R29

R30

R31

R32

R33

R34

General introduction

9

1Changes in healthcare demands and related changes in thinking about training

healthcare professionals have inspired innovation of medical education. For the

people who participate in the current practice of medical education this means

that they have to deal with a variety of changes. The ones who have started to

use new methods in specialty training include trainees and registered specialists

working in training departments. Some of them are in charge of bringing about the

intended innovations too. The subject of interest in this thesis is how the people who

participate in medical specialty training deal with innovations in this training.

This introduction first sketches roughly the changes in the fields of healthcare and

medical education of the recent past. It continues with an overview of resulting

innovations in specialty training and current insight in their implications for training.

This overview makes clear that there is a need for better understanding how to

approach innovating specialty training. The introduction concludes with an outline

of the studies that were performed in order to add to this understanding.

The term innovation is used frequently in this thesis. There is not one exact definition

of the concept because of the differing terminology in various fields and overlap

with the concept of change.1 In this thesis, the term innovation in medical education

indicates something new or changed to training departments, which requires certain

intended behaviour of the people involved. Moreover, it typically includes new

concrete methods and tangible tools. For example, workplace-based assessment is

considered an innovation in this thesis, because it is was a new concept in specialty

training for which behavioural change of trainees and specialists was needed to enact

it in practice, including observation of performance and registering feedback on

forms. It comprised new methods like organized observation and specific feedback

directly afterwards, and new tools to support that like the mini-clinical evaluation

exercise (mini-CEX) form for structuring and registering feedback. The word change

in this thesis usually indicates matters that are less intentional than innovations or do

not include tangible tools. Terminology regarding medical education and the people

involved is diverse as well. Table 1 presents an overview of widely used terms across

the world.

R1

R2

R3

R4

R5

R6

R7

R8

R9

R10

R11

R12

R13

R14

R15

R16

R17

R18

R19

R20

R21

R22

R23

R24

R25

R26

R27

R28

R29

R30

R31

R32

R33

R34

Chapter 1

10

Table 1. Terminology regarding medical education*

Stage in medicaltrajectory

Terminology inthe Netherlands

Terminology in the UK Terminology in the USA

Phase followingundergraduatetraining

Postgraduate medical educationSpecialty training

Postgraduate medical educationSpecialty trainingSpecialist registrar training

Graduate medical educationResidency training

Trainee withinspecialty training

Specialist traineeTrainee

Specialist registrarRegistrar

Resident

Registered medical specialist

Medical specialist Hospital consultant Attending physicianAttending

Person in charge of specialty training

Specialty trainerLead consultant

Dean Program director

* Table adapted with author’s permission from Westerman, M. Mind the gap. 2012

Changes in healthcare and medical education

Undergraduate medical education of the past century was organized in academic

curricula with a firm ground in the basic sciences. Meanwhile, postgraduate medical

education consisted almost exclusively of unstructured on the job learning in an

apprenticeship model. These characteristics fitted the state of science, practice

and societal demands of that time.2 Yet, healthcare has continued to increase

in complexity, related to developments like endless possibilities in diagnostic and

treatment options, and ageing of populations leading a to large share of patients

with multi morbidity. This has been accompanied by sub specialization of patient

care and rising healthcare costs. At the same time, societal demand has risen for

accountability of these costs and of performance of physicians. Also, apart from

having excellent medical or surgical skills, patients expect from their doctors that

they communicate clearly, organize care adequately, and possess other general

qualities like these.3

These developments ask for advancement of medical education. Undergraduate

medical curricula have become more student-centred including problem-based

leaning approaches that are directed at active learning and integrating basic and

clinical sciences.4 The current requirements for postgraduate medical education

include that it needs to prepare physicians in such a way that they are competent to

meet current and future demands, while at the same time delivering good, safe and

accountable care. As a solution to do so, the contemporary approach to specialty

R1

R2

R3

R4

R5

R6

R7

R8

R9

R10

R11

R12

R13

R14

R15

R16

R17

R18

R19

R20

R21

R22

R23

R24

R25

R26

R27

R28

R29

R30

R31

R32

R33

R34

General introduction

11

1training is competency based and outcome oriented.5 This includes the use of sets

of attributes that trainees should develop during training as a basis for designing

training programs.6 It also means that demonstrated performance of trainees is

increasingly considered important for certification rather than time in training only.7;8

Contemporary innovations in specialty training and their implications

The above described developments in medical education have been accompanied

by a variety of innovative applications intended to support safe and competency

based specialty training and assessment of trainees. One example is the structured

practicing of skills of in simulated settings, before trainees are allowed to perform

those in practice, called simulation-based training.9 Also, a variety of new assessment

methods and tools has been developed for both formative and summative

assessment of learners. Well known is the example of workplace-based assessment

(WBA), meaning the assessment of actual performance of trainees at work, of which

there are several variations.10 It can concern single patient encounters of a trainee

observed by a registered specialist or senior trainee, followed by immediate feedback.

Widely used tools for this variation are the mini-clinical evaluation exercise (mini-

CEX)11 and the objective structured assessment of technical skill (OSATS)12. Another

variation of WBA is collecting ratings and comments of various people that a trainee

has worked with in a certain time period, typically called multi-source feedback

(MSF)10. Furthermore, a tangible new tool now used in most training programs is the

portfolio. This is a collection, now usually digital, of evidence of a trainee’s activities

and assessments. Portfolios are intended to keep track of trainee development,

create an overview to define learning goals and to support overall assessment. 13

Medical education research about these innovations up until now has mainly

focused on the tools and methods, and the extent to which desired effects can

be reached by using those.10;14;15 For example, the properties of the mini-CEX have

been scrutinized.14;16 Also, the educational effects of portfolios and various types of

workplace based assessment have been looked into thoroughly.13;17-20 Very generally

speaking, all of these tools have been found to be able to support learning and

assessment in specialty training. However, it has become clear that the effects of

the innovations depend largely on how they are used by people in daily practice

R1

R2

R3

R4

R5

R6

R7

R8

R9

R10

R11

R12

R13

R14

R15

R16

R17

R18

R19

R20

R21

R22

R23

R24

R25

R26

R27

R28

R29

R30

R31

R32

R33

R34

Chapter 1

12

of training.21 Experience with mini-CEXs illustrates that an innovation does not

automatically come into play fully. This tool is often used as a check-box and regarded

an administrative procedure that does not contribute to training, while the extra task

increases workload.22 Good psychometric properties of mini-CEXs are then merely

a theoretical benefit, but not relevant for the contribution to training. It is the way

in which supervisors and trainees handle the required feedback that makes them

contributory to learning or not. Thus, innovations for specialty training not only have

to be apt themselves. It is important even more so that using the innovations catches

on the people who participate in training. This means that innovations have to be

introduced into practice in a way that enables meaningful use. This is where the field

of medical education meets the field of change management.

Different aspects of innovation and change processes have since long received

interest in various fields, including business organization and management, sociology

and economics, and healthcare. In business, the main focus is on how people in

charge of change should lead these processes to do this effectively and bring about

intended effects.23 Sociologic and economic interest has since long been about how

innovations diffuse through groups of people, and when people adopt the use of a

novelty.24 The research of these fields combined is extensive and diverse, and more

detailed discussion of these fields is beyond the scope of this introduction. In the

field possibly most closely related to medical education, being that of healthcare,

implementation science is growing with the aim to understand complex issues like

implementing evidence-based guidelines for optimizing quality of care.25-27

In medical education research, innovation processes have received some attention

as well. They mainly concern undergraduate educational change. Reflections

on curriculum changes (for instance into using team-based learning28) aimed at

uncovering elements important for successful change are most abundant, and

have led to conclusions that most issues known from the above discussed fields are

important for medical education innovation as well. This means that issues related

to the people involved, the innovation, the context and the process are recognized

as important for medical education innovation. For instance, elements like strong

leadership, expertise, buy-in, fit with practice, approaching the process in phases,

R1

R2

R3

R4

R5

R6

R7

R8

R9

R10

R11

R12

R13

R14

R15

R16

R17

R18

R19

R20

R21

R22

R23

R24

R25

R26

R27

R28

R29

R30

R31

R32

R33

R34

General introduction

13

1and ensuring adequate time have been acknowledged as important for successful

curriculum innovation.28-30 Also, there is a vast body of reports that are based on

one of these elements, including the person in charge of curricular change31, or on

specific qualities that are deemed important for change like leadership qualities32;33,

communication skills34 or societal and organizational culture.4

Compared to the interest in innovating undergraduate medical education, processes

of innovating postgraduate medical education have remained underexposed.

However, innovating undergraduate and postgraduate training cannot expected to be

similar, since specialty training innovation involves the dynamic area of the practice

of patient care. The existing reports on postgraduate reforms are mainly thoughtful

reflections or advice.21;35-39 Empiric studies about innovating postgraduate medical

education are still scant5;40-42 and theory based insight into innovation processes has

only just started to form.43-45 Thus, while it is clear that innovating specialty training is

challenging, it is still largely unclear how innovations in this area should be dealt with

by the people in daily practice, in order to contribute to training. Therefore, insight is

needed into how the people who participate in medical specialty training deal with

innovations. This kind of knowledge can potentially support approaching innovation

of specialty training in such a way that it leads to high quality training, does not waste

precious energy of all, and ultimately leads to excellent patient care.

Aim of this thesis

The aim of this thesis is to contribute to the knowledge about innovating specialty

training. It uses knowledge about innovation processes from other fields as a basis,

and focuses on the experiences of the people who participate in medical specialty

training: residency program directors, consultants, and residents. The overall

question of this thesis is: how do people who participate in medical specialty training

deal with innovations in this training?

We approach this question by first looking into three distinct aspects that are

involved with innovation, respectively: approaches of the people in charge of training

for bringing about change, the effects in practice that using an innovation brings

about and the perceptions of the people involved with training regarding these

effects. These elements are combined in a fourth study of an innovation process,

R1

R2

R3

R4

R5

R6

R7

R8

R9

R10

R11

R12

R13

R14

R15

R16

R17

R18

R19

R20

R21

R22

R23

R24

R25

R26

R27

R28

R29

R30

R31

R32

R33

R34

Chapter 1

14

looking into the mutual influence of implementation approach and arising effects in

an innovation process.

Starting off with the people in charge of programmatic changes in actual specialty

training, chapter 2 addresses the question how program directors approach bringing

about change at a training department. Specifically, it concerns the senior consultant

with this responsibility for a department, designated in this chapter by the term

‘lead consultant’. An exploratory qualitative study using semi-structured interviews

is described, for which concepts relating to change management from business and

social psychology were used as a basis. The specific research questions of this study

were: which approaches to changes in specialty training are used by lead consultants?

And what factors influence these approaches?

Chapter 3 then looks into how the people involved with daily practice of specialty

training feel they are affected by an innovation. It describes a study among (lead)

consultants and trainees as users of workplace-based assessment (WBA), which

was studied as a case of an innovation in specialty training. In order to take into

account that their perceptions are not necessarily limited to the strictly education-

related domain, the study design was informed by sociologic theory on diffusion

of innovation. It sought to establish what types of effects of WBA are perceived by

consultants and trainees in the course of using WBA in the clinical workplace.

Chapter 4 looks further into a finding from the previous chapter by focussing on the

distinct perceptions that users of the same innovation can have about the effects

of using it. This was studied using Q methodology, which combines aspects of

qualitative and quantitative approaches for systematic investigation of perceptions

of trainees and consultants across various departments. The research question was:

What perceptions of the effects of using WBA exist among its users?

Combining the elements from the preceding chapters, chapter 5 looks into how the

effects of an innovation and of the approach to its implementation are intertwined.

The innovation in this study is the concept of transparency and competition in

R1

R2

R3

R4

R5

R6

R7

R8

R9

R10

R11

R12

R13

R14

R15

R16

R17

R18

R19

R20

R21

R22

R23

R24

R25

R26

R27

R28

R29

R30

R31

R32

R33

R34

General introduction

15

1specialty training, aimed at stimulating accountability and quality of training.

As a case in which this innovation was introduced, a Dutch national project was

studied. This was done using a theory-driven methodology based on theory from

implementation science. Qualitative analysis of project generated documents and

stakeholder insight was conducted guided by the question: what are the effects of

implementing transparency and competition in specialty training, and how are these

effects influenced by the implementation approach?

Finally, in chapter 6 the main findings from this thesis are discussed, and accordingly

a new conceptualization of innovating medical specialty training is proposed.

Furthermore, reflections on the research in this thesis are provided, including its

strengths and limitations and implications for further research and practice.

R1

R2

R3

R4

R5

R6

R7

R8

R9

R10

R11

R12

R13

R14

R15

R16

R17

R18

R19

R20

R21

R22

R23

R24

R25

R26

R27

R28

R29

R30

R31

R32

R33

R34

Chapter 1

16

REFERENCES

(1) Fagerberg J. Innovation: a guide to the literature. In: Fagerberg J, Mowery DC, Nelson RR, eds. The Oxford handbook of innovation. Oxford [etc.]: Oxford University Press; 2005;1-26.

(2) Flexner A. Medical education in the United States and Canada: a report to the Carnegie Foundation for the Advancement of Teaching. no. 44. 1910. New York ; Buffalo. Bulletin (Carnegie Foundation for the Advancement of Teaching) ; no. 4.

(3) Neufeld VR, Maudsley RF, Pickering RJ et al. Educating future physicians for Ontario. Acad Med 1998;73:1133-1148.

(4) Jippes M. Culture matters in medical schools. How values shape a successful curriculum change. 2013.

(5) Scheele F, Teunissen P, Van Luijk S et al. Introducing competency-based postgraduate medical education in the Netherlands. Med Teach 2008;30:248-253.

(6) Frank JR, Snell LS, Cate OT et al. Competency-based medical education: theory to practice. Med Teach 2010;32:638-645.

(7) Nasca TJ, Philibert I, Brigham T, Flynn TC. The next GME accreditation system--rationale and benefits. N Engl J Med 2012;366:1051-1056.

(8) Leung WC. Competency based medical training: review. BMJ 2002;325:693-696. (9) Motola I, Devine LA, Chung HS, Sullivan JE, Issenberg SB. Simulation in healthcare

education: a best evidence practical guide. AMEE Guide No. 82. Med Teach 2013;35:e1511-e1530.

(10) Miller A, Archer J. Impact of workplace based assessment on doctors’ education and performance: a systematic review. BMJ 2010;341:c5064.

(11) Norcini JJ, Blank LL, Arnold GK, Kimball HR. The mini-CEX (clinical evaluation exercise): a preliminary investigation. Ann Intern Med 1995;123:795-799.

(12) Martin JA, Regehr G, Reznick R et al. Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 1997;84:273-278.

(13) van Tartwijk J, Driessen EW. Portfolios for assessment and learning: AMEE Guide no. 45. Med Teach 2009;31:790-801.

(14) Norcini J, Burch V. Workplace-based assessment as an educational tool: AMEE Guide No. 31. Med Teach 2007;29:855-871.

(15) Ma L, Brindle M, Ronksley P, Lorenzetti D, Sauve R, Ghali W. Use of Simulation-Based Education to Improve Outcomes of Central Venous Catheterization: A Systematic Review and Meta-Analysis. Acad Med 2011.

(16) Hawkins RE, Margolis MJ, Durning SJ, Norcini JJ. Constructing a validity argument for the mini-Clinical Evaluation Exercise: a review of the research. Acad Med 2010;85:1453-1461.

(17) Driessen E, van Tartwijk J, van der Vleuten C, Wass V. Portfolios in medical education: why do they meet with mixed success? A systematic review. Med Educ 2007;41:1224-1233.

(18) Pelgrim EAM, Kramer AWM, Mokkink HGA, van den Elsen L, Grol RPTM, van der Vleuten CPM. In-training assessment using direct observation of single-patient encounters: a literature review. Adv Health Sci Educ Theory Pract 2011;16:131-142.

R1

R2

R3

R4

R5

R6

R7

R8

R9

R10

R11

R12

R13

R14

R15

R16

R17

R18

R19

R20

R21

R22

R23

R24

R25

R26

R27

R28

R29

R30

R31

R32

R33

R34

General introduction

17

1 (19) Ringsted C, Henriksen AH, Skaarup AM, van der Vleuten CPM. Educational impact of

in-training assessment (ITA) in postgraduate medical education: a qualitative study of an ITA programme in actual practice. Med Educ 2004;38:767-777.

(20) Daelmans HEM, Overmeer RM, van der Hem-Stokroos HH, Scherpbier AJJA, Stehouwer CDA, van der Vleuten CPM. In-training assessment: qualitative study of effects on supervision and feedback in an undergraduate clinical rotation. Med Educ 2006;40:51-58.

(21) van der Vleuten C, Verhoeven B. In-training assessment developments in postgraduate education in Europe. ANZ J Surg 2013;83:454-459.

(22) Bindal T, Wall D, Goodyear HM. Trainee doctors’ views on workplace-based assessments: Are they just a tick box exercise? Med Teach 2011;33:919-927.

(23) Kotter JP. Leading change. Boston: Harvard Business School Press, 1996. (24) Rogers EM. Diffusion of innovations. 5 ed. New York: Free Press, 2003. (25) Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations

in service organizations: systematic review and recommendations. Milbank Quarterly 2004;82:581-629.

(26) Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients’ care. Lancet 2003;362:1225-1230.

(27) Huis A, Schoonhoven L, Grol R et al. Helping hands: a cluster randomised trial to evaluate the effectiveness of two different strategies for promoting hand hygiene in hospital nurses. Implement Sci 2011;6:101.

(28) Thompson BM, Schneider VF, Haidet P, Perkowski LC, Richards BF. Factors influencing implementation of team-based learning in health sciences education. Acad Med 2007;82:S53-S56.

(29) Bland CJP, Starnaman SP, Wersal LM, Moorhead-Rosenberg LP, Zonia SP, Henry RP. Curricular Change in Medical Schools: How to Succeed. Academic Medicine 2000;75:575-594.

(30) Loeser H, O’Sullivan P, Irby DM. Leadership lessons from curricular change at the University of California, San Francisco, School of Medicine. Acad Med 2007;82:324-330.

(31) Rich ECM, Magrane DM, Kirch DGM. Qualities of the Medical School Dean: Insights From the Literature. Academic Medicine 2008;83:483-487.

(32) Souba WW, Day DV. Leadership values in academic medicine. Acad Med 2006;81:20-26. (33) Kaufman A. Leadership and governance. Acad Med 1998;73:S11-S15. (34) Dannefer EF, Johnston MA, Krackov SK. Communication and the process of educational

change. Acad Med 1998;73:S16-S23. (35) Calman KC, Temple JG, Naysmith R, Cairncross RG, Bennett SJ. Reforming higher

specialist training in the United Kingdom--a step along the continuum of medical education. Med Educ 1999;33:28-33.

(36) Dzau VJ, Yoediono Z, Ellaissi WF, Cho AH. Fostering innovation in medicine and health care: what must academic health centers do? Acad Med 2013;88:1424-1429.

(37) Grant JR. Changing postgraduate medical education: a commentary from the United Kingdom. Med J Aust 2007;186:S9-13.

(38) Ludmerer KM, Johns MM. Reforming graduate medical education. JAMA 2005;294:1083-1087.

R1

R2

R3

R4

R5

R6

R7

R8

R9

R10

R11

R12

R13

R14

R15

R16

R17

R18

R19

R20

R21

R22

R23

R24

R25

R26

R27

R28

R29

R30

R31

R32

R33

R34

Chapter 1

18

(39) Snelgrove H, Familiari G, Gallo P et al. The challenge of reform: 10 years of curricula change in Italian medical schools. Med Teach 2009;31:1047-1055.

(40) Lillevang G, Bugge L, Beck H, Joost-Rethans J, Ringsted C. Evaluation of a national process of reforming curricula in postgraduate medical education. Med Teach 2009;31:e260-e266.

(41) Malling B, Bonderup T, Mortensen L, Ringsted C, Scherpbier A. Effects of multi-source feedback on developmental plans for leaders of postgraduate medical education. Med Educ 2009;43:159-167.

(42) Wallenburg I, van Exel J, Stolk E, Scheele F, de Bont A, Meurs P. Between Trust and Accountability: Different Perspectives on the Modernization of Postgraduate Medical Training in the Netherlands. Academic Medicine 2010;85:1082-1090.

(43) Varpio L, Bell R, Hollingworth G et al. Is transferring an educational innovation actually a process of transformation? Adv Health Sci Educ Theory Pract 2012;17:357-367.

(44) Ginsburg L, Tregunno D. New approaches to interprofessional education and collaborative practice: lessons from the organizational change literature. J Interprof Care 2005;19 Suppl 1:177-187.

(45) Jippes E. The role of social communication networks in implementing educational innovations in healthcare

R1

R2

R3

R4

R5

R6

R7

R8

R9

R10

R11

R12

R13

R14

R15

R16

R17

R18

R19

R20

R21

R22

R23

R24

R25

R26

R27

R28

R29

R30

R31

R32

R33

R34