simulation training: a systematic review of simulation in arthroscopy and proposal of a new...

Q4

Q1

lable at ScienceDirect

International Journal of Surgery xxx (2014) 1e8

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354

55

IJSU1333_proof ■ 30 April 2014 ■ 1/8

Contents lists avai

International Journal of Surgery

journal homepage: www.journal-surgery.net

565758596061626364
Review 6566676869707172
Simulation training: A review of simulation in arthroscopy andproposal of a new competency-based training framework

Charison Tay a,*, Ankur Khajuria a, Chinmay Gupte a,b

aDepartment of Surgery and Cancer, Imperial College London, Charing Cross Hospital, MSk Lab, 7th Floor, Lab Block, London W6 8RF, UKbNorth West London Deanery Higher Surgical Rotation in Orthopaedics, UK

7374
75767778798081828384858687
a r t i c l e i n f o

Article history:Received 4 December 2013Received in revised form28 March 2014Accepted 20 April 2014Available online xxx

Keywords:SimulationArthroscopyVirtual realityTrainingValidity

* Corresponding author.E-mail address: [email protected] (C.

http://dx.doi.org/10.1016/j.ijsu.2014.04.0051743-9191/� 2014 Published by Elsevier Ltd on behal

88899091929394

Please cite this article in press as: C. Tay, et abased training framework, International Jou

a b s t r a c t

Introduction: Traditional orthopaedic training has followed an apprenticeship model whereby traineesenhance their skills by operating under guidance. However the introduction of limitations on traininghours and shorter training programmes mean that alternative training strategies are required.Aims: To perform a literature review on simulation training in arthroscopy and devise a framework thatstructures different simulation techniques that could be used in arthroscopic training.Methods: A systematic search of Medline, Embase, Google Scholar and the Cochrane Databases wereperformed. Search terms included “virtual reality OR simulator OR simulation” and “arthroscopy ORarthroscopic”.Results: 14 studies evaluating simulators in knee, shoulder and hip arthroplasty were included. Themajority of the studies demonstrated construct and transference validity but only one showed concur-rent validity. More studies are required to assess its potential as a training and assessment tool, skillstransference between simulators and to determine the extent of skills decay from prolonged delays intraining. We also devised a “ladder of arthroscopic simulation” that provides a competency-basedframework to implement different simulation strategies.Conclusion: The incorporation of simulation into an orthopaedic curriculum will depend on a coordi-nated approach between many bodies. But the successful integration of simulators in other areas ofsurgery supports a possible role for simulation in advancing orthopaedic education.

� 2014 Published by Elsevier Ltd on behalf of Surgical Associates Ltd.

95
96 979899
100101102103104105106107108109110111

1. Introduction

The conventional surgical training model e see one, do one,teach one e lacks the necessary elements to ensure consistentguidance, objective assessment of performance and feedback, andis difficult to justify in terms of patient safety, care and costs [1e4].Shorter working hours in Europe [5] and America [6], has also beendetrimental to surgical training and raised concerns over patientcare and safety, and the acquisition of surgical skills for juniorsurgeons [7e10].

One possible method to address these issues is through simu-lation, which has long been used by the aviation industry to allowpilots control over their training environment while receivingstructured, quantitative feedback without risk to passengers [11].Parallels can be drawn between aviation and surgery as both

Tay).

f of Surgical Associates Ltd.

112113114115116

l., Simulation training: A revrnal of Surgery (2014), http:/

involve work of a critical, stressful nature with time constraints[12].

Satava had originally proposed the use of virtual reality (VR)simulation as an adjunct to surgical training over a decade ago,recognising the enormous benefits and savings in time, cost,equipment and safety [13]. However its acceptance into surgicaleducation was delayed then due to primitive technology, highexpense, the lack of sensory input, inadequate scientific evidencesupporting its role in surgical training and a lack of understandingof how to apply these simulations into a surgical training program.Many of these issues have now been remedied with the advent ofnewer technology providing more realistic simulations, hapticfeedback and lower costs. Yet the improvement in technology doesnot validate the use of simulations in surgical training and requiresindividual surgical specialities to properly evaluate its use. Litera-ture in general surgery has already shown a correlation betweentraining on VR simulators and improved performance in the oper-ating theatre [14e17]. However fewer studies have evaluated if thiscorrelation applies in orthopaedic surgery. At the minimum, thesimulator needs to demonstrate face validity, which is the extent to

117118119

iew of simulation in arthroscopy and proposal of a new competency-/dx.doi.org/10.1016/j.ijsu.2014.04.005

Delta:1_given name

Delta:1_surname

Delta:1_given name

mailto:[email protected]

www.sciencedirect.com/science/journal/17439191

http://www.journal-surgery.net

http://dx.doi.org/10.1016/j.ijsu.2014.04.005



C. Tay et al. / International Journal of Surgery xxx (2014) 1e82

1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465

66676869707172737475767778798081828384858687888990919293949596979899

100101102103104105106107108109

IJSU1333_proof ■ 30 April 2014 ■ 2/8

which the simulator is identical to real life scenarios. The ultimateaim of simulation studies would be to demonstrate concurrentvalidity, which would show that training on simulators is compa-rable with practice in the operating theatre as the gold standard(Table 1).

Arthroscopic education provides a unique area for the evalua-tion of simulation as a teaching tool because its use of videoequipment, limited field of vision and grasping instrumentsincluding forceps, graspers and scissors, can be easily mimicked inan artificial setting. Hence simulation provides an opportunity tointroduce a new method of teaching and evaluating arthroscopicskills.

2. Aims

To perform a review of the current literature on simulationtraining in arthroscopy and devise a framework that structures thedifferent simulation techniques that could be used in arthroscopictraining.

3. Methods

A search of Medline (PubMed), Embase, Google Scholar and TheCochrane Database was conducted using the search terms “simu-lator OR simulation” combinedwith “arthroscopy OR arthroscopic”.Inclusion criteria were English-language studies with evidencelevels I, II, III and IV [18]. Exclusion criteria include non-English-language articles, scientific meetings/proceedings, pharmacolog-ical studies and biomechanical studies.

4. Results

4.1. Simulators as a training tool

A total of 14 studies met the inclusion criteria. Five studies [19e23] assessed simulated knee arthroscopy, eight studies [2,24e30]assessed simulator shoulder arthroscopy and one study [31]assessed simulated hip arthroscopy (Table 2). Seven studies[2,20,22,24e26,28] evaluated and demonstrated construct validity,nine studies [20,21,23,26e31] demonstrated transfer validity andone study showed concurrent validity [21].

Howells et al. [21] is the only arthroscopic simulator study todemonstrate concurrent validity to date. In this study, 20 junior

Table 1Definitions of validity and reliability.

Type Definition

Face Validity Extent to which the simulator resembles clinicalscenarios

Content Validity Whether the domain or criteria being measured isactually being measured by the assessment tool orsimulator

Construct Validity Ability of the simulator to discriminate betweendifferent levels of expertise

Concurrent Validity To what extent the results of the simulatorcorrelates with the gold standard for the samedomain

Transfer Validity A measure of whether the system has the effect thatit proposes to have (i.e. is the simulator able toproduce a learning effect and improve performancewith continued use)

Test-Retest Reliability Similarity of results from a test when conducted attwo different time points

Inter-Rater Reliability Similarity of results from two or more differentobservers rating the performance of the sameindividual

110111112113114115116117118119120121122123124125126127128129130

Please cite this article in press as: C. Tay, et al., Simulation training: A revbased training framework, International Journal of Surgery (2014), http:/

orthopaedic trainees received traditional instruction and demon-strations of a diagnostic knee arthroscopy and were subsequentlyrandomised to either a fixed arthroscopic simulator training pro-tocol or received no additional training before performing theprocedure in theatre under supervision by the same experiencedorthopaedic surgeon who was blinded to the participant’s training.Howells found that the simulator-trained group significantly out-scored the control group on both the Orthopaedic CompetenceAssessment Project (OCAP) criteria and the Objective StructuredAssessment of Technical Skill (OSATS) global rating scale [21] asassessed by the supervising orthopaedic surgeon, demonstratingthat learning achieved on a simulator leads to better performancein the operating theatre.

Martin et al. [28] further showed that performance of traineesand surgeons on a standardised object localisation task on a VRshoulder arthroscopy simulator correlated with their performanceof the same task on a cadaveric model. This is important, ascadaveric simulations have long been held as the next best mode ofpractice and teaching, below actual experience in the operatingtheatre. However due to the costs of maintaining a bioskills labo-ratory, specimen availability and concerns of uniformity of cadav-eric specimens [32], simulators could supplant cadavericsimulations.

4.2. Simulators as an assessment tool

Simulators may also be used in the assessment of basic ortho-paedic skills. Vankipuram et al. [33] first described and evaluatedthe use of simulation for drilling, a basic orthopaedic skill that re-quires high levels of dexterity and expertise as inefficient drillingcan be a source of complications with adverse effects. The studydemonstrated that practice with the simulator tasks lead to anincrease in proficiency on real tasks, with participants who prac-ticed on the simulator prior to the practical test, performing betterthan participants who were put directly to the test [33]. Tashiroet al. [22] also proposed that simulations could be used to evaluatearthroscopic skills based on trajectory and force data after findingthat experienced arthroscopic surgeons performed significantlybetter and faster on the Sawbones Knee Simulator than surgicaltrainees. Several studies have also validated [34] the use of, andhave used motion analysis systems [23,27,31] as an objectiveassessment tool of arthroscopic skills; the commonest system usedbeing a validated motion tracking system (PATRIOT; Polhemus,Colchester, Vermont).

4.3. Transference of skills between different simulators

The skills learnt from one simulator may not necessarily betransferred into similar or improved performance on a differentsimulator. It is assumed that more common components betweentwo simulators would constitute greater similarity and greatertransfer of skills whereas fewer common components would leadto a lesser degree of transfer. This was investigated by Strom et al.[19] who examined the performance of 28 medical students whoinitially trained on a Procedicus Virtual Arthroscopy (VA) KneeSimulator. 14 medical students then trained on three other simu-lators with different visual-spatial components; the Procedicus VAShoulder, Procedicus MIST and Procedicus KSA Simulator, for anhour before all the participants were assessed again on the sametask on the Procedicus VA Knee Simulator. The results showed noimprovement in performance on the Procedicus VA Knee Simulatorafter training on the different simulators when compared with thecontrol group who were only given training on a Procedicus VAKnee Simulator [19]. However the authors noted that the 1-hallocated to training on the different simulators may not have


Table 2Studies on simulation in arthroscopy.

Simulation Author (Year) No. of participants Simulator type Validity type Method Measured outcomes Conclusion

Knee Strom et al., (2004) [19] 28 medicalstudents

Procedicus VA KneeSimulator

Quasi-Transfer(did notdemonstrate)

Control Group (n ¼ 14)performed a task to probe 6locations throughout a virtualknee.Experimental Group (n ¼ 14)Experimental Group received1hr of training on 3 differentsimulators before performingthe task.

Time to completionMovement economyNumber of collisions with thescope Number of collisions withthe probe

No significant difference wasobserved between theexperimental and controlgroups in any of the analysedvariables.

McCarthy et al. (2006) [20] 23 orthopaedicsurgeons

SKATS Face Orthopaedic surgeons weresplit into 3 groups.Group 1 (n ¼ 6) completed 5e50 knee arthroscopiesGroup 2 (n ¼ 7) completed 51e100 knee arthroscopiesGroup 3 (n ¼ 12) completedmore than 1000 kneearthroscopiesParticipants tasked to find fiveloose bodies. 3 non-surgeonnovices were also given 10practice sessions to establishtransfer validity

Time to completionNumber of loose bodies(pathology) foundNumber of arthroscope tip tocartilage contactScope path lengthProbe path lengthProbe observation score

Group 3 surgeons performedsignificantly faster, locatedmore loose bodies and hadshorter scope path length thanGroups 1 and 2.

ConstructTransfer

Howells et al. (2008) [21] 20 orthopaedictrainees

Sawbones KneeSimulator

Transfer Orthopaedic traineesrandomised to either 18sessions of simulator trainingfor a diagnostic kneearthroscopy or no additionaltraining. All trainees receivedtraditional instruction anddemonstration.

Orthopaedic trainees wereassessed on a diagnostic kneearthroscopy in the operatingtheatre by the sameexperienced orthopaedicsurgeon, who was blinded totheir status using OCAP and amodified OSATS.

Simulator-trained orthopaedictrainees had significantlyhigher scores on OCAP(p ¼ 0.0007) and OSATS(p ¼ 0.0011).

Concurrent

Tashiro et al. (2009) [22] 30 surgeons Sawbones KneeSimulator

Construct Surgical trainees (n ¼ 12)Orthopaedic trainees (n ¼ 12)Orthopaedic surgeons (n ¼ 6)Participants performed a kneejoint inspection and probingtask, and a partialmeniscectomy.

Time to completionPath length of arthroscopescissorsPath length of arthroscopeprobeSurgical force

The orthopaedic surgeonsperformed better than theorthopaedic trainees, whoperformed better than thesurgical trainees.

Jackson et al. (2012) [23] 19 orthopaedictrainees

Sawbones KneeSimulator

Transfer All participants had an initialtraining phase where theyperformed 12 meniscal repairson the simulator over 3-weeks.Group A (n ¼ 7) trained once amonth for 5 monthsGroup B (n ¼ 6) trained once, 3months after the initial trainingphaseGroup C (n ¼ 6) did not trainAll participants were assessed 6months after the initial trainingphase 12 times over 3 weeks

Time to completionTotal distance travelled by thesurgeon’s handsNumber of hand movements

A learning curve wasdemonstrated in the initialtraining phase for all trainees.A plateau in performance wasreached by 21st session for allresidents.Group C did not showdeterioration in performanceafter a 6 month interruption.

Shoulder Pedowitz et al. (2002) [24] 78 participantswith varying levelsof experience

ProcedicusArthroscopySimulator

Construct Medical students (n ¼ 35)Orthopaedic trainees (n ¼ 22)Orthopaedic surgeons (n ¼ 21)Participants performed a task

Time taken to touch eachspherePath ratioNumber of probe collisions

Participants with greaterarthroscopic experienceperformed better in path ratio

(continued on next page)

C.Tayet

al./International

Journalof

Surgeryxxx

(2014)1e8

3

1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465

66676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130

IJSU1333_proof

■30

April

2014■3/8

Pleasecite

thisarticle

inpress

as:C.Tay,etal.,Sim

ulationtraining:A

reviewofsim

ulationin

arthroscopyand

proposalofanew

competency-

basedtraining

framew

ork,InternationalJournalofSurgery

(2014),http://dx.doi.org/10.1016/j.ijsu.2014.04.005

Table 2 (continued )

Simulation Author (Year) No. of participants Simulator type Validity type Method Measured outcomes Conclusion

using a probe to touch a sphereat 11 different locations in thejoint

with tissueInjuries: collisions beyond thethreshold force

and time taken to touch eachsphere.

Srivastava et al. (2004) [2] 35 participantswith varying levelsof experience


Construct Novice (n ¼ 21) neverperformed a shoulderarthroscopyIntermediate (n ¼ 5) performed1-50 shoulder arthroscopies inlast 5 yearsExpert (n ¼ 9) performed >50shoulder arthroscopies in last 5yearsAll participants completed thesame study session with threemodules.

Hook manipulationAnatomic identificationArthroscopic navigation

No difference was observed inanatomic identification. Theexpert group performedsignificantly better in hookmanipulation and navigation.

Gomoll et al. (2007) [25] 43 participantswith varying levelsof experience


Construct No surgical experience (n ¼ 8)Junior orthopaedic trainees(n ¼ 11)Senior orthopaedic trainees(n ¼ 14)Orthopaedic surgeons (n ¼ 10)All participants were tested ona standardised test protocol onthe simulator

Time to completionDistance travelled with theprobeAverage velocity of probemovementNumber of probe collisions

Participants’ level of surgicalexperience correlated withtheir performance on thesimulator in all parameters

Gomoll et al. (2008) [26] 10 orthopaedictrainees


Transfer 10 orthopaedic trainees whowere previously evaluated onthe simulator were retested 3years later after gaining furtherarthroscopic experience

Time to completionDistance travelled with theprobeNumber of probe collisionswith tissueAverage velocity of probemovement

All participants hadsignificantly improved in allparameters

Construct

Howells et al. (2009) [27] 6 orthopaedicsurgeons

Alex shoulderprofessor

Transfer 6 lower-limb orthopaedicsurgeons were instructed onthe technique of anarthroscopic Bankart suture.They performed 3 singlesutures each on 4 occasions andwere re-tested 6 months later.

Total path length of handsNumber of hand movementsTime taken to perform thesutures

A learning curve withsignificant and objectiveimprovement wasdemonstrated for allparameters. There was also aloss of all initial improvementin performance of the sutureafter 6 months.

Martin et al. (2011) [28] 19 participantswith varying levelsof experience

InsightArthro VRShoulder Simulator

Transfer Novice (n ¼ 15) orthopaedictraineesExpert (n ¼ 4) orthopaedicsurgeonsAll participants were assessedon an object localisation task inthe simulator three times. Theywere re-assessed at least 2weeks later on a cadavericsimulator.

Time to completion The performance of theparticipants on the simulatorcorrelated with theirperformance on the cadavers.The expert group completedboth the simulator andcadaveric tasks significantlyfaster than the novice group

Construct

Martin et al. (2012) [29] 27 orthopaedictrainees

InsightArthro VRShoulder Simulator

Transfer Each orthopaedic trainee wasassessed on the simulatorannually for 3 consecutive yearson an object localisation task.The trainee’s surgical logbookwas also analysed for totalnumber of arthroscopies,

Time to completionDistance travelled with theprobeDistance travelled with thecamera

There was an inversecorrelation between year ofpost-graduate training, thenumber of shoulderarthroscopies previouslyperformed and time taken tocompletion of the task.

C.Tayet

al./International

Journalof

Surgeryxxx

(2014)1e8

4

1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465

66676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130

IJSU1333_proof

■30

April

2014■4/8

Pleasecite

thisarticle

inpress

as:C.Tay,etal.,Sim

ulationtraining:A

reviewofsim

ulationin

arthroscopyand

proposalofanew

competency-

basedtraining

framew

ork,InternationalJournalofSurgery

(2014),http://dx.doi.org/10.1016/j.ijsu.2014.04.005

shou

lder

arthroscop

iesan

dsu

rgical

cases.

Hen

net

al.(20

13)[30]

17med

ical

studen

tsProc

edicus

Arthroscop

ySimulator

Tran

sfer

Con

trol

Group(n

¼8)

Interven

tion

Group(n

¼9)

Allparticipan

tsco

mpletedan

initialb

aselinearthroscop

yon

acadav

eric

shou

lder.T

he

interven

tion

grou

psu

bseq

uen

tlyreceived

training

onasimulatoran

dall

participan

tsthen

repea

tedthe

samecadav

eric

arthroscop

y

Timeto

completion

Mod

ified

GOALS

score

Nodifference

atba

selin

e.Interven

tion

grou

pwas

sign

ificantlyfaster

than

the

control

aftertraining.

Both

interven

tion

andco

ntrol

grou

ps

also

had

sign

ificantlybe

tter

scores

onrepea

tco

mpared

totheirba

selin

e.

Hip

Polla

rdet

al.(20

12)[31]

20orthop

aedic

trainee

sHip

Arthroscop

yBen

ch-top

Simulator

Tran

sfer

LateralGroup

SupineGroup

Each

trainee

perform

ed12

diagn

ostichip

arthroscop

ies

across

4wee

ks

Timeto

completion

Totalpathlengthof

han

ds

Totalnumbe

rof

han

dmov

emen

ts

Initially

,therewas

grea

ter

variab

ility

inthelateralg

roup’s

learningcu

rvebu

tthey

caugh

tupby

episod

e9.

Bothgrou

pssign

ificantly

improve

dfrom

baselin

e.

C. Tay et al. / International Journal of Surgery xxx (2014) 1e8 5

1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465

66676869707172737475767778798081828384858687888990919293949596979899

100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130

IJSU1333_proof ■ 30 April 2014 ■ 5/8


been enough to substantially improve the performance of theparticipants in virtual arthroscopy tasks and experienced partici-pants may be able to improve their performance by cross trainingon different simulators due to better knowledge and familiaritywith different real-life scenarios and visual-spatial cues. Thereforethe transfer of skills may require some specificity, with identicalcomponents required between two situations in order for thetransfer to be successful. The transfer of a skill may also be possiblebetween two situations if the skill has the same logical sequence inboth situations although this would require investigation withfurther studies.

4.4. Prolonged delay in simulation training

Another interesting finding from the studies is the potentialinfluence of a prolonged delay between training on the simulatoron skills decay. Jackson et al. [23] found that a 6-month interrup-tion between an initial training phase to perform a simulatedmeniscal repair on a knee arthroscopy simulator, and the finalassessment phase, did not deteriorate the level of skill and per-formance of the participants who were all orthopaedic trainees.However the results of this study were disputed by Howells et al.[27] who found that in a group of fellowship-trained lower-limbsurgeons taught to perform a Bankart suture on a shoulderarthroscopy simulator, there was a deterioration in their perfor-mance of the same procedure 6-months later without simulatortraining during the interval. This discrepancy in the retention ofarthroscopic technical skills could be attributed to the participantsin Howells’ study being less receptive to retaining the skills thatthey learnt as they were lower-limb surgeons who had little vestedinterest in shoulder arthroscopy. On the other hand, participants inJackson’s study were orthopaedic trainees who had more of anincentive to retain the skills they learnt, as it may be applicable forthem in the future. Hence it is possible that the learning andretention of arthroscopic technical, and possibly surgical, skills isboth task- and surgical group-specific. Understanding the results ofthese studies may have practical implications such as whetherdifferent surgical specialities may share a basic set of surgical skillsand if training on different surgical simulators may or may not helpdevelop a particular skill for a certain surgical speciality.

For these simulators to become widely accepted and used astraining and assessment adjuncts and tools, researchers will alsoneed to introduce human factors and stressors into the simulationto more accurately represent the operating theatre environment.Additionally, most of the research has focused on arthroscopicmodels of simulation but only represents a fraction of proceduresperformed by orthopaedic surgeons. Hence further research isrequired to develop simulators that model other procedures.

4.5. Framework for arthroscopy simulation training

For the successful implementation of simulation into ortho-paedic training, a competency-based curriculum is required thatwill allow simulation to be incorporated into the current form ofsurgical education. Such a curriculum however currently does notexist in the United Kingdom, however its potential is gainingtraction and for the first time, the British Orthopaedic Association(BOA) has included a section on simulation as a training adjunct inits guidelines for trauma and orthopaedic curriculum, approved bythe General Medical Council [35]. The guidelines provide 3 broadalternative simulation pathways through which orthopaedictrainees can use simulation as a training tool. We propose that forthe purpose of arthroscopic learning, these pathways could befurther narrowed down to a framework called the “Imperial Collegeladder of arthroscopy simulation” where trainees begin with being


Fig. 1. a) Depicts the process through which trainees would train using the arthroscopy simulators and b) is the corresponding skills learnt at each step of the training ladder.


1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465

66676869707172737475767778798081828384858687888990919293949596979899

100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130

IJSU1333_proof ■ 30 April 2014 ■ 6/8

able to describe the theory and principles of the procedure,culminating with performing the procedure in the operatingtheatre (Fig. 1).

The ladder describes many different steps with a particularsimulator-type ascribed to each. However practically, one simulatorcould be used for several steps, with the same simulator havingdifferent consoles such as joysticks, keyboards and grasping in-struments attached to it. Many types of simulators have also beenvalidated including SKATS, Sawbones Knee Simulator, ProcedicusVA Knee Simulator, Procedicus Arthroscopy Simulator, AlexShoulder Professor, InsightArtho VR Shoulder Simulator and HipArthroscopy Bench-Top Simulator. Its introduction will first requireapproval by accreditation bodies including the Specialist AdvisoryCommittee and BOA with evaluation and assessment byrandomised-controlled trials and feedback from participants.

We view this ladder merely as a framework for enabling theeducator to select the right technique of simulation appropriate forthe procedure, competency of the trainee and the facilities avail-able. As such, it may be appropriate to join the ladder on a higherrunwithout necessarily passing through the lower rung. Thus thereis no compulsion to start at the lower rung of the ladder and work


up to the next rung. Taking the example of an anterior cruciateligament reconstruction, a junior trainee could perform the simu-lated surgery on an IPad application, “Touch surgery” [36] in orderto learn the steps of the procedure. However a more senior traineewho has assisted in this operation several times may be familiarwith the steps and hence can move straight to training on high fi-delity simulations such as sawbones or cadaveric models.

5. Discussion

Arthroscopy has grown in popularity since its introduction inthe 1960s as both a diagnostic and therapeutic tool [37]. It ishowever, a technically challenging and demanding technique toacquire and the current method of teaching and learning arthro-scopic skills is associated with iatrogenic injury to the patient suchas scuffing of the articular cartilage or missing pathologies [23,30].The steep learning curve coupled with shorter working hours andless patient contact raises the concern of inadequate training andtrainees have revealed that they feel less prepared and confident inarthroscopy compared with open procedures [38]. As such there isa move towards teaching and developing these skills outside the


Q3

C. Tay et al. / International Journal of Surgery xxx (2014) 1e8 7

1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465

66676869707172737475767778798081828384858687888990919293949596979899

100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130

IJSU1333_proof ■ 30 April 2014 ■ 7/8

operating theatre and one way to do so is through the use ofsimulators.

It is easy to see that simulators can have many different appli-cations in surgical education. Simulators may be used to instructsurgeons in the fundamentals of a procedure, assess their ability onobjectively measured parameters, and allow skills required for anoperation to be taught in a similar environment to that inwhich theskills will be used but without the pressures and distractions of anoperating theatre. Predetermined criteria can also be used to assessa surgeon’s progress in mastering a particular skill, thus allowingthe learner to advance at their own individual speed. Surgeons canalso use the simulators to rehearse and master complex and chal-lenging operations, and overcome steep learning curves beforecoming into contact with a patient, thus ensuring patient safety andoptimising the overall clinical experience of the surgeon at thesame time. Simulators will also allow practicing surgeons to learnnew laparoscopic and robotic techniques that may not be easilytaught in an operating theatre.

Many studies have evaluated and demonstrated construct andtransfer validity of a range of knee and shoulder simulators. How-ever only one study has been published demonstrating concurrentvalidity in a sawbones knee simulator [21]. Hence more research isrequired to assess the feasibility of simulators as a tool to improvethe technical skills of orthopaedic trainees before they enter theoperating theatre.

5.1. Other methods of assessing training in arthroscopy

VR simulators can be used as an assessment tool using objectivedata collected by the simulators as the subject runs through in thesimulation. However other forms of assessment tools could be usedin conjunction with simulators to quantify performance. The mostwidely used method is an assessment form that consists of ascoring system grading the participant on several skills or cate-gories with a numerical scale. Several such scales exist for theassessment of arthroscopic technique. For the knee, these includethe Arthroscopic Skills Assessment Form [39], Modified Ortho-paedic Competence Assessment Project [21], The Basic Arthro-scopic Knee Scoring System [40], Objective Assessment ofArthroscopic Skills global assessment form [41] and The Arthro-scopic Surgical Skill Evaluation Tool [42]. As for the shoulder, thereis the Modified Global Operative Assessment of Laparoscopic Skills[30]. Additionally, the Objective Structured Assessment of TechnicalSkills [43] is a scoring system widely used across different surgicalspecialities and may also be applicable for the assessment ofarthroscopic technical skills.

A scoring system for the non-technical aspects of surgery alsoexists, known as the Non-Technical Skills for Surgeons (NOTSS)rating system [44]. It consists of scores in 4 domains, situationawareness, decision-making, communication and teamwork, andleadership, all of which are applicable to the orthopaedic surgeon.

5.2. Challenges in implementation

Some of the challenges that directors of orthopaedic trainingprogramsmay face include the cost of the simulators. The cost of anAlex II Shoulder Professor model (Pacific Research Laboratories Inc,Vashon, WA) is USD$682.75 whilst the cost of an arthroscopy kneemodel (Pacific Research Laboratories Inc, Vashon, WA) isUSD$296.50 at the time of writing. These numbers do not take intoaccount the software and hardware that some simulation labora-tories may want to include in their simulators. Funding for simu-lators and facilities to house them may come from national grantsor hospital funds, with the belief that any additional costs incurredfrom running a simulation facility will be recouped from potential


savings in terms of shorter operating times and fewer complica-tions due to better-trained orthopaedic trainees. Ultimately studieswill be required to assess the cost-benefit analysis of running asimulation skills laboratory compared to the cost of teaching sur-gical trainees in the operating theatre.

Another challenge that teaching centres may face is having adedicated space with faculty to take responsibility of the facilityand training of the surgeons. A recent survey in the UK suggestedthat orthopaedic trainee access to a surgical skills simulator facilitywas 26.6%, well below the national average of 41.2% [45]. Onepossible solution is to have a shared facility between several hos-pitals in the same region and allocating training slots to eachhospital. Additionally the trainer does not always have to be pre-sent when the trainee is using the simulator as the simulators canrecord and provide feedback on performance in real time and alsoallow the trainee to see how he or she is progressing over time.Hence trainees can schedule their training based on their schedule,adding some flexibility to the training program.

6. Conclusion

Many studies have validated construct and transfer validity onvarious simulators in arthroscopy. However only one study evalu-ated concurrent validity, which would be the next step in providingevidence that training on a simulator provides comparable devel-opment of skills compared to training in the operating theatre.Future development of simulators will also require detailed anal-ysis of procedures and the identification of logical sequences oftasks to allow the transfer of skills from one procedure to another.

Ultimately the incorporation of simulation into an orthopaediccurriculum depends on a coordinated approach between manybodies. In the UK, these include the individual hospitals, nationaltraining bodies such as the joint committee on higher surgicaltraining and local educational training boards, and medical edu-cation directors. The successful integration of simulators in otherareas of surgery supports a possible role for simulation inadvancing orthopaedic education.

Ethical approval

No ethical approval was required.

Funding

No funding was required.

Author contribution

Charison Tay: Conception and design of paper, review of theliterature, drafting and final approval of the paper.

Ankur Khajuria: Conception, design, drafting and final approvalof the paper.

Chinmay Gupte: Conception, design and final approval of thepaper.

Conflict of Interest

The authors declare no conflict of interest.

Uncited references[18].



12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364

6566676869707172737475767778798081828384858687888990919293949596979899

100101102103104105106107108109110111112113114115116117118119120121122123124

IJSU1333_proof ■ 30 April 2014 ■ 8/8

References

[1] J.M. Rodriguez-Paz, M. Kennedy, E. Salas, et al., Beyond “see one, do one, teachone”-toward a different training paradigm, Postgraduate Medical Journal 85(2009) 244e249.

[2] S. Srivastava, P.L. Youngblood, C. Rawn, S. Hariri, W.L. Heinrichs, A.L. Ladd,Initial evaluation of a shoulder arthroscopy simulator: establishing constructvalidity, Journal of Shoulder and Elbow Surgery/American Shoulder and ElbowSurgeons 13 (2004) 196e205.

[3] L.R. Farnworth, D.E. Lemay, T. Wooldridge, et al., A comparison of operativetimes in arthroscopic ACL reconstruction between orthopaedic faculty andresidents: the financial impact of orthopaedic surgical training in the oper-ating room, The Iowa Orthopaedic Journal 21 (2001) 31e35.

[4] M. Bridges, D.L. Diamond, The financial impact of teaching surgical residentsin the operating room, American Journal of Surgery 177 (1999) 28e32.

[5] Department of Health. European Working Time Directive. [Online]. Availablefrom: http://webarchive.nationalarchives.gov.uk/20130107105354/ (accessed12.10.13).

[6] Accreditation Council for Graduate Medical Education. ACGME Duty HoursArchive: Approved Standards. Available from: http://www.acgme.org(accessed 12.10.13).

[7] C. Egan, R. Elliot, P. Fleming, European working time directive and the use ofsimulators and models in Irish orthopaedics, Irish Journal of Medical Science181 (2012) 143e146.

[8] J.D. Zuckerman, E.N. Kubiak, I. Immerman, P. Dicesare, The early effects of code405 work rules on attitudes of orthopaedic residents and attending surgeons,The Journal of Bone and Joint Surgery. American Volume 87 (2005) 903e908.

[9] K.J. Breen, A.M. Hogan, K. Mealy, The detrimental impact of the imple-mentation of the European working time directive (EWTD) on surgical seniorhouse officer (SHO) operative experience, Irish Journal of Medical Science 182(2013) 383e387.

[10] R.J.E. Skipworth, J.D. Terrace, L.A. Fulton, D.N. Anderson, Basic surgical trainingin the era of the European working time directive: what are the problems andsolutions? Scottish Medical Journal 53 (2008) 18e21.

[11] A. Mohan, M. Proctor, Virtual reality e a ‘play station’ of the future: a reviewof virtual reality and orthopaedics, Acta Orthopaedica Belgica 72 (2006) 659e663.

[12] J.D. Michelson, Simulation in orthopaedic education: an overview of theoryand practice, The Journal of Bone and Joint Surgery. American Volume 88(2006) 1405e1411.

[13] R.M. Satava, Virtual reality surgical simulator: the first steps, Clinical Ortho-paedics and Related Research 442 (1993) 2e4.

[14] A.G. Gallagher, N.E. Seymour, J.A. Jordan-Black, B.P. Bunting, K. McGlade,R.M. Satava, Prospective randomized assessment of transfer of training (ToT)and transfer effectiveness ratio (TER) of virtual reality simulation training forlaparoscopic skill acquisition, Annals of Surgery 257 (2013) 1025e1031.

[15] J.R. Korndorffer, J.B. Dunne, R. Sierra, D. Stefanidis, C.L. Touchard, D.J. Scott,Simulator training for laparoscopic suturing using performance goals trans-lates to the operating room, Journal of the American College of Surgeons 201(2005) 23e29.

[16] J. Price, V. Naik, M. Boodhwani, T. Brandys, P. Hendry, B.K. Lam, A randomizedevaluation of simulation training on performance of vascular anastomosis ona high-fidelity in vivo model: the role of deliberate practice, The Journal ofThoracic and Cardiovascular Surgery 142 (2011) 496e503.

[17] D. Stefanidis, M.W. Scerbo, P.N. Montero, C.E. Acker, W.D. Smith, Simulatortraining to automaticity leads to improved skill transfer compared withtraditional proficiency-based training: a randomized controlled trial, Annalsof Surgery 255 (2012) 30e37.

[18] OCEBM Levels of Evidence Working Group. The Oxford 2011 Levels of Evi-dence. Oxford Centre for Evidence-Based Medicine. [Online]. http://www.cebm.net/index.aspx?o¼5653 (accessed 10.10.13).

[19] P. Strom, A. Kjellin, L. Hedman, T. Wredmark, L. Fellander-Tsai, Training intasks with different visual-spatial components does not improve virtualarthroscopy performance, Surgical Endoscopy 18 (2004) 115e120.

[20] A.D. McCarthy, L. Moody, A.R. Waterworth, D.R. Bickerstaff, Passive haptics ina knee arthroscopy simulator: is it valid for core skills training? Clinical Or-thopaedics and Related Research 442 (2006) 13e20.

[21] N.R. Howells, H.S. Gill, A.J. Carr, A.J. Price, J.L. Rees, Transferring simulatedarthroscopic skills to the operating theatre: a randomised blinded study, TheJournal of Bone & Joint Surgery 90 (2008) 494e499.

[22] Y. Tashiro, H. Miura, Y. Nakanishi, K. Okazaki, Y. Iwamoto, Evaluation of skillsin arthroscopic training based on trajectory and force data, Clinical Ortho-paedics and Related Research 467 (2009) 546e552.

[23] W.F.M. Jackson, T. Khan, A. Alvand, et al., Learning and retaining simulatedarthroscopic meniscal repair skills, The Journal of Bone and Joint Surgery.American Volume 94 (2012) e132.

[24] R.A. Pedowitz, J. Esch, S. Snyder, Evaluation of a virtual reality simulator forarthroscopy skills development, Arthroscopy: the Journal of Arthroscopic &


Related Surgery: Official Publication of the Arthroscopy Association of NorthAmerica and the International Arthroscopy Association 18 (2002) E29.

[25] A.H. Gomoll, R.V. O’Toole, J. Czarnecki, J.J. Warner, Surgical experience cor-relates with performance on a virtual reality simulator for shoulder arthros-copy, The American Journal of Sports Medicine 35 (2007) 883e888.

[26] A.H. Gomoll, G. Pappas, B. Forsythe, J.J. Warner, Individual skill progression ona virtual reality simulator for shoulder arthroscopy: a 3-year follow-up study,The American Journal of Sports Medicine 36 (2008) 1139e1142.

[27] N.R. Howells, S. Auplish, G.C. Hand, H.S. Gill, A.J. Carr, J.L. Rees, Retention ofarthroscopic shoulder skills learned with use of a simulator. Demonstration ofa learning curve and loss of performance level after a time delay, The Journalof Bone and Joint Surgery. American Volume 91 (2009) 1207e1213.

[28] K.D. Martin, P.J. Belmonth, A.J. Schoenfeld, M. Todd, K.L. Cameron, B.D. Owens,Arthroscopic basic task performance in shoulder simulator model correlateswith similar task performance in cadavers, The Journal of Bone and JointSurgery. American Volume 93 (2011) e1271ee1275.

[29] K.D. Martin, K. Cameron, P.J. Belmont, A. Schoenfeld, B.D. Owens, Shoulderarthroscopy simulator performance correlates with resident and shoulderarthroscopy experience, The Journal of Bone and Joint Surgery. AmericanVolume 94 (2012) e160.

[30] R.F. Henn III, N. Shah, J.J. Warner, A.H. Gomoll, Shoulder arthroscopy simulatortraining improves shoulder arthroscopy performance in a cadaveric model,Arthroscopy: the Journal of Arthroscopic & Related Surgery: Official Publica-tion of the Arthroscopy Association of North America and the InternationalArthroscopy Association 29 (2013) 982e985.

[31] T.C. Pollard, T. Khan, A.J. Price, H.S. Gill, S. Glyn-Jones, J.L. Rees, Simulated hiparthroscopy skills-learning curves with the lateral and supine patient posi-tions, The Journal of Bone and Joint Surgery. American Volume 94 (2012) e68.

[32] W.D. Cannon, D.G. Eckhoff, W.E. Garrett Jr., R.E. Hunter, H.J. Sweeney, Reportof a group developing a virtual reality simulator for arthroscopic surgery ofthe knee joint, Clinical Orthopaedics and Related Research 442 (2006) 21e29.

[33] M. Vankipuram, K. Kahol, A. McLaren, S. Panchanathan, A virtual realitysimulator for orthopedic basic skills: a design and validation study, Journal ofBiomedical Informatics 43 (2010) 661e668.

[34] N.R. Howells, M.D. Brinsden, R.S. Gill, A.J. Carr, J.L. Rees, Motion analysis: avalidated method for showing skill levels in arthroscopy, Arthroscopy: theJournal of Arthroscopic & Related Surgery: Official Publication of theArthroscopy Association of North America and the International ArthroscopyAssociation 24 (2008) 335e342.

[35] General Medical Council. Approved Specialty Curriculum and AssociatedAssessment System 2013. [Online]. Available from: www.gmc-uk.org/(accessed 24.10.13).

[36] Touch Surgery. [Online]. http://www.touch-surgery.com (accessed 12.03.14).[37] W.E. Garrett Jr., M.F. Swiontkowski, J.N. Weinstein, et al., American board of

orthopaedic surgery practice of the orthopaedic surgeon: part-II, certificationexamination case mix, The Journal of Bone and Joint Surgery. American Vol-ume 88 (2006) 660e667.

[38] M.P. Hall, K.M. Kaplan, C.T. Gorczynski, J.D. Zuckerman, J.E. Rosen, Assessmentof arthroscopic training in U.S. orthopaedic surgery residency programs: aresident self-assessment, Bulletin of the NYU Hospital for Joint Diseases 68(2010) 5e10.

[39] M.J. Elliot, P.A. Caprise, A.E. Henning, C.A. Kurtz, J.K. Sekiya, Diagnostic kneearthroscopy: a pilot study to evaluate surgical skills, Arthroscopy: the Journalof Arthroscopic & Related Surgery: Official Publication of the ArthroscopyAssociation of North America and the International Arthroscopy Association28 (2012) 218e224.

[40] A. Insel, B. Carofino, R. Leger, R. Arciero, A.D. Mazzocca, The development of anobjective model to assess arthroscopic performance, The Journal of Bone andJoint Surgery. American Volume 91 (2009) 2287e2295.

[41] J.A. Slade Shantz, J.R. Leiter, J.B. Collins, P.B. MacDonald, Validation of a globalassessment of arthroscopic skills in a cadaveric knee model, Arthroscopy: theJournal of Arthroscopic & Related Surgery: Official Publication of theArthroscopy Association of North America and the International ArthroscopyAssociation 29 (2013) 106e112.

[42] R.J. Koehler, S. Amsdell, E.A. Arendt, et al., The arthroscopic surgical skillevaluation tool (ASSET), The American Journal of Sports Medicine 41 (2013)1229e1237.

[43] C.P. Winckel, R.K. Reznick, R. Cohen, B. Taylor, Reliability and construct val-idity of a structured technical skills assessment form, American Journal ofSurgery 167 (1994) 423e427.

[44] S. Yule, R. Flin, N. Maran, D. Rowley, G. Youngson, S. Paterson-Brown, Sur-geons’ non-technical skills in the operating room: reliability testing of theNOTSS behaviour rating system, World Journal of Surgery 32 (2008) 548e556.

[45] J.A. Milburn, G. Khera, S.T. Hornby, P.S. Malone, J.E. Fitzgerald, Introduction,availability and role of simulation in surgical education and training: reviewof current evidence and recommendations from the association of surgeons intraining, International Journal of Surgery (London, England) 10 (2012) 393e398.

125126127128


http://webarchive.nationalarchives.gov.uk/20130107105354/

http://www.acgme.org

http://www.cebm.net/index.aspx?o=5653



http://www.gmc-uk.org/

http://www.touch-surgery.com

simulation training: a systematic review of simulation in arthroscopy and proposal of a new...

Documents