validation of the uw virtual reality turp simulator version 1.0 presented by: timothy kowalewski...
Post on 15-Jan-2016
213 views
TRANSCRIPT
Validation of the UW Virtual Validation of the UW Virtual Reality TURP SimulatorReality TURP Simulator
Version 1.0Version 1.0
Presented By:
Timothy Kowalewski
Robert Sweet, MD
Peter Oppenheimer
Suzanne Weghorst
Jeff Berkley, Ph.D
What is TURP?What is TURP?
Transuretheral Resection of the ProstateTransuretheral Resection of the Prostate Gold-standard for treating obstructive Gold-standard for treating obstructive
urinary symptomsurinary symptoms Challenging to teach and learnChallenging to teach and learn
Goals of the ProjectGoals of the Project
Overall Project Goal:Overall Project Goal:• To help streamline urological procedure To help streamline urological procedure
education with the aid of simulation technologyeducation with the aid of simulation technology
Specific Goals of This Study:Specific Goals of This Study:• Establish validation status of Version 1.0, UW Establish validation status of Version 1.0, UW
TURP Simulator as a training and assessment tool.TURP Simulator as a training and assessment tool.
TURP: a good model for SimulationTURP: a good model for Simulation
Objective assessmentObjective assessment High risk to patientsHigh risk to patients
Apparent training gap Apparent training gap Common problemCommon problem
National Average Number of TURPS performed by residentsNational Average Number of TURPS performed by residents
Source: ACGMESource: ACGME
19901990 19921992 19941994 19961996 19981998
Sweet, et al. Journal of Endourology. October, 2002.
Mainstay operation Mainstay operation
Apparent demandApparent demand
Amenable to current Amenable to current technologytechnology
99%1%
99%: Validated Simulator Useful in Training
2%
75%
23%
75%: Validated Simulator Useful after Residency
……YESYES
98%2%
98%: TURP is the Gold Standard of care
TURP: a good model for SimulationTURP: a good model for Simulation
MethodsMethods
Simulator ConstructionSimulator Construction Virtual anatomy and visual elementsVirtual anatomy and visual elements
Real-time force / tactile feedbackReal-time force / tactile feedback
O.R. instrumentationO.R. instrumentation
External physical modelExternal physical model
Auditory cuesAuditory cues
MethodsMethods
Simulator ConstructionSimulator Construction Training VideoTraining Video Definition of task and errorsDefinition of task and errors
Statement of goal:Statement of goal:
“ “Efficiently resect as much tissue as possible while avoiding errors Efficiently resect as much tissue as possible while avoiding errors and minimizing blood loss, amount of irrigant used, coagulation and minimizing blood loss, amount of irrigant used, coagulation current and number of cuts…”current and number of cuts…”
Special Thanks to Anthony Gallager, Ph.D
MethodsMethods
Simulator ConstructionSimulator Construction Training VideoTraining Video Pre-Task QuestionnairePre-Task Questionnaire Demographics, gender, professional experience, education, ‘video Demographics, gender, professional experience, education, ‘video
game’ experience, TURP-related questions, etc… game’ experience, TURP-related questions, etc…
Stratification of databaseStratification of database
MethodsMethods
Simulator ConstructionSimulator Construction Training VideoTraining Video Pre-Task QuestionnairePre-Task Questionnaire Pre-Compiled, 5-minute TaskPre-Compiled, 5-minute Task Three trained, non-medical techniciansThree trained, non-medical technicians Consistent, pre-determined responses Consistent, pre-determined responses Privacy and AnonymityPrivacy and Anonymity
MethodsMethods
Simulator ConstructionSimulator Construction Training VideoTraining Video Pre-Task QuestionnairePre-Task Questionnaire Pre-Compiled, 5-minute TaskPre-Compiled, 5-minute Task
Post-Task QuestionnairePost-Task Questionnaire Critique of simulator according to its elementsCritique of simulator according to its elements Open feedback Open feedback
MethodsMethods
Simulator ConstructionSimulator Construction Training VideoTraining Video Pre-Task QuestionnairePre-Task Questionnaire Pre-Compiled, 5-minute TaskPre-Compiled, 5-minute Task
Post-Task QuestionnairePost-Task Questionnaire AUA Annual ConferenceAUA Annual Conference 72 Experts72 Experts 19 Trainees19 Trainees 19 Novices19 Novices
MetricsMetrics
Grams of tissue resectedGrams of tissue resected Amount of irrigant usedAmount of irrigant used Quantity of ambient bloodQuantity of ambient blood Number and severity of bleeding vesselsNumber and severity of bleeding vessels Instrument position and interactionInstrument position and interaction Number of cuts at tissueNumber of cuts at tissue ~ 12 Hz resolution~ 12 Hz resolution Combinations thereof …Combinations thereof …
Results: Results: Face ValidityFace Validity
Do you feel the UW TURP Simulator would be useful as a Do you feel the UW TURP Simulator would be useful as a training tool?training tool?
93%
7%YESNO
Results: Results: Face ValidityFace Validity
Would you like to see it implemented into the curriculum Would you like to see it implemented into the curriculum of residency programs?of residency programs?
88%
12%
YESNO
Results: Results: Face ValidityFace Validity
58%
40%
2%YESNOMAYBE
Do you feel it would be useful as a tool for accreditation?Do you feel it would be useful as a tool for accreditation?
Results: Content ValidityResults: Content Validity
Categorical Standardized Likert scale of global acceptability
0.650.32
1.93
0.27
2.2
1.231.46
-3
-2
-1
0
1
2
3
Visual Realism
TactileRealism
AuditoryRealism
Bleeding/Bubbles
Chips Instrumen-tation
Overall
Acceptability threshold
Slightly acceptable
Moderately acceptable
Totally acceptable
Slightly unacceptable
Moderately unacceptable
Totally unacceptable
Results: Results: Novices vs. ExpertsNovices vs. Experts
( p < )
0.0001 -0.0001+0.0001+
0.001+0.002+0.002+0.002+0.006+0.016+0.020 -0.032 -0.080+
0.097 -
Result of Levene-conditioned two-tailed T-test quantifies resolving potential of selected metrics to determine differences between novices and experts.
Results: Results: Construct ValidityConstruct Validity
Performance of Errors
0%5%
10%15%20%25%30%35%40%45%50%
Novice Trainee Expert
SphincterRedoutCapsule
Construct/Concurrent ValidityConstruct/Concurrent Validity (Cross-sectional)(Cross-sectional)
Metrics correlations mimicked the operating room.
Experts and trainees were more efficient at cutting than novices. (P<.01)
Experts were more efficient at coagulating bleeders than trainees who were more efficient than novices. (P<.05, P<.01)
Performance among experts did not exhibit “decay”
Video game experience influenced performance in novices and experts only.
ConclusionsConclusions
Established Face and Content ValidityEstablished Face and Content Validity Established Construct Validity Established Construct Validity (X-sectional)(X-sectional)
Addressed Concurrent Validity Addressed Concurrent Validity (incomplete)(incomplete)
NO assessment predictive validity or NO assessment predictive validity or longitudinal studieslongitudinal studies
- - - - - - For Version 1.0For Version 1.0 - - - - - -
Version 2.0Version 2.0Predictive and Construct ValidationPredictive and Construct Validation
Anatomy ID ModuleAnatomy ID Module
Coagulation ModuleCoagulation Module
Cutting ModuleCutting Module
Advanced Resection Advanced Resection ModuleModule