use of simulation for assessment in anesthesiology · · 2015-01-28use of simulation for...
TRANSCRIPT
10/16/2012
1
Use of simulation
for assessment
in anesthesiology
Marguerite Dupré, MD, FRCPC, DESS (bioethics)
Johanne Thiffault, MSc (statistics)
François Goulet, MD, FCMFC, MA (education)
Practice Enhancement Division
Collège des médecins du Québec
10th International Conference on Medical Regulation
2012 Meeting – October 2-5, Ottawa, Canada
“We have no actual or potential
conflict of interest in relation
to this presentation”
Marguerite Dupré, MD
Johanne Thiffault, MSc
Disclosure statement
François Goulet, MD
10/16/2012
2
1
The structured
oral interview in
anesthesiology
2
Simulation
3
A closer look at
Quebec data
4
Discussion and
conclusion
PRESENTATION PLAN
Photo: © Andrew Olney/Getty Images
Section 2
Section 3
The structured oral interview
in anesthesiology
Section 1
Section 4
10/16/2012
3
Historical perspective
1990 : SOI in Family practice
1997 : Introduction of standardized patients
2002 : SOI in Diagnostic radiology
2006 : SOI in Psychiatry
SOI in Pathology
2008 : SOI in General surgery
2009 : SOI in Dermatology
SOI in anesthesiology
2004 : Death of a 51-year-old female patient
post esthetic surgery
2006 : Coroner’s recommandation: assessment
of performance in emergency situations
using high-fidelity simulators
2008 : Working group
2010 : External review
2011 : Pilot tests (2)
First SOI candidates (4)
2012 : 3 SOI candidates
10/16/2012
4
SOI in anesthesiology
Evaluation ≠ exam
Evaluation done by two peers
Length: one day
Standardized clinical vignettes based on
key-features approach
12 cases, including 4-5 with simulation
Menu based on practice characteristics
SOI in anesthesiology
Introductory visit: 40 minutes
Practice case and debriefing: 20 minutes
No debriefing for the next 12 cases
Global appreciation: strengths and weaknesses
Parametric analysis
Final report
Peer review Committee
10/16/2012
5
SOI in anesthesiology
Domains
Number of cases
TOTAL Simulation
General anesthesia 6 2
Loco-regional anesthesia 4 1
Difficult airway 7 2
Pediatrics 3
Obstetrics 5
Traumatology 5 2
Post-op analgesia 3 1
Emergencies (anaphylaxis, arythmia,
malignant hyperthermia) 5 2
Equipment 2 1
Evaluated dimensions
Preoperative evaluation
Anesthetic plan
Management of complications
Critical care and resuscitation
Postoperative conduct and analgesia
Patient communication and consent
10/16/2012
6
Evaluated dimensions
Preoperative evaluation
Anesthetic plan
Management of complications
Critical care and resuscitation
Postoperative conduct and analgesia
Patient communication and consent
Section 4
Section 3
Section 1
Simulation
Section 2
10/16/2012
7
Simulation
Case ALR-01 : Blood patch
10/16/2012
8
Case ALR-01 : Blood patch
Case ALR-01 : Blood patch
10/16/2012
9
Case AU-04 : 3rd degree AV block
Case AU-04 : 3rd degree AV block
10/16/2012
10
Case AU-04 : 3rd degree AV block
Case AU-04 : 3rd degree AV block
10/16/2012
11
Section 4
Section 1
Section 2
A closer look at Quebec data
Section 3
Candidates profile
Sex Age Type of practice Reason for SOI
M 75 Locum Reported by colleague
M 49 Hospital IMG
M 33 Hospital IMG
F 64 Locum Locum
M 73 Locum Complaint
M 43 Locum IMG
M 58 Hospital Peer review
10/16/2012
12
Sample parametric analysis
Parametric analysis by dimension
for the 7 SOI candidates
10/16/2012
13
SOI results
SUCCESS rate
Age Origin % cases* # dimensions
(%)
Recommandation
following SOI
Candidate’s
decision
75 Reported by
colleague 45% 1 (17%)
Full-time training –
3 months with limitation Retired
49 IMG 83% 5 (83%) APLS + ATLS workshops Attended
workshops
33 IMG 83% 5 (83%) CPD plan Submitted
CPD plan
64 Locum 83% 4 (67%) CPD plan CPD plan not
yet submitted
73 Inquiry 14% 0 (0%) Full-time training –
6 months with limitation Retired
43 IMG 75% 4 (67%) Recommandations na
58 Reported by
colleague 100% 6 (100%) Recommandations na
* Passing grade = 70%
Cases and candidates analysis
(preliminary findings)
Two cases (AT-01, AU-01) have a higher degree of difficulty
4/4 candidates failed case AT-01 (Patient with unstable spine –
simulation case)
5/7 candidates failed case AU-01 (Anaphylaxis)
The dimension Critical care and resuscitation has a higher
degree of difficulty (5/7 candidates failed that dimension)
Candidates’ weaknesses become clearly apparent in
simulation cases, especially those involving critical care
10/16/2012
14
Section 3
Section 1
Section 2
Discussion and conclusion
Section 4
Discussion
Assessment
Knowledge
Skills
Procedural
Non technical
– Communication
– Situation awareness
– Professionalism
– Team work
10/16/2012
15
What can be measured ?
Simulation – Clinical performance
Prepare and plan sequentially
Efficiently combine steps to induce anesthesia
Maintain vigilance
Interpret monitoring data
Remain situationally aware
Conduct a rapid logical assessment
Make swift decisions
What can be measured ?
Simulation – Communication skills
Listen effectively
Talk about patient management options
Discuss and disclose risks
Elicit information
Build rapport
10/16/2012
16
What can be measured ?
Simulation – Professionalism
Respectful
Keeping the best interest of patient at heart
Accepting personal errors
Organization
Preparedness
Candidates’ opinions and
impressions
Well-received
Long and tiring day
Stressful
Relevant
Realistic
10/16/2012
17
Important considerations
Relevance to practice (matching the
assessment content to the candidate’s practice profile)
Key-features approach Bias
Context effect
Number of cases
Fatigue
Generalizability
Costs and personnel resources
Conclusion
Different types of assessment are needed
Simulation allows for assessment of procedural and non
technical skills
Simulation assesses the highest level of Miller’s pyramid
Assessment of recognition of critical events and their
management
Assessment of performance and behavior
Weaker candidates have difficulties across dimensions
10/16/2012
18
Collège des médecins du Québec
2170, boul. René-Lévesque West
Montréal (Québec)
Canada H3H 2T8
514.933.4441 #5499
514.933.4668
QUESTIONS?