assessment of professionals m. schürch. how do you assess performance? how do you currently assess...
TRANSCRIPT
Assessment of Professionals
M. Schürch
How do you assess performance?
• How do you currently assess the performance of your residents?
• What standards do you apply?
• What evidence can you provide?
• Is the evidence robust enough?
• Could the resident challenge your assessment?
Outline
• Principles of assessment
• Methods of assessment
• What standards do apply
• Assessment in the new curriculum
• Conclusion
Why assess?
• Provide direction and motivation for future learning: assessment drives learning …
• Screening out those who are incompetent
• Meet public expectations
• Choose applicants for advanced training
What to assess?
• Knowledge
• Skills– technical– non-technical
• Attitude
How to assess?• Here is the problem• Depends on why and what!
Formative vs summative assessment• Formative assessment
– guides future learning– provides reassurance– promotes reflection– shapes values
• Summative assessment– overall jugement of
competence– fitness to practice– qualification for advancement
can: reinforce intrinsic motivationinspire to set higher standards
Formative vs summative assessment• Formative assessment
– guides future learning– provides reassurance– promotes reflection– shapes values
• Summative assessment– overall jugement of
competence– fitness to practice– qualification for advancement
can: reinforce intrinsic motivationinspire to set higher standards
Methods of assessment
• Simulation– standardized patients– computer based simulation
computer programms, high fidelty simulators
• Work-based assessment– case based discussion– mini-clinical evaluation exercise (mCEX)– direct observation of … (DOPS, DOCE)– multisource (360°) assessments
Hierarchy of professionalism
Competent Knowledge
Technical Skills
Non-Technical Skills
Pro-fessional Behaviour
MCQ
VIVA
OSCE
VIVA
MCQ: multiple choice questionsOSCE: objective structured
clinical examinationVIVA: oral exam
Hierarchy of professionalism
Competent Knowledge
Technical Skills
Non-Technical Skills
Pro-fessional Behaviour
mini-CEX
CBD
DOCE
MSF (RET?)
mini-CEX: clinical evaluation exerciseCBD: case based discussionDOCE: direct observation of
clinical encounterMSF: multi source feedback
Criteria to choose methods
• Reliability accurate, reproducible
• Validity measures what it claims to
• Impact on future learning and practice
• Acceptability to learners and facultyfair, feasible
• Costs trainee, institution, society
What standards do apply?
• Limen referenced– Written exam, MCQ– Standardized patients, simulation
• Relying on assessors judgement of competency against expectation at a particular level of training– Workplace based assessment
• Patient difficulty• Faculty stringency• Equivalence of assessments?
Challanges of work based methods• No absolute standards
– most trainees will be satisfactory
• Alternative assessments for those who are unsatisfactory
• Importance of selection of assessors• Equivalence
– not useful for ranking TraineesMultiple assessments by multiple
assessors
Who should assess?
„ … one of the capabilities of a competent individual is the skill of recognising competence in
others.“
Ronald Burnett: The Limits of Competence 1991
Assessment in the new Curriculum
www.sgar-ssar.ch
defines competencies and standards
Assessments in the curriculum Fo
rmati
ve
Su
mm
ati
ve
Formative assessments
DOCE
RET (Resident Evaluation Tool)
• the Resident Evaluation Tool RET (Anästhesie Basel, Abbott) can be used to give Multisource-Feedback
• RET combines items of mini-CEX and ANTS (non technical Skills)
Example of a rating, that needs an action plan
Example of an excellent rating
Conclusion
• Assessment is essential for learning• Good assessment is a challenge
„ … one of the capabilities of a competent individual is the skill of recognising competence in others.“
Ronald Burnett, 1991