assessment of clinical skills joseph cacciottolo josanne vassallo university of malta annual...
TRANSCRIPT
![Page 1: Assessment of clinical skills Joseph Cacciottolo Josanne Vassallo UNIVERSITY OF MALTA ANNUAL CONFERENCE - OSLO - MAY 2007](https://reader036.vdocuments.us/reader036/viewer/2022062421/56649d975503460f94a804b7/html5/thumbnails/1.jpg)
Assessment of clinical skills
Joseph Cacciottolo Josanne Vassallo
UNIVERSITY OF MALTA
ANNUAL CONFERENCE - OSLO - MAY 2007
![Page 2: Assessment of clinical skills Joseph Cacciottolo Josanne Vassallo UNIVERSITY OF MALTA ANNUAL CONFERENCE - OSLO - MAY 2007](https://reader036.vdocuments.us/reader036/viewer/2022062421/56649d975503460f94a804b7/html5/thumbnails/2.jpg)
Assessing clinical skills
1. WHY do we need to assess ?
2. WHAT do we want to measure ?
![Page 3: Assessment of clinical skills Joseph Cacciottolo Josanne Vassallo UNIVERSITY OF MALTA ANNUAL CONFERENCE - OSLO - MAY 2007](https://reader036.vdocuments.us/reader036/viewer/2022062421/56649d975503460f94a804b7/html5/thumbnails/3.jpg)
WHY do we assess ?
• To ensure safety of patientsour responsibility to the public
• Achievement of a minimum standardresponsibility to the candidate and University
In principle…
![Page 4: Assessment of clinical skills Joseph Cacciottolo Josanne Vassallo UNIVERSITY OF MALTA ANNUAL CONFERENCE - OSLO - MAY 2007](https://reader036.vdocuments.us/reader036/viewer/2022062421/56649d975503460f94a804b7/html5/thumbnails/4.jpg)
WHY do we assess ?
• To ensure competence
• Often as a means of academic competition
In practice: the scope
![Page 5: Assessment of clinical skills Joseph Cacciottolo Josanne Vassallo UNIVERSITY OF MALTA ANNUAL CONFERENCE - OSLO - MAY 2007](https://reader036.vdocuments.us/reader036/viewer/2022062421/56649d975503460f94a804b7/html5/thumbnails/5.jpg)
WHY do we assess ?
• Formative: to give feedback and advice
• Summative: to grade
• Qualificative or licensing
In practice: the purpose
![Page 6: Assessment of clinical skills Joseph Cacciottolo Josanne Vassallo UNIVERSITY OF MALTA ANNUAL CONFERENCE - OSLO - MAY 2007](https://reader036.vdocuments.us/reader036/viewer/2022062421/56649d975503460f94a804b7/html5/thumbnails/6.jpg)
WHAT do we measure ?
To test not only presence of knowledge
…but also the application of knowledge
In principle…
![Page 7: Assessment of clinical skills Joseph Cacciottolo Josanne Vassallo UNIVERSITY OF MALTA ANNUAL CONFERENCE - OSLO - MAY 2007](https://reader036.vdocuments.us/reader036/viewer/2022062421/56649d975503460f94a804b7/html5/thumbnails/7.jpg)
Several types of clinical assessment
One model presented in some detail to
provide a framework for later discussion
![Page 8: Assessment of clinical skills Joseph Cacciottolo Josanne Vassallo UNIVERSITY OF MALTA ANNUAL CONFERENCE - OSLO - MAY 2007](https://reader036.vdocuments.us/reader036/viewer/2022062421/56649d975503460f94a804b7/html5/thumbnails/8.jpg)
One model of clinical assessment
• Certification of competence - pass / fail a state (and legal) requirement
• Grading in rank orderfor employment / placement purposes
• A competition for the award of a prize
In principle…a three-fold aim
![Page 9: Assessment of clinical skills Joseph Cacciottolo Josanne Vassallo UNIVERSITY OF MALTA ANNUAL CONFERENCE - OSLO - MAY 2007](https://reader036.vdocuments.us/reader036/viewer/2022062421/56649d975503460f94a804b7/html5/thumbnails/9.jpg)
One model of clinical assessment
Measurement of:
• adequacy of basic clinical skills • ability to interpret clinical findings • facility of communication in practical settings• ability to think analytically about diagnosis• ability to discuss management logically
In practice…
![Page 10: Assessment of clinical skills Joseph Cacciottolo Josanne Vassallo UNIVERSITY OF MALTA ANNUAL CONFERENCE - OSLO - MAY 2007](https://reader036.vdocuments.us/reader036/viewer/2022062421/56649d975503460f94a804b7/html5/thumbnails/10.jpg)
Practical steps for assessment
• 6 encounters with different clinical situations
• Two examiners at every encounter, each examiner giving an individual assessment
• Highly structured examination and detailed assessment of skills
• Examiners from other Universities for process evaluation and quality control
![Page 11: Assessment of clinical skills Joseph Cacciottolo Josanne Vassallo UNIVERSITY OF MALTA ANNUAL CONFERENCE - OSLO - MAY 2007](https://reader036.vdocuments.us/reader036/viewer/2022062421/56649d975503460f94a804b7/html5/thumbnails/11.jpg)
Practical steps for assessment
• Encounters with at least six real patients
• Ability to interpret and discuss clinical data
• Management of an emergency scenario
• Appraisal of communication skills and attitude
![Page 12: Assessment of clinical skills Joseph Cacciottolo Josanne Vassallo UNIVERSITY OF MALTA ANNUAL CONFERENCE - OSLO - MAY 2007](https://reader036.vdocuments.us/reader036/viewer/2022062421/56649d975503460f94a804b7/html5/thumbnails/12.jpg)
Set-points for testing
• Attitude to patient
• Actual examination skills
• Presentation of findings
• Clinical judgment
Clinical examination of patients
![Page 13: Assessment of clinical skills Joseph Cacciottolo Josanne Vassallo UNIVERSITY OF MALTA ANNUAL CONFERENCE - OSLO - MAY 2007](https://reader036.vdocuments.us/reader036/viewer/2022062421/56649d975503460f94a804b7/html5/thumbnails/13.jpg)
Set-points for testing
• Evaluation of data
• Significance of data
• Clinical reasoning
Interpretation of clinical data
![Page 14: Assessment of clinical skills Joseph Cacciottolo Josanne Vassallo UNIVERSITY OF MALTA ANNUAL CONFERENCE - OSLO - MAY 2007](https://reader036.vdocuments.us/reader036/viewer/2022062421/56649d975503460f94a804b7/html5/thumbnails/14.jpg)
Set-points for testing
• Ability to solve problems
• Ability to discuss logically
• Clinical judgment and prioritization
• General medical knowledge
Management of emergency situations
![Page 15: Assessment of clinical skills Joseph Cacciottolo Josanne Vassallo UNIVERSITY OF MALTA ANNUAL CONFERENCE - OSLO - MAY 2007](https://reader036.vdocuments.us/reader036/viewer/2022062421/56649d975503460f94a804b7/html5/thumbnails/15.jpg)
Set-points for testing
• Attitude to ‘patient’
• Ability to communicate well
• Clinical judgment
• General medical knowledge
Communication skills
![Page 16: Assessment of clinical skills Joseph Cacciottolo Josanne Vassallo UNIVERSITY OF MALTA ANNUAL CONFERENCE - OSLO - MAY 2007](https://reader036.vdocuments.us/reader036/viewer/2022062421/56649d975503460f94a804b7/html5/thumbnails/16.jpg)
Evaluation
• Subjective evaluation: looking in the mirror • Objective evaluation of outcome
![Page 17: Assessment of clinical skills Joseph Cacciottolo Josanne Vassallo UNIVERSITY OF MALTA ANNUAL CONFERENCE - OSLO - MAY 2007](https://reader036.vdocuments.us/reader036/viewer/2022062421/56649d975503460f94a804b7/html5/thumbnails/17.jpg)
Evaluating ourselves
“Examinations are formidable even to the best prepared…
…for the greatest fool may ask more than the wisest man can answer”
Charles Caleb Colton 1780 - 1832
![Page 18: Assessment of clinical skills Joseph Cacciottolo Josanne Vassallo UNIVERSITY OF MALTA ANNUAL CONFERENCE - OSLO - MAY 2007](https://reader036.vdocuments.us/reader036/viewer/2022062421/56649d975503460f94a804b7/html5/thumbnails/18.jpg)
Evaluating outcome
• Pattern of results
• Consistency of results
• Patterns of marking
• Process shortcomings
• Basis for improvement
Analysis of data to assess effectiveness
![Page 19: Assessment of clinical skills Joseph Cacciottolo Josanne Vassallo UNIVERSITY OF MALTA ANNUAL CONFERENCE - OSLO - MAY 2007](https://reader036.vdocuments.us/reader036/viewer/2022062421/56649d975503460f94a804b7/html5/thumbnails/19.jpg)
What happens to candidates who fail ?
• Review of performance – a formative exercise
• Counselling at a personal level
• Specific attention and individual training
• Repeat assessment after a period of time
![Page 20: Assessment of clinical skills Joseph Cacciottolo Josanne Vassallo UNIVERSITY OF MALTA ANNUAL CONFERENCE - OSLO - MAY 2007](https://reader036.vdocuments.us/reader036/viewer/2022062421/56649d975503460f94a804b7/html5/thumbnails/20.jpg)
Points for discussion
• Competence versus performance
• Exam based versus continuous assessment
• Methodology related issues
• Organisational issues
![Page 21: Assessment of clinical skills Joseph Cacciottolo Josanne Vassallo UNIVERSITY OF MALTA ANNUAL CONFERENCE - OSLO - MAY 2007](https://reader036.vdocuments.us/reader036/viewer/2022062421/56649d975503460f94a804b7/html5/thumbnails/21.jpg)
Competence versus Performance
• In relation to methods used
• Use of core curriculum with additional modules as required
• Eventual medical practice in different environment
• Impact on mobility
![Page 22: Assessment of clinical skills Joseph Cacciottolo Josanne Vassallo UNIVERSITY OF MALTA ANNUAL CONFERENCE - OSLO - MAY 2007](https://reader036.vdocuments.us/reader036/viewer/2022062421/56649d975503460f94a804b7/html5/thumbnails/22.jpg)
Competence versus Performance
• In relation to methods used
• Use of core curriculum with additional modules as required
• Eventual medical practice in different environment
• Impact on mobility
![Page 23: Assessment of clinical skills Joseph Cacciottolo Josanne Vassallo UNIVERSITY OF MALTA ANNUAL CONFERENCE - OSLO - MAY 2007](https://reader036.vdocuments.us/reader036/viewer/2022062421/56649d975503460f94a804b7/html5/thumbnails/23.jpg)
Competence versus Performance
• In relation to methods used
• Use of core curriculum with additional modules as required
• Eventual medical practice in different environment
• Impact on mobility
![Page 24: Assessment of clinical skills Joseph Cacciottolo Josanne Vassallo UNIVERSITY OF MALTA ANNUAL CONFERENCE - OSLO - MAY 2007](https://reader036.vdocuments.us/reader036/viewer/2022062421/56649d975503460f94a804b7/html5/thumbnails/24.jpg)
Competence versus Performance
• In relation to methods used
• Use of core curriculum with additional modules as required
• Eventual medical practice in different environment
• Impact on mobility
![Page 25: Assessment of clinical skills Joseph Cacciottolo Josanne Vassallo UNIVERSITY OF MALTA ANNUAL CONFERENCE - OSLO - MAY 2007](https://reader036.vdocuments.us/reader036/viewer/2022062421/56649d975503460f94a804b7/html5/thumbnails/25.jpg)
Exam based vs continuous assessment
• Assessment of a modular curriculum
• Written exams: MCQs, SAQ, Essays
• Real-time exams
• Orals: clinical / table viva, OSCEs
• Clinical assessment on site: mini-CEX, DOPS, multi-source feedback
• Log books, portofolios, CATs
![Page 26: Assessment of clinical skills Joseph Cacciottolo Josanne Vassallo UNIVERSITY OF MALTA ANNUAL CONFERENCE - OSLO - MAY 2007](https://reader036.vdocuments.us/reader036/viewer/2022062421/56649d975503460f94a804b7/html5/thumbnails/26.jpg)
Exam based vs continuous assessment
• Assessment of a modular curriculum
• Written exams: MCQs, SAQ, Essays
• Real-time exams
• Orals: clinical / table viva, OSCEs
• Clinical assessment on site: mini-CEX, DOPS, multi-source feedback
• Log-books, portfolios, CATs
![Page 27: Assessment of clinical skills Joseph Cacciottolo Josanne Vassallo UNIVERSITY OF MALTA ANNUAL CONFERENCE - OSLO - MAY 2007](https://reader036.vdocuments.us/reader036/viewer/2022062421/56649d975503460f94a804b7/html5/thumbnails/27.jpg)
Methodology-related issues
• Reliability
• Validity
• Practicality
• Transparency
![Page 28: Assessment of clinical skills Joseph Cacciottolo Josanne Vassallo UNIVERSITY OF MALTA ANNUAL CONFERENCE - OSLO - MAY 2007](https://reader036.vdocuments.us/reader036/viewer/2022062421/56649d975503460f94a804b7/html5/thumbnails/28.jpg)
Organisational issues
• Resources available
• Student numbers
• Patient availability
• Invasive procedures
• Scheduling
![Page 29: Assessment of clinical skills Joseph Cacciottolo Josanne Vassallo UNIVERSITY OF MALTA ANNUAL CONFERENCE - OSLO - MAY 2007](https://reader036.vdocuments.us/reader036/viewer/2022062421/56649d975503460f94a804b7/html5/thumbnails/29.jpg)
Role of grading
• Is there need for grading ?
• Selection process for employment
• Selection for postgraduate training
• Quality of assessment method and performance of candidates
![Page 30: Assessment of clinical skills Joseph Cacciottolo Josanne Vassallo UNIVERSITY OF MALTA ANNUAL CONFERENCE - OSLO - MAY 2007](https://reader036.vdocuments.us/reader036/viewer/2022062421/56649d975503460f94a804b7/html5/thumbnails/30.jpg)
Conclusions
There is wide diversity among European medical schoolsregarding methods of assessment of clinical skills:
• some schools aim at pass/fail outcomes, others use systems that lead to grading
• several quality assurance mechanisms are used to varying degrees
There is place for widespread application of agreedstandard methods to:
• assess clinical competences in core curricula • assess additional competences essential to individual practice
![Page 31: Assessment of clinical skills Joseph Cacciottolo Josanne Vassallo UNIVERSITY OF MALTA ANNUAL CONFERENCE - OSLO - MAY 2007](https://reader036.vdocuments.us/reader036/viewer/2022062421/56649d975503460f94a804b7/html5/thumbnails/31.jpg)
UNIVERSITY OF MALTA MEDICAL SCHOOL
1676 - 2007
EU UNIVERSITY PARTNER
![Page 32: Assessment of clinical skills Joseph Cacciottolo Josanne Vassallo UNIVERSITY OF MALTA ANNUAL CONFERENCE - OSLO - MAY 2007](https://reader036.vdocuments.us/reader036/viewer/2022062421/56649d975503460f94a804b7/html5/thumbnails/32.jpg)
Thank you
ANNUAL CONFERENCE - OSLO - MAY 2007