new challenges in assessment of clinical competence
DESCRIPTION
Neil Spike Director of Training & Assessment School of Primary Health Care Monash University, Australia. New challenges in assessment of clinical competence. Answers?. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: New challenges in assessment of clinical competence](https://reader035.vdocuments.us/reader035/viewer/2022062308/56812d0d550346895d91e321/html5/thumbnails/1.jpg)
www.monash.edu.au
Neil SpikeDirector of Training & AssessmentSchool of Primary Health CareMonash University, Australia
New challenges in assessment of clinical competence
![Page 2: New challenges in assessment of clinical competence](https://reader035.vdocuments.us/reader035/viewer/2022062308/56812d0d550346895d91e321/html5/thumbnails/2.jpg)
www.monash.edu.au
2
Answers?
We’re not actually going to tell you how to play the game; that’s something you’ll have to figure out for yourself. After all the object of the game is to discover The Meaning of Life, and life does not come with an instruction book. If it did, you’d be wasting all your days in the troubleshooting section, hoping to figure out where you went wrong. That would be no fun.
Manual for Monty Python’s Meaning of Life CD-Rom game.
![Page 3: New challenges in assessment of clinical competence](https://reader035.vdocuments.us/reader035/viewer/2022062308/56812d0d550346895d91e321/html5/thumbnails/3.jpg)
www.monash.edu.au
3
Answers?
If you have the same ideas as everybody else but have them one week earlier than everyone else then you will be hailed as a visionary. But if you have them five years earlier, you will be called a lunatic.
Barry Jones
![Page 4: New challenges in assessment of clinical competence](https://reader035.vdocuments.us/reader035/viewer/2022062308/56812d0d550346895d91e321/html5/thumbnails/4.jpg)
www.monash.edu.au
4
Questions?
Confidence never comes from having all the answers; it comes from being open to all the questions.
– Earl Gary Stevens
![Page 5: New challenges in assessment of clinical competence](https://reader035.vdocuments.us/reader035/viewer/2022062308/56812d0d550346895d91e321/html5/thumbnails/5.jpg)
www.monash.edu.au
5
Overview
• Other parameters of utility• Changes in students• Changes in academic roles• Performance assessment• General professional competencies
![Page 6: New challenges in assessment of clinical competence](https://reader035.vdocuments.us/reader035/viewer/2022062308/56812d0d550346895d91e321/html5/thumbnails/6.jpg)
www.monash.edu.au
6
Overview
• Other parameters of utility• Changes in students• Changes in academic roles• Performance assessment• General professional competencies
![Page 7: New challenges in assessment of clinical competence](https://reader035.vdocuments.us/reader035/viewer/2022062308/56812d0d550346895d91e321/html5/thumbnails/7.jpg)
www.monash.edu.au
7
Utility function
• U = Utility
• R = Reliability
• V = Validity
• E = Educational impact
• A = Acceptability
• C = Cost
• W = Weight
U = wrR x wv
V x weE x wa
A x wcC
Adapted from Cees van der Vleuten
![Page 8: New challenges in assessment of clinical competence](https://reader035.vdocuments.us/reader035/viewer/2022062308/56812d0d550346895d91e321/html5/thumbnails/8.jpg)
www.monash.edu.au
8
Utility function
• U = Utility
• R = Reliability
• V = Validity
• E = Educational impact
• A = Acceptability• C = Cost• W = Weight
U = wrR x wv
V x weE x wa
A x wcC
![Page 9: New challenges in assessment of clinical competence](https://reader035.vdocuments.us/reader035/viewer/2022062308/56812d0d550346895d91e321/html5/thumbnails/9.jpg)
www.monash.edu.au
9
Utility function
• Introduce a new factor:– Feasibility
![Page 10: New challenges in assessment of clinical competence](https://reader035.vdocuments.us/reader035/viewer/2022062308/56812d0d550346895d91e321/html5/thumbnails/10.jpg)
www.monash.edu.au
10
![Page 11: New challenges in assessment of clinical competence](https://reader035.vdocuments.us/reader035/viewer/2022062308/56812d0d550346895d91e321/html5/thumbnails/11.jpg)
www.monash.edu.au
11
![Page 12: New challenges in assessment of clinical competence](https://reader035.vdocuments.us/reader035/viewer/2022062308/56812d0d550346895d91e321/html5/thumbnails/12.jpg)
www.monash.edu.au
12
![Page 13: New challenges in assessment of clinical competence](https://reader035.vdocuments.us/reader035/viewer/2022062308/56812d0d550346895d91e321/html5/thumbnails/13.jpg)
www.monash.edu.au
13
Overview
• Other parameters of utility• Changes in students• Changes in academic roles• Performance assessment• General professional competencies
![Page 14: New challenges in assessment of clinical competence](https://reader035.vdocuments.us/reader035/viewer/2022062308/56812d0d550346895d91e321/html5/thumbnails/14.jpg)
www.monash.edu.au
14
Universiteit Maastricht
Student-centred assessment
- self-assessment
- peer-assessment
- co-assessment
self-evaluation skills
assessing the unmeasurable
assessing the unmeasurable
peer pressure?
peer-evaluation skills
self, peer and co-evaluation skills
peer pressure?
joint responsibility
![Page 15: New challenges in assessment of clinical competence](https://reader035.vdocuments.us/reader035/viewer/2022062308/56812d0d550346895d91e321/html5/thumbnails/15.jpg)
www.monash.edu.au
15
Student changes
• Increasing diversity– Ethnicity
– Socio-economic status
– Previous school/educational experience
– Equity categories
– Basis of admission
Adapted from Frank Tesoriero
![Page 16: New challenges in assessment of clinical competence](https://reader035.vdocuments.us/reader035/viewer/2022062308/56812d0d550346895d91e321/html5/thumbnails/16.jpg)
www.monash.edu.au
16
Student changes
• Student diversity means that previous assumptions on student characteristics may no longer hold true– Common values
– Preferred learning styles
– Levels of language proficiency
![Page 17: New challenges in assessment of clinical competence](https://reader035.vdocuments.us/reader035/viewer/2022062308/56812d0d550346895d91e321/html5/thumbnails/17.jpg)
www.monash.edu.au
17
Student centredness
• Early needs assessment• Individual learning plans & portfolios• Changes to staffing ratios• Professional development for students• Increased time for increased assessments• Collating numerous assessments• Translating the standard into a cut score
![Page 18: New challenges in assessment of clinical competence](https://reader035.vdocuments.us/reader035/viewer/2022062308/56812d0d550346895d91e321/html5/thumbnails/18.jpg)
www.monash.edu.au
18
![Page 19: New challenges in assessment of clinical competence](https://reader035.vdocuments.us/reader035/viewer/2022062308/56812d0d550346895d91e321/html5/thumbnails/19.jpg)
www.monash.edu.au
19
Overview
• Other parameters of utility• Changes in students• Changes in academic roles• Performance assessment• General professional competencies
![Page 20: New challenges in assessment of clinical competence](https://reader035.vdocuments.us/reader035/viewer/2022062308/56812d0d550346895d91e321/html5/thumbnails/20.jpg)
www.monash.edu.au
20
Universiteit Maastricht
SuggestionsAssessment is not mainly a measurement problem but more an educational design problem.
- built-in quality control procedures and bodies
- sampling in stead of (over) structuring
- variation of methods, compensation, regulations
- saturation of information approach
- training of students, teachers, examiners, board members
Masterplan of the assessment system
![Page 21: New challenges in assessment of clinical competence](https://reader035.vdocuments.us/reader035/viewer/2022062308/56812d0d550346895d91e321/html5/thumbnails/21.jpg)
www.monash.edu.au
21
Academic changes
• In addition to traditional roles:– Research– Clinical service– Teaching– Consultancy
• Management of casualised teaching & assessment teams
![Page 22: New challenges in assessment of clinical competence](https://reader035.vdocuments.us/reader035/viewer/2022062308/56812d0d550346895d91e321/html5/thumbnails/22.jpg)
www.monash.edu.au
22
Academic changes
• Need to maintain program integrity when teachers are casual and teach outside of a full program context.
• Need to adequately select, induct, support and manage of casual staff both individually and collectively.
• Need to engage casual staff who are often at margins of academic life, in the teaching and learning debates which have significant impact on them, their teaching and their students.
Adapted from Frank Tesoriero
![Page 23: New challenges in assessment of clinical competence](https://reader035.vdocuments.us/reader035/viewer/2022062308/56812d0d550346895d91e321/html5/thumbnails/23.jpg)
www.monash.edu.au
23
Overview
• Other parameters of utility• Changes in students• Changes in academic roles• Performance assessment• General professional competencies
![Page 24: New challenges in assessment of clinical competence](https://reader035.vdocuments.us/reader035/viewer/2022062308/56812d0d550346895d91e321/html5/thumbnails/24.jpg)
www.monash.edu.au
24
Universiteit MaastrichtBroaden the measurement time frame
Specific test occasions
Longitudinal testing
- specific preparation of students
- inflexibility
- unwanted influence on learning
- certain aspects unmeasurable
- no specific preparation - congruent with flexible curricula
- wanted influence on learning
- more performance assessment
![Page 25: New challenges in assessment of clinical competence](https://reader035.vdocuments.us/reader035/viewer/2022062308/56812d0d550346895d91e321/html5/thumbnails/25.jpg)
www.monash.edu.au
25
Performance Assessment
• Indirect methods– Chart audit– Practice audits– Prescription data………
• Direct methods– Clinical (supervisor) ratings– Masked standardized patients– Video assessment– Patient reports– Peer reports– Clinical work samples………
![Page 26: New challenges in assessment of clinical competence](https://reader035.vdocuments.us/reader035/viewer/2022062308/56812d0d550346895d91e321/html5/thumbnails/26.jpg)
www.monash.edu.au
26
Overview
• Other parameters of utility• Changes in students• Changes in academic roles• Performance assessment• General professional competencies
![Page 27: New challenges in assessment of clinical competence](https://reader035.vdocuments.us/reader035/viewer/2022062308/56812d0d550346895d91e321/html5/thumbnails/27.jpg)
www.monash.edu.au
27
Universiteit Maastricht
General professional competencies
dealing with tasks
dealing with others
dealing with oneself
planning and time management
information gathering
co-operation
communicating
giving feedback and criticising
self-responsibility
self-evaluation
self-reflection
![Page 28: New challenges in assessment of clinical competence](https://reader035.vdocuments.us/reader035/viewer/2022062308/56812d0d550346895d91e321/html5/thumbnails/28.jpg)
www.monash.edu.au
28
The aim of education is the knowledge not of facts but of values.
William Ralph Inge
![Page 29: New challenges in assessment of clinical competence](https://reader035.vdocuments.us/reader035/viewer/2022062308/56812d0d550346895d91e321/html5/thumbnails/29.jpg)
www.monash.edu.au
29
Professional competencies
Quality assurance focus
• Traditionally includes:– The curriculum– The teaching and learning process– The assessment of competence
• Should also include:– Monitoring– Evaluation– Staff development– “Fitness for purpose”
![Page 30: New challenges in assessment of clinical competence](https://reader035.vdocuments.us/reader035/viewer/2022062308/56812d0d550346895d91e321/html5/thumbnails/30.jpg)
www.monash.edu.au
30
Where do we start?
‘Would you tell me please, which way I ought to go from here?’
‘That depends a good deal on where you want to get to,’ said the Cat.
‘I don’t much care where’ said Alice.‘Then it doesn’t matter which way
you go,’ said the Cat.
Alice’s Adventures in Wonderland - Lewis Carroll
![Page 31: New challenges in assessment of clinical competence](https://reader035.vdocuments.us/reader035/viewer/2022062308/56812d0d550346895d91e321/html5/thumbnails/31.jpg)
www.monash.edu.au
31