assessment in gp/fm
DESCRIPTION
ASSESSMENT IN GP/FM. Mladenka Vrcic-Keglevic, MD, PhD Head of Family Medicine Department, Medical School, Univerity of Zagreb [email protected]. Content of the session. Definition of the terms How to use assessment (summative, formative) Assessment methods (AM) - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: ASSESSMENT IN GP/FM](https://reader035.vdocuments.us/reader035/viewer/2022062305/5681587a550346895dc5dbce/html5/thumbnails/1.jpg)
ASSESSMENT IN GP/FM
Mladenka Vrcic-Keglevic, MD, PhD
Head of Family Medicine Department, Medical School, Univerity of Zagreb
![Page 2: ASSESSMENT IN GP/FM](https://reader035.vdocuments.us/reader035/viewer/2022062305/5681587a550346895dc5dbce/html5/thumbnails/2.jpg)
Content of the session
• Definition of the terms• How to use assessment (summative,
formative)• Assessment methods (AM)• What can be assessed by different AM• Metric’s characteristics of AM• …..• Interactive session
![Page 3: ASSESSMENT IN GP/FM](https://reader035.vdocuments.us/reader035/viewer/2022062305/5681587a550346895dc5dbce/html5/thumbnails/3.jpg)
Definition of the terms
• Evaluation: A process that attempts to systematically determine the relevance, effectiveness, and impact of activities in light of their objectives. Can be related to structure, process or outcome of (programme, person).
• Assessment: A system of evaluation of professional accomplishments in light of educational objectives, using defined criteria and usually including an attempt at measurement either by grading on a rough scale or by assigning numerical values.
AMEE – Glosary of Medical Education Terms
![Page 4: ASSESSMENT IN GP/FM](https://reader035.vdocuments.us/reader035/viewer/2022062305/5681587a550346895dc5dbce/html5/thumbnails/4.jpg)
Well defined educational objectives - prerequsite for assessment
Example for this session:
At the end of this session participants will be able:• To named at list three differences between
summative and formative assessment• To make a list of at least three written AM• To name the most effective AM to assess clinical
skills• To describe the most effective AM to assess
attitudes
![Page 5: ASSESSMENT IN GP/FM](https://reader035.vdocuments.us/reader035/viewer/2022062305/5681587a550346895dc5dbce/html5/thumbnails/5.jpg)
How to use assessment?
• Summative: This is usually undertaken at the end of a training programme or teaching course and determines whether the educational objectives have been successfully achieved. With summative assessment the students usually receives a grade or a mark. Exam
• Formative: This is testing that is part of developmental or ongoing teaching / learning process. It should emanate from a wish to foster learning and understanding. It should include delivery of feedback to the student.
![Page 6: ASSESSMENT IN GP/FM](https://reader035.vdocuments.us/reader035/viewer/2022062305/5681587a550346895dc5dbce/html5/thumbnails/6.jpg)
Summative - Examination
What the exams are?
• For students – A difficult and unpleasent steeplechase to run on a way for diploma
• For teachers – A less desirable teaching activity
• For public – An important protection from un-competent doctors
![Page 7: ASSESSMENT IN GP/FM](https://reader035.vdocuments.us/reader035/viewer/2022062305/5681587a550346895dc5dbce/html5/thumbnails/7.jpg)
Summative assessment
The reasons:• A statement of achievement - university degree
(diploma)• An entrance requirement to an educational
institution• A guide as to the wisdom of continuing with
further study• A certification of competence – public
responsibility (licence)• A determinant of programme effectiveness
![Page 8: ASSESSMENT IN GP/FM](https://reader035.vdocuments.us/reader035/viewer/2022062305/5681587a550346895dc5dbce/html5/thumbnails/8.jpg)
Assessment methods (AM)
Written:• MCQ - Multiple Choice
Questions
• Un-Q – An-cued Questions
• Essay• MEQ – Modified Essay
Questions
• EMI - extended matching items
• PMP – Patient Management Problem
• Oral, viva• Traditional practical• OSCE – Objective
Structured Clinical Examination
• SP - Standardised Patient
• Portfolio• ACC – Assessment of
Clinical Competence
![Page 9: ASSESSMENT IN GP/FM](https://reader035.vdocuments.us/reader035/viewer/2022062305/5681587a550346895dc5dbce/html5/thumbnails/9.jpg)
What can be assessed by different AM
• Factual knowledge
• Interpretations
• Problem-solving skills
• Ethical
• Clinical skills
• Emotional reactions
• Communication
![Page 10: ASSESSMENT IN GP/FM](https://reader035.vdocuments.us/reader035/viewer/2022062305/5681587a550346895dc5dbce/html5/thumbnails/10.jpg)
What can be assessed?
Fact Inter P-S Ethi Skill Emo Com
Oral ++ + +/- - - - - + +
Practic. + ++ ++ + - + + - + -
MCQ ++ + - + - - - - - - - - -
Un-Q ++ + - + - - - - - - - - -
MEQ + ++ ++ + - - - - - - -
Essay ++ + - + - + - + - - - - -
OSCE + + + + - + - ++ + - + -
PMP +- ++ ++ +- - - - - - -
ACC ++ ++ ++ ++ ++ ++ ++
![Page 11: ASSESSMENT IN GP/FM](https://reader035.vdocuments.us/reader035/viewer/2022062305/5681587a550346895dc5dbce/html5/thumbnails/11.jpg)
Metric characteristics of AM
• Validity - the degree to which a measurement instrument truly measures what it is intended to measure
• Reliability – it is an expression of the precision, consistency and reproducibility. Ideally, measurements should be the same when repeated by the same student or made by the different assessors.
• Relevance – it is a degree to which the assessment
questions and educational objectives are in concordance • Objectivity – it is a level of agreement among independent
assessors (experts) about the right answer to certain question
![Page 12: ASSESSMENT IN GP/FM](https://reader035.vdocuments.us/reader035/viewer/2022062305/5681587a550346895dc5dbce/html5/thumbnails/12.jpg)
Metric’s characteristics?
Validity Reliability Relevent Objectivity
Oral - - - - - - - -
Practic. - - + -
MCQ ++ ++ ++ ++
Un-Q ++ ++ ++ ++
MEQ ++ ++ ++ ++
Essay - - + -
OSCE ++ ++ ++ ++
PMP/SP ++ ++ ++ ++
ACC - + - + + - +
![Page 13: ASSESSMENT IN GP/FM](https://reader035.vdocuments.us/reader035/viewer/2022062305/5681587a550346895dc5dbce/html5/thumbnails/13.jpg)
Practicality?
• Preparation – resources (time, human…)
• Easy to perform, aplay
• NC – number of candidates
• NC/teacher – number of candidates per one teacher
![Page 14: ASSESSMENT IN GP/FM](https://reader035.vdocuments.us/reader035/viewer/2022062305/5681587a550346895dc5dbce/html5/thumbnails/14.jpg)
Practicality
Preparation Ease to applay
NC- No of candidates
NC / teacher
Oral ++ - - - - - -
Practic. ++ + - - - -
MCQ +++ ++ ++ ++
Un-Q +++ ++ ++ ++
MEQ - ++ ++ -
Essay + + + - -
OSCE + + + +
PMP/SP + + - ++ +
ACC ++ + - - - -
![Page 15: ASSESSMENT IN GP/FM](https://reader035.vdocuments.us/reader035/viewer/2022062305/5681587a550346895dc5dbce/html5/thumbnails/15.jpg)
MCQ – Multiple Choice Questions
• Difficulty index
• Discrimination index
• Minimum pass level
![Page 16: ASSESSMENT IN GP/FM](https://reader035.vdocuments.us/reader035/viewer/2022062305/5681587a550346895dc5dbce/html5/thumbnails/16.jpg)
Indeks of difficultyQD = (G + P)/N
• QD – question’s difficulty• G – number of right
answers in good group• P – number of right
answers in poor group• N – number of students in
both groups
a b c d e 0 U
1
2
3
![Page 17: ASSESSMENT IN GP/FM](https://reader035.vdocuments.us/reader035/viewer/2022062305/5681587a550346895dc5dbce/html5/thumbnails/17.jpg)
Difficulty Index
0,3 0,5 0,6 0,7
------/---------------(------------)----------/--------- recommended
--------------------------------------------------------------
acceptable
too difficult too difficult
![Page 18: ASSESSMENT IN GP/FM](https://reader035.vdocuments.us/reader035/viewer/2022062305/5681587a550346895dc5dbce/html5/thumbnails/18.jpg)
Discrimination Index
• DI = 2 (G-P)/N• DI – discrimination Index• G – number of right
answer in “good” group• P – number of right
answer in “poor” group• N – number of students in
both group
a b c d e 0 U
1
2
3
![Page 19: ASSESSMENT IN GP/FM](https://reader035.vdocuments.us/reader035/viewer/2022062305/5681587a550346895dc5dbce/html5/thumbnails/19.jpg)
Diskrimination Index
0.15 0.25 0.35
----------/----------/----------/---------- throw off to check good excelent
![Page 20: ASSESSMENT IN GP/FM](https://reader035.vdocuments.us/reader035/viewer/2022062305/5681587a550346895dc5dbce/html5/thumbnails/20.jpg)
What AM to use in reality?
• There is no perfect AM• A choice depends of what is to be measured –
educational objectives (knowledge, skills, competences, attitudes)
• Careful preparation, trained assessors• Multiple methods – better then one
![Page 21: ASSESSMENT IN GP/FM](https://reader035.vdocuments.us/reader035/viewer/2022062305/5681587a550346895dc5dbce/html5/thumbnails/21.jpg)
Specialistic exam in GP in Croatia
Should be submeted in advance:• Portfolio• 8 essays• Questionary of professional competence –
signed by trainer• Trainer’s report about trainee’s achievements• Trainee’s report – How to organise my practice,
what to invent?• Medical records (videos) of 3 patients / family
![Page 22: ASSESSMENT IN GP/FM](https://reader035.vdocuments.us/reader035/viewer/2022062305/5681587a550346895dc5dbce/html5/thumbnails/22.jpg)
Specialistic exam in GP in Croatia
• MCQ test – 150 questions
• OSCE – 20-25 stations
• Oral – jury (3 members): 1 academic GP + 1 well recognised GP (publication) + 1 trainer
![Page 23: ASSESSMENT IN GP/FM](https://reader035.vdocuments.us/reader035/viewer/2022062305/5681587a550346895dc5dbce/html5/thumbnails/23.jpg)
Formative assessment
The reasons:
• Information for the student about his/her achievement of educational objectives
• Repetitive – progress measurement
• Discover “week points”–teacher’s support
• Help to a teacher to correct programme
• The results should not be used in summative assessment
![Page 24: ASSESSMENT IN GP/FM](https://reader035.vdocuments.us/reader035/viewer/2022062305/5681587a550346895dc5dbce/html5/thumbnails/24.jpg)
Formative assesssment -which AM?
• The same as for summative – progress test• Protocols for observing clinical work
(consultation, communication, p-s abilities, professional competences)
• Checklists• Diaries – what has been done• Appraising projects, audit…• Portfolio• Self and peer assessment
![Page 25: ASSESSMENT IN GP/FM](https://reader035.vdocuments.us/reader035/viewer/2022062305/5681587a550346895dc5dbce/html5/thumbnails/25.jpg)
Formative assesssment
• Feedback
• Feedback
• Feedback
• Feedback
• Feedback
![Page 26: ASSESSMENT IN GP/FM](https://reader035.vdocuments.us/reader035/viewer/2022062305/5681587a550346895dc5dbce/html5/thumbnails/26.jpg)
Educational objectives in this session – achieved?
At the end of this session participants will be able to:
• name at least three differences between summative and formative assessment
• make a list of at least three written AM• name the most effective AM to assess
clinical skills• describe the most effective AM to assess
attitudes