students’ assessment and evaluation prof. namir g. al-tawil m.b.ch.b., f.i.c.m.s./cm hawler...
TRANSCRIPT
Students’ Assessment and EvaluationProf. Namir G. Al-Tawil
M.B.Ch.B., F.I.C.M.S./CMHawler Medical University
College of [email protected]
Contents
• Objectives• Definitions of assessment and evaluation• Types of assessment– Formative assessment– Summative assessment
• Assessment of students’ knowledge• Assessment of students’ performance
Objectives
• At the end of the lecture, the audience must be able to know:
• The differences between formative and summative assessment.
• How to assess students’ knowledge.• How to assess students’ performance.
Definitions
• Assessment– It is the process of data gathering by instructors
about their teaching and their students’ learning.– Pre-test, observations, examinations are forms of
assessment.• Evaluation– It is the decision making process based on the
assessment data.
Diagnostic assessment
• Helps to identify students’ current knowledge, skills, and capabilities; and to clarify misconception before teaching takes place.
• Types:– Pre-tests– Self assessment– Interview
Formative assessment
• Provides feedback during a teaching process.• It assesses student’s and teacher’s progress.• These assessments are typically not graded.• E.g: if after 2 weeks of the start of the course,
you made a test (by clickers for e.g) and you discovered that the majority didn’t answer correctly, then you have to repeat this point, and to correct yourself in the next course
Types of formative assessment
• Observations during in-class activities.• Homework exercises.• Question and answer sessions (planned and
spontaneous).• Conferences between instructor and students.• Student feedback about their instructors.
Summative assessment
• Takes place at the end of learning process.• It assesses the final product.• Grades are usually an outcome of it.– To decide whether the student is eligible for the
next step or not.
Criteria of the assessment tool
• Validity• Reliability• Educational impact, on teaching and learning• Acceptability to staff, students, and others• CostA variety of assessment methods are needed to
test competency
“When the cook tastes the soup, that’s formative evaluation. When the guests taste the soup, that’s summative evaluation”
Bob Stake 1991
Assessment of knowledge and its application
• The most common method for knowledge assessment is the written method.
• Several formats are available like MCQs and essays….
• The content of the question is more important than the format.
• A variety of formats are preferable.
Questions formatsMCQs (one best answer)
• Single choice questions are more preferred than the true/false (multiple response item).
• Anatomy of the item (question):– Stem• Vignette (e.g case study)• Lead-in question (what is the most likely Dx?)
– Options• Correct answers• Distractors (wrong-response options)
Tips for designing MCQs
• Use Plausible Distractors• Use a Question Format (rather than
incomplete sentence format)• Keep Option Lengths Similar• Balance the Placement of the Correct Answer• Be Grammatically Correct
• Avoid Clues to the Correct Answer• Avoid Negative Questions• Use Only One Correct Option • Give Clear Instructions• Avoid the "All the Above" Option• Avoid the "None of the Above" Option• Don't Use MCQs When Other Item Types Are
More Appropriate
MCQs (Extended matching items)
• Extended matching questions consist of lettered options followed by a list of numbered problems/questions.
• For each numbered problem/question select the one lettered option that most closely answers the question.
• You can use the lettered options once, more than once, or not at all.
Extended matching, example• For each case below, choose the SINGLE most appropriate treatment
option from the above list of options. Each option may be used once, more than once, or not at all.
• Options: a. Laser treatment as laparoscopyb. Danazolc. Oral contraceptivesd. Mefenamic acid (Ponstan)e. Gonadotrophin releasing hormone agonist (Soladex)f. Total abdominal or vaginal hysterectomyg. No treatmenth. Pelvic sympathectomyi. Laparotomy
1. A 30 year old woman complains of painful periods and pain during intercourse. She is afebrile and has a firm, tender nodule into the pouch of Douglas. Small foci of endometriosis are found at the time of laparoscopy. Answer: A
2. A 28 year old woman is diagnosed with endometriosis. Her symptoms are not incapacitating. She would like to start a family. Answer: G
3. A 45 year old woman complains of intractable dysmenorrhoea and menorrhagia. She does not want any more children. Answer: F
4. A 40 year old woman is diagnosed with endometriosis. The pain is severe, and she refuses surgical treatment. She does not want any more children. Answer: B
5. A 20 year old woman presents with severe left lower abdominal pain, increasing abdominal girth. Painful periods, and menorrhagia. Ultrasound demonstrates a 20 cm left ovarian cyst. Answer: I
Key features questions
• They are short clinical cases or scenarios which are followed by questions aimed at key features or essential decisions of the case.
• The questions could be MCQs or open ended questions.
• More than one correct answer could be provided.
e.g. Key features questions
• Fatima, aged 70 years, has been seeing you intermittently over the past 2 years since her previous GP (a colleague) retired. She presents this morning complaining of pain in her left knee. She explains she has had the pain for 4-5 years but it is now getting worse. Pain is worse with exercising and is relieved at rest. She also has pain at night, with stiffness in the morning and after an afternoon lie down. She now finds it difficult to walk to her local supermarket and mosque. Always she was active in the past, she now finds she looks for reasons to avoid walking.
Question 1: (2 marks)
• What possible differential diagnosis would you consider with this limited information:
• 1.Osteoarthritis• 2.Crystal induced inflammatory arthropathy
such as Gout or pseudogout• 3.Rheumatoid arthritis and other causes
popliteal cyst(baker cyst) or septic arthritis
Question 2: (1 marks)
• What should you look for on physical examination?
• 1.body weight• 2.tenderness, swelling, effusion• 3.reduced range of movement, presence of
crepitus, deformity, involve other joints.
Question 3: (1 marks)
• What investigation may be helpful?• 1.full blood count, ESR, CRP• 2.urate level, joint aspiration and microscopy
if effusion is present• 3.plan x-ray of knee, rheumatoid factor,
Antinuclear antibody,
Short answers questions
• Open ended questions.• Their answer is usually 1 or 2 words.• Usually used when closed formats are not
possible.• Each teacher should correct the same
question for all the students. So more reliable scores.
Essay questions
• Requires more time to answer.• Less reliable scoring.• Using correction scheme (checklist) can
improve reliability.
Modified essay questions
• Is a special type of essay question that consists of a case followed by a series of questions that relate to the case and that must be answered in the sequence asked.
• A student answering the first question incorrectly is likely to answer the subsequent questions incorrectly too.
Assessment of performance
• Categorized into:– In vitro assessment (using simulation)– In vivo assessment (real condition)
• In both, the student make a skill or behavior and observed by the examiner.
Assessment of performance, cont
• Many methods like check-lists and rating scales can be used to report observation.– OSCE and OSPE– Direct observation of procedural skills (DOPS)– Peer assessment– Self-assessment
Methods for assessing performance
• Checklists– E.g OSCE
• Rating scales– Problem of subjectivity and low reliability– Use more than one rater (observer)
• OSCE• Short cases to assess clinical competence
• Long cases (to assess clinical competence)– No more used in many colleges– Of help for formative examinations– Modification done in many places to improve
reliability• 360 degrees evaluation– Of help in formative assessment– Multiple evaluators like supervisors, peers,
students, administrative staff, and patients.– Time consuming and not practical.
• Mini Clinical Evaluation Exercises (MINI-CEX)– Of help in formative assessment– They are relatively short observations (15-20
minutes) in which performance is recorded on a 4 point scale where 1 is unacceptable, 2 is below expectation, 3 is met expectations, and 4 is exceeded expectations.
– Several competencies are evaluated: history taking, physical examination, clinical judgment, counseling, professionalism and other generic qualities
• Portfolio– A portfolio is a collection of student work which
provides evidence that learning has taken place.– Mainly for formative assessment.
Conclusion
• Various assessment methods that test a range of competencies are available for examiners
• The choice depends on purpose of examination, i.e. either summative, formative or both, in addition to other factors.