assessment units

Assessment instruments

assessment units

Types classroom assessment

Placement assessment• Screening assessment• Formative assessment• Summative assessment(Berk R A, 2002)

Are the students qualified, to advance to the next, course level?

assessment units

Millers Pyramid

Types of classroom assessmentPlacement assessment• Screening assessment• Formative assessment• Summative assessment(Berk R A, 2002)

assessment units

Exercise

Reflect for a moment and write downassessment instruments that youknow and how they influence yourlearning!Remember difference between method and

instrument:Method of travel = by land, sea, air…Instrument of travel = train, bus, ship,

plane…

assessment units

Taxonomies

Classification of objectives or skills• Hierarchic– To achieve higher level one mustmaster lower one• Motivation for holistic form ofeducation

assessment units

Bloom’s taxonomy

assessment units

Millers pyramid

assessment units

Millers pyramid

assessment units

Millers pyramid

assessment units

Validity

Proper assessment instrument?– Thermometer -> Temperature– Thermometer x-> Weight– MCQ -> Know & understand

– MCQ x-> Show, demonstrate, do

assessment units

• Content (=direct) validity– Relevance?– Representative?– Subjective!

• Construct (=indirect) validity– Test result -> Competence?

assessment units

Reliability

Test-retest reliability• Equivalent forms reliability• Split-half reliability• Scorer/rater reliability

– Inter- vs. intrarater reliability

assessment units

Factors in clinical skills assessment that lead to lower reliability

Two few stations or too little testing timeCheck list that do not discriminate (too

easy or too hard)Unreliable patients or inconsistent role

playersIdiosyncratic examinersAdministration problems

assessment units

Engaging with data

Write down some observationsabout the data. What do you see?

2. Suggest explanations (hypotheses)for the data.3. How would you test the hypotheses?

assessment units

Feasibility

Cost in terms of resources and time of Construction

• Marking• Organisation• Quality feedback

assessment units

Utility

• U = Utility• R = Reliability• V = Validity• E = Educational impact• A = Acceptability• C = Cost

assessment units

Types of assessment

Multiple Choice Questions (MCQs) Modified Essay Questions (MEQs) Patient Management Problems

(PMPs) Traditional essays questions Structured questions .filling the blanks True or false questions

assessment units

Able to do

assessment units

Does

Performance assessment in vivo

Undercover,sps, videos logo

Shows how

Performance assessment in virtro, OSCE,sp based tests

Knows how

Clinical context based tests MCQs,essays,orals

Knows-

Factual tests-mcq,essays,oral

assessment units

12 curriculum outcomes. SMS

What the

Clinician is able

to do-doing the right

thing

How the clinician approaches

Their practice-doing the thing

rightThe clinician as

professional

The right

person doing it

assessment units

What the doctor is able to do 1-7

Competence in clinical skillsCompetence to perform proceduresCompetence to investigate a patientCompetence to manage a patient Competence in health promotion and disease

controlCompetence in skills of communicationCompetence to retrieve and handle information

assessment units

How the doctor approaches their practice 8-10With understanding of basic and clinical

sciences With appropriate attitudes, ethical

understanding and understanding of legal responsibilities.

With appropriate decision making skills and clinical reasoning and judgment

assessment units

The doctor as a professional 11-12

Appreciation of the role of the doctor within the health service

Attitude for personal development

assessment units

Multiple Choice Questions

They are designed to test knowledgeOutcomes assessed- knowledge of basic

skills and clinical sciences8patient investigation3 Patient management 4 health promotionClinical reasoning ands judgment

assessment units

Multiple Choice Questions

They are designed to test knowledge

Outcomes assessed- knowledge of basic skills and clinical sciences8patient investigation3 Patient management 4 health promotionClinical reasoning ands judgment

assessment units

What do multiple choices questions measure    Knowledge of specific facts   Knowledge of terminology     Knowledge of principles   Knowledge of procedures  The understanding and application of

facts and principles  The understanding of proceduresReasoning to some extent

assessment units

How to construct MCQs

   State the question positively Construct options that are grammatically

consistent with the item.  Do not use MCQs when other instruments

are more appropriate. 

assessment units

Multiple choice questions- principles(rules)  The question itself or the stem must be

appropriate  Have one correct response to the

question Distracters or the incorrect responses

among the choices should be plausible

assessment units

Extended Matched Items

Management of diabetes in the community?

1.

1. start on oral hypoglycemic2. do diabetic annual review3. three monthly blood tests and review4. monthly blood tests and review5. Refer to podiatrist6. refer to hospital diabetes specialist nurse7. Trial diet control for 3 months8. 2start on twice daily insulin injection 9. Do fasting blood sugars10. Do haemoglobin A/c blood test

assessment units

Extended matching

Select the most appropriate course of action for the following patients

1. Woman, mid 20s ,14 weeks pregnant to have glycosuria on routine resting in GP ante natal clinic

2. Woman,mid 50s ,BMI 30, just found to have fasting blood sugar 15mmol/l,HBA a18.5%

3. South Asian woman, mid 30s complaining of fatique, thirst and dry mouth

assessment units

Extended Matched Items (EMIs)

Purpose – test knowledge and reasoning skills

Outcomes assessed- all outcomes with an emphasis on knowledge and clinical science8patient investigation3 Patient management 4 health promotion5Clinical reasoning ands judgment

assessment units

MCQ

Multiple Choice Questions are used to tests students’ factual knowledge and to some extent the critical thinking of students. An attempt is made to test higher levels of Bloom’s taxonomy and not mere recall of facts.

Integratory

assessment units

1.     A cervical smear taken from a thirty five year old lady was found to have the following:

Mitotic cells Multinucleated cells Loss of chromatin granules Cells that have lost adhesiveness

assessment units

Mcq

Which of the following will be the best classification of this smear?

a)   CIN Ib)   CIN IIIc)   CIN IVd)   CIN Ve)   CIN II

assessment units

Which of the following will be the best classification of this smear? 

a)   CIN Ib)   CIN IIIc)   CIN IVd)   CIN Ve)   CIN II

assessment units

2.     Tissue Processing in Histology is the most critical stage in entire processes that go on in a Histology lab. Which of the following will determine the processing procedure?Urgent diagnosis The element to be demonstrated The materials availableThe skills involvedThe size of tissue for examination

                                                           

assessment units

MEQ

Modified Essay Questions will be used to assess the students’ ability to integrate facts and problem solving skills, which are fundamental to good practice. Question requiring consolidation of facts and critical thinking will be asked.

assessment units

1.     A tissue section was found to be having a brown colour that was intracellular upon staining with H/E staining technique. The pigment was found to be soluble in alcoholic picric acid.                                                       

assessment units

Possible scenarios

  i.      What is the possible name of this pigment?

   ii.      What features enabled you to identify the pigment?

     iii.      What class would this pigment fall into?Assuming the pigment was not soluble in

alcoholic acid picric acid, how would this affect your identification

assessment units

Formatively as learning tool to stimulate

discussion and top plan future development

Can be used to stimulate and provide feedback – may be every three months what is going , intriguing / exciting ,worrying phenomenon.

assessment units

What can MEQ measure ?     MEQ can be used as a formal assessment

technique      It can be used as an aid to learning.     It can be used as an adjunct to a lecture. It presents good questions for group

discussion.  It can be used as a self-instruction method.It can be used as a self-audit device.

assessment units

Construction of MEQ

  Drawing up a draft of the subject area.  Circulating the draft to a team of fellow

professionals to study it.  Each of the professional colleagues should be

asked to provide the answer to each of the questions.

    These responses should be studied and forms the basis for model answers.

The marks should be allocated according to consensus decisions

assessment units

PATIENT MANAGEMENT PROBLEM (PMP)

Patient management problem (PMP) consists of short questions that require also short answers that revolve around managing a patient. They are intended to equip the student with knowledge, skills and attitude that are necessary in professional practice.

The main feature is that a series of questions are asked about a case and options are provided that leads to proper management of the patient. It tests students on a wide range of subjects.

assessment units

What PMP can assess

Like the MEQ, PMP provides the opportunity for students to demonstrate their ability in patient management skills and problem solving skills in a variety of scenario.

The students are presented with simulated patients, real patient or imaginary patient in a paper and pencil format. The unfolding scenario is presented logically and the student is evaluated on the basis of the proper steps taken leading to the proper management of the patient.

assessment units

Constructing a PMP

Constructing a PMP begins with a description of the symptoms of the patient and the resources that are available. The student is presented with the problem followed by a series of questions relating to management, diagnosis and investigations. Each question has a number of options from which the student must select. Beside each option are hidden instructions about how the student should proceed.

assessment units

Steps in constructing PMP

  Clearly define what it is you wish to test in the problem. Set realistic objectives for the group you are testing.

   Select a problem that is meaningful and suitable.

  Ensure that the sequence of management is logical.

  Prepare the opening scene. This should compose of: the actual setting for the case, the presenting features of the patient, and the role of the user, the resources available and clear instructions.

assessment units

Steps continue

   Prepare a list of clinical choices which are consistent with the problem.

    Pretest the PMP using a group of subject experts. Also try it out with one or two potential candidates.

  Score the PMP. Allocate a mark for each individual choice on the PMP.