PREPARING EFFECTIVE MULTIPLE PREPARING EFFECTIVE MULTIPLE CHOICE EXAMINATION CHOICE EXAMINATION

QUESTIONSQUESTIONS

PATRICK DUFF, M.D.PATRICK DUFF, M.D.

LEARNING OBJECTIVESLEARNING OBJECTIVESDescribe the levels of complexity Describe the levels of complexity

of an examination questionof an examination questionDescribe the types of Describe the types of

examination questionsexamination questionsDescribe desirable and Describe desirable and

undesirable features of undesirable features of examination questionsexamination questions

KEY REQUIREMENTS FOR AN KEY REQUIREMENTS FOR AN EXAMINATIONEXAMINATION

REQUIREMENTREQUIREMENT DEFINITIONDEFINITIONValidityValidity Measures what it is Measures what it is

supposed to supposed to measuremeasure

ReliabilityReliability Provides Provides reproducible and reproducible and consistent resultsconsistent results

DiscriminationDiscrimination Distinguishes Distinguishes between the between the excellent, very excellent, very good, and fair good, and fair studentstudent

COMPLEXITY OF COMPLEXITY OF EXAMINATION QUESTIONSEXAMINATION QUESTIONS

CLASSIFICATIONCLASSIFICATION DESCRIPTIONDESCRIPTION

RecallRecall Recall of a specific Recall of a specific factfact

InterpretationInterpretation Identification and Identification and interpretation of interpretation of illustrative materialillustrative material

Problem solvingProblem solving Recall, Recall, interpretation, and interpretation, and practical applicationpractical application

TYPES OF EXAMINATION TYPES OF EXAMINATION QUESTIONSQUESTIONS

TYPETYPE REMARKSREMARKS

C – either, or, both, C – either, or, both, neitherneither

UnacceptableUnacceptable

K – select among K – select among several several alternativesalternatives

UnacceptableUnacceptable

True – falseTrue – false UnacceptableUnacceptable

TYPES OF EXAMINATION TYPES OF EXAMINATION QUESTIONSQUESTIONS

TYPETYPE REMARKSREMARKSA – select the one A – select the one best answerbest answer

Ideal question – Ideal question – appropriate for appropriate for recall, recall, interpretation, interpretation, problem solvingproblem solving

B – matchingB – matching Appropriate for Appropriate for recall, recall, interpretation, and interpretation, and simple management simple management problemsproblems

EXAMPLEEXAMPLERECALL RECALL

Which of the following is the most Which of the following is the most important risk factor for important risk factor for squamous cell carcinoma of the squamous cell carcinoma of the lung?lung?A. Age > 50A. Age > 50B. DiabetesB. DiabetesC. SmokingC. SmokingD. ObesityD. Obesity

EXAMPLEEXAMPLEINTERPRETATIONINTERPRETATION

A 24 year old nulliparous woman at A 24 year old nulliparous woman at 16 weeks gestation had the following 16 weeks gestation had the following values on her serum analyte values on her serum analyte screening test:screening test:

AFP …below the meanAFP …below the meanHCG…above the meanHCG…above the meanEstriol…below the meanEstriol…below the meanInhibin…above the meanInhibin…above the mean

EXAMPLE EXAMPLE INTERPRETATIONINTERPRETATION

Which of the following chromosome Which of the following chromosome abnormalities is her baby most likely abnormalities is her baby most likely to have?to have?A. TriploidyA. TriploidyB. Trisomy 21B. Trisomy 21C. 45, XC. 45, XD. Trisomy 18D. Trisomy 18

EXAMPLEEXAMPLEINTERPRETATIONINTERPRETATION

A term neonate A term neonate has has hepatosplenomegahepatosplenomegaly, ly, thrombocytopenia, thrombocytopenia, and the diffuse and the diffuse rash noted in the rash noted in the photograph.photograph.

EXAMPLEEXAMPLEINTERPRETATIONINTERPRETATION

Which congenital infection is Which congenital infection is most likely?most likely?A. Parvovirus A. Parvovirus B. CMV B. CMV C. HIV C. HIV D. Mumps D. Mumps

EXAMPLEEXAMPLEPROBLEM SOLVINGPROBLEM SOLVING

A 22 year old male has a 48 hour A 22 year old male has a 48 hour history of malaise, fever (39 deg C), history of malaise, fever (39 deg C), shaking chills, and cough productive shaking chills, and cough productive of rust-colored sputum. On physical of rust-colored sputum. On physical examination, there is dullness to examination, there is dullness to percussion in the lower left lung field, percussion in the lower left lung field, and tubular breath sounds can be and tubular breath sounds can be heard in this area.heard in this area.

EXAMPLEEXAMPLEPROBLEM SOLVINGPROBLEM SOLVING

Which of the following is most Which of the following is most appropriate for treatment of this appropriate for treatment of this patient?patient?A. IsoniazidA. IsoniazidB. FluconazoleB. FluconazoleC. CeftriaxoneC. CeftriaxoneD. AcyclovirD. Acyclovir

APPROPRIATE MIX OF APPROPRIATE MIX OF EXAMINATION QUESTIONSEXAMINATION QUESTIONS

RecallInterpProblem

GUIDELINES FOR AN EFFECTIVE GUIDELINES FOR AN EFFECTIVE TYPE-A QUESTIONTYPE-A QUESTION

Determine a single clear Determine a single clear objective prior to writing the objective prior to writing the questionquestion

Have only one objective per Have only one objective per questionquestion

Make sure the question is at an Make sure the question is at an appropriate level of difficulty for appropriate level of difficulty for the target audiencethe target audience

GUIDELINES FOR AN EFFECTIVE GUIDELINES FOR AN EFFECTIVE TYPE-A QUESTIONTYPE-A QUESTION

Correct response should be Correct response should be obvious to the well-informed obvious to the well-informed examinee before the answers are examinee before the answers are readread

If the examinee has to read all of If the examinee has to read all of the answers before deciding, the the answers before deciding, the question is probably a K-type question is probably a K-type “disguised” as an A-type“disguised” as an A-type

TYPE A QUESTIONTYPE A QUESTIONGOOD EXAMPLEGOOD EXAMPLE

Which of the following is the Which of the following is the most common complication of most common complication of multiple pregnancy?multiple pregnancy?A. Gestational diabetesA. Gestational diabetesB. Placenta previaB. Placenta previaC. Postpartum hemorrhageC. Postpartum hemorrhageD. Preterm deliveryD. Preterm delivery

BEWARE THE PSEUDO-A BEWARE THE PSEUDO-A QUESTIONQUESTION

All of the following statements about All of the following statements about preterm delivery are true exceptpreterm delivery are true exceptThe frequency is 10 %.The frequency is 10 %.It can be prevented by use of It can be prevented by use of

progesterone.progesterone.It is more common in whites than in It is more common in whites than in

African-Americans.African-Americans.It may result in neonatal RDS.It may result in neonatal RDS.

GUIDELINES FOR AN EFFECTIVE GUIDELINES FOR AN EFFECTIVE TYPE-A QUESTIONTYPE-A QUESTION

Include sufficient information Include sufficient information in the stem, but avoid in the stem, but avoid unnecessary verbiageunnecessary verbiage

Be certain the question has a Be certain the question has a clear, clear, unambiguousunambiguous answer answer

GUIDELINES FOR AN EFFECTIVE GUIDELINES FOR AN EFFECTIVE TYPE-A QUESTIONTYPE-A QUESTION

Include 4 to 5 possible Include 4 to 5 possible answersanswers

Be certain the incorrect Be certain the incorrect answers are reasonably answers are reasonably plausibleplausible

TYPE A QUESTIONTYPE A QUESTIONBAD EXAMPLEBAD EXAMPLE

What is the risk of pregnancy What is the risk of pregnancy loss after a second trimester loss after a second trimester amniocentesis?amniocentesis?A. 0.5%A. 0.5%B. 10 %B. 10 %C. 25 %C. 25 %D. 50 % D. 50 %

GUIDELINES FOR AN EFFECTIVE GUIDELINES FOR AN EFFECTIVE TYPE-A QUESTIONTYPE-A QUESTION

Do not use “none of the Do not use “none of the above” or “all of the above” or “all of the above” as distractorsabove” as distractorsConfusingConfusingRewards guessingRewards guessing

GUIDELINES FOR AN EFFECTIVE GUIDELINES FOR AN EFFECTIVE TYPE-A QUESTIONTYPE-A QUESTION

Avoid clues in the Avoid clues in the responses such as responses such as answers of unequal answers of unequal length or repetition of length or repetition of key wordskey words

GUIDELINES FOR AN EFFECTIVE GUIDELINES FOR AN EFFECTIVE TYPE-A QUESTIONTYPE-A QUESTION

Avoid compound Avoid compound answers, ie, answers answers, ie, answers that include multiple that include multiple actionsactions

TYPE A QUESTIONTYPE A QUESTIONBAD EXAMPLEBAD EXAMPLE

Which of the following is the most Which of the following is the most appropriate treatment for appropriate treatment for community-acquired pnemonia?community-acquired pnemonia?A. ClindamycinA. ClindamycinB. VancomycinB. VancomycinC. ValacyclovirC. ValacyclovirD. Administer oxygen, check blood D. Administer oxygen, check blood

gases, and administer levofloxacin gases, and administer levofloxacin

GUIDELINES FOR AN EFFECTIVE GUIDELINES FOR AN EFFECTIVE TYPE-A QUESTIONTYPE-A QUESTION

Be wary of “except” type Be wary of “except” type questions, especially when questions, especially when a “double negative” is a “double negative” is presentpresent

GUIDELINES FOR AN EFFECTIVE GUIDELINES FOR AN EFFECTIVE TYPE-A QUESTIONTYPE-A QUESTION

In certain instances, an In certain instances, an “except” type question may “except” type question may be appropriate and may be appropriate and may accurately assess the breadth accurately assess the breadth of an examinee’s knowledgeof an examinee’s knowledge

TYPE A QUESTIONTYPE A QUESTIONGOOD EXAMPLEGOOD EXAMPLE

All of the following are risk All of the following are risk factors for gestational factors for gestational diabetes exceptdiabetes except

A. ObesityA. ObesityB. Positive family historyB. Positive family historyC. Age < 25 yC. Age < 25 yD. History of a macrosomic childD. History of a macrosomic child

GUIDELINES FOR AN EFFECTIVE GUIDELINES FOR AN EFFECTIVE TYPE-A QUESTIONTYPE-A QUESTION

Be certain that illustrative Be certain that illustrative materials are clear and that materials are clear and that they reproduce wellthey reproduce well

Vary the correct response Vary the correct response among the letters among the letters a - da - d

GUIDELINES FOR AN EFFECTIVE GUIDELINES FOR AN EFFECTIVE TYPE-B QUESTIONTYPE-B QUESTION

Clearly identify Clearly identify the nature of the nature of the items in the items in column A and column A and column Bcolumn B

Include at least Include at least 1 or 2 unused 1 or 2 unused distractorsdistractors

GUIDELINES FOR AN EFFECTIVE GUIDELINES FOR AN EFFECTIVE TYPE-B QUESTIONTYPE-B QUESTION

Indicate if answers in one of Indicate if answers in one of the columns may be used the columns may be used only once, more than once, or only once, more than once, or not at allnot at all

Be certain the matching Be certain the matching items are reasonably related items are reasonably related to one anotherto one another

GUIDELINES FOR AN EFFECTIVE GUIDELINES FOR AN EFFECTIVE TYPE-B QUESTIONTYPE-B QUESTION

Matching items are particularly Matching items are particularly useful for testing interpretation useful for testing interpretation skillsskillsX-rayX-rayECGECGSonogramSonogramHistologyHistology

TYPE B QUESTIONTYPE B QUESTIONGOOD EXAMPLEGOOD EXAMPLE

COLUMN ACOLUMN AAntibioticsAntibiotics

COLUMN BCOLUMN BTarget OrganismsTarget Organisms

1. Clindamycin1. Clindamycin a. a. Bacteroides Bacteroides speciesspecies

2. Penicillin2. Penicillin b. Staphylococcib. Staphylococci

3. Gentamicin3. Gentamicin c. c. E. coliE. coli

d. Group B d. Group B streptococcistreptococci

TYPE B QUESTIONTYPE B QUESTIONBAD EXAMPLEBAD EXAMPLE

DRUGDRUG PRINCIPAL SIDE PRINCIPAL SIDE EFFECTEFFECT

1. Nifedipine1. Nifedipine a. Marrow a. Marrow suppressionsuppression

2. Ampicillin2. Ampicillin b. Decreased libidob. Decreased libido

3. Fluoxetine3. Fluoxetine c. Rashc. Rash

d. Hypotensiond. Hypotension

CONCLUSIONSCONCLUSIONSUse only type-A and type-B Use only type-A and type-B

questionsquestionsMinimize recall questions and Minimize recall questions and

maximize questions that test maximize questions that test interpretive and problem-interpretive and problem-solving skillssolving skills

CONCLUSIONSCONCLUSIONSBe certain that each question Be certain that each question

has a single and specific has a single and specific objectiveobjective

Be certain the questions can Be certain the questions can be answered in the time be answered in the time allottedallotted