texas tech university health sciences center school of pharmacy krystal k. haase, pharm.d., fccp,...
TRANSCRIPT
T E X A S T E C H U N I V E R S I T Y H E A LT H S C I E N C E S C E N T E R S C H O O L O F P H A R M A C Y
KRYSTAL K. HAASE, PHARM.D., FCCP, BCPSASSOCIATE PROFESSOR
BEYOND MULTIPLE CHOICE
QUESTIONS
OBJECTIVES
• Describe the pro’s and con’s of using different question formats other than multiple choice.• Identify and resolve common problems
when constructing open-ended questions.• Develop standardized grading procedures
for open-ended questions.• Discuss exam length and other challenges
when using open-ended question formats.
ASKING THE RIGHT QUESTION
REVIEW
Blo
om
’s
Evaluation
Synthesis
Analysis
Application
Comprehension
Knowledge Low
High
QUESTION TYPES
Selection Response
Construction Response
Recognition Recall
QUESTION TYPES
Selection Response
True / False
Matching
Multiple Choice
Construction Response
Fill in the Blank
Short Answer
Essay
TRUE / FALSE
• Pro’s• easy to write• easy to score
• Con’s• limited ability to assess mastery• high probability of guessing
• Best Use: • Dichotomous, factual info
Blo
om
’s
Evaluation
Synthesis
Analysis
Application
Comprehension
Knowledge
MATCHING
• Pro’s• Can assess a lot of info in a confined space• Fairly low probability of guessing
• Con’s• Assess recognition not recall
• Best Use: • Knowledge recall• Add additional “distractor” items
to increase rigor Blo
om
’s
Evaluation
Synthesis
Analysis
Application
Comprehension
Knowledge
MULTIPLE CHOICE
• Pro’s• Very versatile• Easy to score
• Con’s• More challenging to write• Assess recognition over recall
• Best Use: • Factual, conceptual, or
procedural information Blo
om
’s
Evaluation
Synthesis
Analysis
Application
Comprehension
Knowledge
SHORT-ANSWER
• Pro’s• Assess unassisted recall• Relatively easy to write
• Con’s• Only useful if you can give a
short answer.• Must be worded carefully to
avoid scoring problems
• Best Use: • Assessing information that you
expect to be memorizedB
loom
’s
Evaluation
Synthesis
Analysis
Application
Comprehension
Knowledge
ESSAY
• Pro’s• Can test higher complex objectives• Can test process / reasoning• Realistic tasks
• Con’s• Take longer to answer• Hard to grade fairly, consistently• Take longer to grade
• Best Use: • Assessing highest level objectives
Blo
om
’s
Evaluation
Synthesis
Analysis
Application
Comprehension
Knowledge
www.siop.org/workplace/employment%20testing/testformats.aspx
SUMMARY
• Each question type has pro’s and con’s• Question type should be guided by the
learning objective to be assessed.• Limited options for assessing higher
taxonomy
CONSTRUCTING OPEN-ENDED QUESTIONS
WELL-DEVELOPED ESSAY QUESTIONS
• Mirror well-defined learning objectives• Assess most appropriate content types• Require content recall, evaluation, and
reasoning • Are clearly written• Provide boundaries• Have well-defined grading criteria
http://testing.byu.edu/info/handbooks/WritingEffectiveEssayQuestions.pdf
APPROPRIATE CONTENT
• Content that justifies high-level mastery• Construction, higher-order taxonomy• Analysis (analyze, compare, contrast, interpret)• Evaluation (evaluate, explain, justify)• Synthesis (develop, construct, modify)
• Complex, multi-step thought processes• Simulation of real-world processes• “given a patient-case scenario”
If content / processes can be assessed by methods other than essay questions, they
probably should.
KEY ELEMENTS
• An ideal essay question requires students to: • Recall facts• Make an evaluative judgment or develop a
novel solution• Explain reasoning behind response
• The question should include:• Task• Problem situation
WRITING FOR CLARITY
• Ensure your question requires higher-order thinking• Make sure the task is defined and focused• Make sure the problem situation includes
adequate detail
SETTING BOUNDARIES
• Increasing structure prevents• grading problems• bluffing
• Avoid indeterminate questions• Students can redefine and answer with info the
know well
• Give time / space limits• Establish rules for answers
ESTABLISHING GRADING CRITERIA
• Must have specific criteria for grading identified a priori• Create a model answer• Assign point values• Peer review
• Identify essentials in response• Determine whether partial credit is
allowable and how will be awarded.• Grade blinded
DIFFERENT GRADING APPROACHES
• Comprehension / Understanding• Screen responses for key elements• Assign points for each element present• Scores for presence of content only (potential
bluffing)
• Reasoning / Complex Processes• Must assess complete response (time consuming)• Consider a rubric approach to limit subjectivity
• Problem – Solution• Encourage requiring students to show work• Can grade multiple steps in process
GRADING TIPS
• Grading scheme should be easily interpreted by other graders • Self-explanatory• Point values that can easily be tallied
• When multiple graders, encourage frequent communication and comparison of results• Before grading, screen a sample of
responses for consistency
OTHER ISSUES
• Exam length• Addressing unidentified “correct” answers• Partial credit or all-or-none• Practice examples• Multiple examples• Should not be the same questions as on exam