authoring high quality digital multiple-choice assessments in online courses
DESCRIPTION
Authoring High Quality Digital Multiple-Choice Assessments in Online Courses Susan M. Moncada, Indiana State University Thomas P. Moncada, Eastern Illinois University MERLOT International Conference August 9, 2008. Test Item Design Jeopardy. Stem Flaws. Stats Analysis. Distractor - PowerPoint PPT PresentationTRANSCRIPT
Authoring High Quality Digital Multiple-Choice Assessments in Online Courses
Susan M. Moncada, Indiana State UniversityThomas P. Moncada, Eastern Illinois University
MERLOT International ConferenceAugust 9, 2008
Stem Flaws
DistractorAvoidance
ClueingConcerns
StatsAnalysis
100 100 100 100
200 200 200 200
300 300 300 300
Test Item Design Jeopardy
Adapted from MERLOT artifact 78866, Financial Accoun-ting Jeopardy Games, by Mike Seda, Shaw University
Stem Flaws 100
The answer is…Which of the following is not a MERLOT review criteria?
Q: What is a negatively worded stem?
Click once to display the question Click on the arrow to return to the game
Stem Flaws 200
Q: What is an unfocussed stem?
The answer is…. MERLOT isa. A line of skin care productsb. A producer of customized tarping systemsc. A repository of digital learning objectsd. A term for “black bird” in French
Click once to display the question Click on the arrow to return to the game
Stem Flaws 300
Q: What is window dressing?
The answer is….unnecessary irrelevant content in a multiple-choice stem.
Click once to display the question Click on the arrow to return to the game
Distractor Avoidance 100
Q: What are overlapping distractors?
The answer is…MERLOT membership benefits includea. Personal profiles and blogsb. Personal collections and personal profilesc. Second Life access and personal collectionsd. Virtual speakers’ bureau and free software
Click once to display the question Click on the arrow to return to the game
Distractor Avoidance 200
Q: What are implausible distractors?
The answer is…Who is the Executive Director of MERLOT?a. Carl Sandburgb. Gerry Hanleyc. Abraham Lincolnd. Spiderman
Click once to display the question Click on the arrow to return to the game
Distractor Avoidance 300
Q: What are specific determiners?
The answer is…always, never, totally, absolutely, and completely.
Click once to display the question Click on the arrow to return to the game
Clueing 100
Q. What are grammatical inconsistencies?
The answer is… Which criteria does “Ease of Use” include?a. Concepts are completely demonstratedb. Content is well summarizedc. Having clear instructionsd. Prerequisite is knowledge identified
Click once to display the question Click on the arrow to return to the game
Clueing 200
Q: What is a longest response key ?
The answer is… Which reviewer phrase is a teaching effectiveness
strength?a. Requires contextb. Uses post-graduate vocabularyc. Demonstrates relationships between all conceptsd. Provides interactivity
Click once to display the question Click on the arrow to return to the game
Clueing 300
Q: What is a clang association?
The answer is… Which criteria does “Quality of Content”
include?a. Accurate contentb. Visual appealc. Potential effectivenessd. Prerequisite knowledge
Click once to display the question Click on the arrow to return to the game
Stats Analysis 100
Q: What is good internal consistency for teacher made tests?
The answer is…A Kuder-Richardson (KR-20) score >= .70.
Click once to display the question Click on the arrow to return to the game
Stats Analysis 200
Q: What is a multiple choice question that discriminates well and is of average
difficulty?
The answer is…An IDR of .6 and a P-value of .5.
Click once to display the question Click on the arrow to return to the game
Stats Analysis 300
Q: What is the chance of scoring > 70% on a ten item multiple choice assessment with 4 distractors?
The answer is…1 out of 285
Click once to display the question Click on the arrow to return to the game