Subverting MCQs for Deep Learning

Learning outcomes

• Understand how formative MCQs can be used to promote deep learning in face-to-face and online learning

• Take a short formative MCQ quiz to see how it works from a participant’s perspective

• Begin to construct an online formative MCQ quiz

Deep approach v. surface approach

LearnersSurface approach Deep approach

Passive listeners Active participants

Understanding information only as it is presented

Trying to make linkages between information presented with other bits of (previously received) information

Processes and retains information as separate facts or bits of information

Looking for an underlying rationale/overarching structure to connect the (independent) facts

Focuses on breadth of content coverage Focuses on depth of content coverage

Assessment-oriented view of education Academically-inclined

Non-academic priorities or interests Intrinsic curiosity or a preference to engage with material at a conceptual level

Is expecting a predictable learning experience and not looking to be stretched

Is looking to be stimulated and challenged

TeachersSurface approach Deep approach

Presenting and expounding (explain) Eliciting (evoke) and probing

Learners as passive listeners Learners as active participants

Learner’s role: listen, take notes, ask the odd question

Learner’s role: active listening, discussion with peers/lecturer, query and counter-argue?

Breadth of coverage is a priority Depth of coverage is a priority

Views unwieldy questions as a distraction from lecture

Welcomes questions and views them as a teachable moment

Provides answers to students’ questions Facilitates students in seeking out the answers to their questions

The multiple choice question (MCQ) as we know it

• What is it?

• How is it currently used?• Why is it not a favoured form of assessment?

What is the most widely spoken language in the world?

i. Englishii. Chineseiii. Spanishiv. Mandarin

Can MCQs be used to promote deep learning?

Formative MCQs

• Improve conceptual understanding• Target areas of potential confusion• Provide feedback• Prime learners• Activate prior knowledge• As a diagnostic test/JIT• In concert with other active learning methods

– peer instruction, discussion fora, etc.• Enable self-regulated and reflective learning

Anatomy of a formative MCQ

Topic Question set Distractors

Feedback Marking schemeIntegrated learning activity

BLOOC 10

Topic

• A key topic or fundamental concept• Lends itself to MCQ-style questions• Not likely to require frequent changes or updates

8 July 2014

Stem and options

Which of the following statements are true of learners adopting a deep approach?i. They retain information and knowledge longerii. They are able to make more connections

between different pieces of informationiii. They experience confusion and uncertainty

when trying to understand new knowledgeiv. They focus on the details when mastering a

topic

stem

Correct options

distractor

BLOOC 12

Feedback

• Adaptive• Interactive• Deferred

8 July 2014

BLOOC 13

Marking scheme

• Multiple correct answers• Negative marking• Certainty-based marking

8 July 2014

BLOOC 14

Results of the self-assessment

8 July 2014

BLOOC 15

Results – graph of results

8 July 2014

Evaluating the effectiveness of MCQs

BLOOC 17

Trivia v. deep learning

8 July 2014

BLOOC 18

Do MCQs promote deep learning?

• Biggs’ theory of learning• Deep/surface/strategic

• Bloom’s taxonomy• Knowledge, comprehension, application, analysis, synthesis,

evaluation

• Nicol’s principles of feedback• Clarifying goals, criteria and standards; self-assessment and

reflection, high-quality feedback; peer dialogue, feedback and motivation; closing the gap; just-in-time teaching

8 July 2014

How can you increase the power of a formative MCQ?

BLOOC 20

Limitations of MCQs for deep learning

• The deep approach is not often used in everyday teaching• Difficulty finding an appropriate topic• Pedagogical design

• Learning aims• Integrating it with other learning activities

• Constructing the test• Item ambiguity• Technology

• Student expectations need to be managed

8 July 2014

BLOOC 21

Can you use MCQs in your teaching?

• In which part of the curriculum could you use an MCQ?• What is the underlying rationale? • What would be the benefits for the students, for yourself?• How would it promote deep learning and better learning

outcomes?• Could you integrate it with other learning activities? • What potential challenges do you foresee with this

approach?

8 July 2014

DIY

• Get into groups of three and design a formative MCQ quiz

Topic Question set Distractors

Feedback Marking schemeIntegrated learning activity

Wrap up and review

BLOOC 24

References• Biggs, J. and Tang, C., 2007. Teaching for Quality Learning at University. 3rd ed. Berkshire: SRHE

and OUP• Bloom’s Taxonomy. Centre for Teaching. Vanderbilt University. [online] Available at:

http://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/ [Accessed 24 June 2014]• Lublin, J., 2003. Deep, surface and strategic approaches to learning. Good Practice in teaching

and Learning. Centre for Teaching and Learning, University College Dublin. [online] Available at: http://www2.warwick.ac.uk/services/ldc/development/pga/introtandl/resources/2a_deep_surfacestrategic_approaches_to_learning.pdf [Accessed 18 June 2014]

• Nicol, D., 2007. E assessment by design: using multiple choice tests to good effect. ‐ ‐ Journal of Further and Higher Education. 31:1, 53-64. Available at http://ewds.strath.ac.uk/REAP/public/Papers/MCQ_paperDN.pdf [Accessed on 30 May 2014

8 July 2014