students as co-producers of learning and assessment content

Students as co-producers of learning and assessment content

Simon BatesAcademic Director, Centre for Teaching, Learning and Technology Professor of Teaching, Department of Physics and AstronomySenior Advisor Teaching and Learning

CHES Celebration of Scholarship Oct 2013

Wednesday, 16 October, 13

Unleashing James Bond, internet memes and partying penguins

in support of learning

Simon BatesAcademic Director, Centre for Teaching, Learning and Technology Professor of Teaching, Department of Physics and AstronomySenior Advisor Teaching and Learning

CHES Celebration of Scholarship Oct 2013


Outline

Why?What?

How?

What next?


Why?What?

How?

What next?


Knowledge

Structure Motivation

Mastery

Practice

Climate

Metacognition


Why?What?

How?

What next?


a"web&based"MCQ"repository"created"by"students"Wednesday, 16 October, 13

Ins$tu$ons((signing(up(per(year:(!

2009:! ! !22(2010:! ! !66(2011:! ! !204(2012:! ! !266(2013!(Jan,Jun):! !214(

Growing(content(repository:(!Courses:! !2,500(Logins/month:! !75,000(Ques<ons:! !600,000(Answers:! !12,000,000(


Student'ownership'over'learning'resource'

Student'familiarity'with'social'so7ware'

Leveraging'student'energy'and'crea9vity'


18

As a question author…..


19 Wednesday, 16 October, 13

20


23

As a question answerer …..


24


25


26


27


Badges' Points' Leaderboards'


Typical implementation

Minimum participation requirements for each of two assessment exercises (PW1, PW2)

Write 1 Answer 5 Rate / comment 3

5% course credit

Physics 101, Energy & WavesWinter Semester: 3 sections, 791 students


Design and MethodologyTOCCLT'.' -

Qest\I\sct

How 1o...

q/t4osTer

Submit ond onswer questionson topics in lhe torget region,just obove lhe physicsyou hove olreody mostered.

Dislroclers

) l*"*"ò

rNrrltngIh s region corrk:ins llte

plrysics knowledgo ond colrco¡lsyou connol leorn yel

becouse the louncalionscrrc nol in plocc

PHYSICS TOPICS IN YOUR TARG T REGION

buo'5ot't1 'Den¡¡lu

B¿:o\v\vì 1 JF:rcq - YOUR CHOSEN TOPIC

à..\auC'q^ x@bôo'osro

pn ro, d logroo, \o 9tS

COMMON MISCONCEPÎIONS AND ERRORS(Sec hHÞ:/jÞhy¡.udofo¡.cdu/CJP/trêconcoÞllo¡t.pdt

tor o llil ol common mhconcepllons)

¿K N(,qht %cce Açxs no\ exi¡þ, c.rrrçr\J On oblec'-t \5 õÞYJrnr cìü?ÀI in rtuiò. ü

lrh'brot4ont eprç,e .I¡1enÀs crqden-s,\{^ c>Ç c.,þ¡ec\ , cìo+ .\*¿n:r

oÇ Çtu,8,fr- btrrqon! $crCe c¿c\s Àoc¡:'cr

fr¿lt{e- r *hcrn ú?

Trr¿e. $ììfifflsz hrrr: ¿lÕoolr-g lnnS

(**= tcpo þ/rnl {otr-* qrÇ bq.\\:. 0" Ooz

Physics knowledgeond conccpluol underslonding

you lrove olrcodyconslrucleC ¡n your heod

¡,<'t.lerJ rt)\^,¿z! Þ.rrp \r- \rr't\¡ ,l?

Chcck lhol yovr on3Íêr ¡t rêo!ànoblê ðnd potrlble

\'f<,,cc\

Photo by Seth Casteelhttp://www.littlefriendsphoto.comPermission to use agreed

Introduced in tutorials

Extensive scaffolding exercises

Revisited in subsequent tutorials

Tutorials delivered by 24 TAs


Why?What?

How?

What next?


Engagement with PeerWise



Number Multiplier Number MultiplierQuestions 1105 [1.7] (1.4) 998 [1.6] (1.3)

Answers 11393 [17.2] (14.4) 11807 [18.7] (15.0)

Comments

4901 [7.4] (6.2) 5509 [8.7] (5.0)

PW 1PW 1 PW 2PW 2



PW1 PW2

Contributed and met minimum requirementsContributed but did not meet mininum requirementsDid not participate

0.84 0.80

Total: 791


Correlation with learning


Question/Explanation Quality



Bloom’s Taxonomy of levels in the cognitive domain

Score Level Description

1 Remember Factual knowledge, trivial plugging in of numbers

2 Understand Basic understanding of content

3 Apply Implement, calculate / determine. Typically one-stage problem

4 Analyze Typical multi-step problem; requires identification of strategy

Evaluate Compare &assess various option possibilities; often conceptual

Synthesize Ideas and topics from disparate course sections combined. Significantly challenging problem.

Text


1 2 3 4 1 2 3 40

20

40

60

Num

ber o

f que

stio

ns

Assessment 1 Assessment 2

Bloom's Taxonomy: Question Quality

Textp>0.05, NS

0%

5%

10%

15%

20%25%

30%

35%

40%

45%

50%

1 2 3 4 5 6

Taxonomic Category

Per

cent

age

of S

ubm

itte

d Q

uest

ions

First semester N = 350

Second semester N = 252



Description of explanation quality

Score Level Description

0 Missing No explanation provided or explanation incoherent/irrelevant

1 Inadequate Wrong reasoning and/or answer; trivial or flippant

2 MinimalCorrect answer but with insufficient explanation/justification/ Some aspects may be unclear/incorrect/confused.

3 Good Clear and detailed exposition of correct method & answer.

4 ExcellentThorough description of relevant physics and solution strategy. Plausibility of all answers considered. Beyond normal expectation for a correct solution


0 1 2 3 4 0 1 2 3 40

20

40

60

Num

ber o

f que

stio

ns

Assessment 1 Assessment 2

Explanation Quality

p=0.02


Why?What?

How?

What next?


Why stop with MCQs ?


Take aways

Who really does own the learning?

Students as consumers, curators or creators of learning content? Or perhaps all 3?

Who should?


AcknowledgementsUBC: Physics 101 course team (Georg Rieger / Firas Moosvi / Emily Altiere) CWSEI

University of Edinburgh: Physics Education Research Group (Ross Galloway / Judy Hardy / Karon McBride / Alison Kay / Keith Brunton / Jonathan Riise / Danny Homer); Chemistry (Peter Kirsop); Biology (Heather McQueen)

University of Glasgow: Physics and Astronomy (Morag Casey)

University of Auckland: Computer Science (Paul Denny)

These slides: http://bit.ly/Bates_CHES2013

[email protected] @simonpbates


http://bit.ly/Bates_CHES2013

http://bit.ly/Bates_CHES2013

mailto:[email protected]

mailto:[email protected]

Resources / links

Student-facing system http://peerwise.cs.auckland.ac.nz/

All the research studies referenced and scaffolding materials referred to are accessible through the PeerWise

community site

http://www.peerwise-community.org/


http://peerwise.cs.auckland.ac.nz

http://peerwise.cs.auckland.ac.nz

http://www.peerwise-community.org

http://www.peerwise-community.org

Publications

JISC-funded multi institution study: https://www.wiki.ed.ac.uk/display/SGC4L/Home

UoE Physics Pilot Study: AIP Conf. Proc. 1413, 359 http://dx.doi.org 10.1063/1.3680069

RSC overview articlehttp://www.rsc.org/Education/EiC/issues/2013January/student-generated-

assessment.asp

UoE Physics scaffolding resources http://www2.ph.ed.ac.uk/elearning/projects/peerwise/


https://www.wiki.ed.ac.uk/display/SGC4L/Home

https://www.wiki.ed.ac.uk/display/SGC4L/Home

http://dx.doi.org

http://dx.doi.org

Resources: How Learning Works

•  1"page"summary":"from"CM"website"•  2"page"summary

Brent,"R."&"Felder,"R."(2011)."Chemical"Engineering"EducaEon"45(4)."257I8."“Random"thoughts…"how"learning"works.”""

•  5"page"summary":"hOp://goo.gl/lqmSI"

hOp://www.cmu.edu/teaching/principles/index.html"


Icon credits !•  Icons:!!–  Stack!of!Books!designed!by!Jeremy!J!Bristol!from!The!Noun!Project!

– Mesh!Network!designed!by!Lance!Weisser!from!The!Noun!Project!

–  Excited!designed!by!AusEn!Condiff!from!The!Noun!Project!–  Components!designed!by!Iris!Q.!Li!from!The!Noun!Project!–  Group!designed!by!Alexandra!Coscovelnita!from!The!Noun!Project!

–  Brain!designed!by!Martha!Ormiston!from!The!Noun!Project!
