Paper 99Can the computerized peer-

assessment of digital stories fully assess student subject

knowledge?

Dr Phil DaviesUniversity of Glamorgan

Email: pdavies@glam.ac.ukTwitter: PhilWales

How to use Digital Storytelling?

• Life stories using media – Dana Atchley• Centre for Digital Storytelling – Joe Lambert <inception

1993>• Social Sciences – interleaved with Narrative• Capture Wales Project <BBC>

• How to assess knowledge/reflection/application of ideas/personalization?

• The student becomes PART OF THE TASK not merely a REPORTER – reports in the 1st not 3rd Person

• Synthesize ideas then present as if personal• THE IMPACT OF MULTIMEDIA

Cynical Essay Scenario

• GOOGLE RULES OK?• Numerous links – actually follow some of them/even make a note

of the reference and COPY into references section• HAVE TO PUT INTO OWN WORDS OTHERWISE THINK IT IS COPIED

– takes approx 2 hours – do it properly!• Place in acceptable format – submit it – forget it – count 28 days

for turnaround – like/dislike mark – may look at comments – difficult to work out comments/feedback as have forgotten what the essay was about in the first place

• Did they really understand it?• Could they apply their knowledge?• Will they remember it for a long time?• Have they really thought about the implications of it?• Do they know what others in their group think about it?

How would we assess their knowledge?

• In order for the student to ‘become’ that person they must perform significant background research in order to understand the needs of an individual in that position.

• The presentation of the story and the associated research links provides a means of evaluating a student’s knowledge. I’ve described this in the past as ‘Technological Role-Play’.

Needs of Marking• Self-Assess own Digital Story• Peer-Assess others – Criteria and Framework• Generate Raw Peer-Assessed Median• Check if Under or Over Markers in General• Modify their Peer-Markings accordingly• Generate a Compensated Peer Average Mark• Reward the Peer-Markers for Showing Consistency:

– Consistently high or low– How helpful were peer-comments– Were comments and marks provided consistent

• Remove the Inconsistent Markers from Peer-Mark Generation and Re-Calculate the Peer Generated Average Mark

• Allow Students to re-submit to tutor for grading after having peer-marked others and also having received peer-feedback not marks

• Each Digital Story they view will be unique – not copied – personalized –produced Digital Story is NOT anonymous - however marking process is still ANONYMOUS

• CAPODS – Computerized Assessment by Peers of Digital Stories

How the final mark was generated

• The final grade (assignment worth 40% of module’s mark) awarded to the students comprised of:

a) peer generated grade for their Digital Story (15%)b) a tutor generated grade for their Digital Story (15%)c) a tutor/system generated grade for their consistency

shown in marking and commenting (10%).

Colour Blind Digital Story

• Colour Blind Digital Story: Paul Stokes

• ‘I never thought about how difficult it was for a student with a visual impairment to do the things on a PC that I find easy – it will really make me think about how I develop my web pages in future’.

EMPOWERING STUDENTS

• Final Year student this year – Scott’s Family DSt – Wanted everyone to know!

• Produce a digital story <to be peer-assessed> that will demonstrate how you learnt a particular skill (preferably computing)– A Student’s (Scott) perspective of studying a

course at the University

Questions/Points Are they true?• Students don’t copy words – they copy images,

audio, video, cartoons, etc– Only really matters if they are gaining marks for these

images etc?– Does a student on writing an essay get more marks for

using ‘big words’ or having neat hand writing or producing something that is grammatically correct – should only be if it is an assessable learning outcomes of the module

• Require media skills to create Digital Stories• Not suitable for non-computing students– not true as a Digital Story is easily created using

Powerpoint – computer literate –> Capture Wales?

Questions/Points Are they true?• Can only develop limited range of suitable Digital Stories for Higher

Education– Anything where we can show the process not just an end product by use

of multimedia

• Examples• Consider how a disability of your choice would affect your use of the

web – how could web pages be developed that could make it easier for you?

• Consider how in the past you’ve learnt a particular skill. Explain how you learnt that skill and then map it to Learning Styles/Theories

• Consider nursing a terminally ill patient with X, how would you support them – could be modified to you are a terminally ill patient with X, show us what is it like for you?

• What effect does Global Warming have upon a particular region of Y? => you live in Y, tell me what it it like for you living in this area since Global Warming occurred, etc.

Questions/Points Are they true?• Only surface knowledge – spend too much time on

media development and not on learning– Again related to the marking scheme and module

outcomes- same could be said of getting it grammatically correct

• Students will not peer-assess ‘well’ and will give high marks to their friends– Not if they are being marked on their marking ability and

consistency – marking is still anonymous

• Students will not learn how to write essays and reports– Is this a learning outcome in ALL modules in HE?

Questions/Points Are they true?• Overseas students will be disadvantaged due

to poor language skills– Easier to hide grammatical and spelling failings

using voice/media presentation

• Students do not learn how to cite references– Probably the opposite is true as they must cite

using textual banners during the course of the Digital Story

Any Questions – main one > can we view these stories?

http://www.comp.glam.ac.uk/pages/staff/pdavies/DSt/DSt.htm