![Page 1: formative e-assessment: cases, patterns and scenarios](https://reader036.vdocuments.us/reader036/viewer/2022062307/555bb455d8b42a0d0f8b4b21/html5/thumbnails/1.jpg)
WLE Centre for ExcellenceInstitute of EducationUniversity of London20 Bedford WayLondon WC1H 0AL
Tel +44 (0)20 7911 5531Fax +44 (0)7092 288 882Email [email protected] www.wlecentre.ac.uk
formative e-assessment: case stories, design patterns, and future scenarios
Caroline Daly, Harvey Mellar, Yishay Mor, Norbert Pachler,
Institute of Education, University of London
![Page 2: formative e-assessment: cases, patterns and scenarios](https://reader036.vdocuments.us/reader036/viewer/2022062307/555bb455d8b42a0d0f8b4b21/html5/thumbnails/2.jpg)
Overview Scoping study commissioned by JISC
Short term, small budget, intended to inform future funding frameworks
Established a commited user group of higher-education teachers & researchers
Adopted and adapted the Planet Project's Participatory Methodology for Practical Design Patterns, and used the Planet platform
![Page 3: formative e-assessment: cases, patterns and scenarios](https://reader036.vdocuments.us/reader036/viewer/2022062307/555bb455d8b42a0d0f8b4b21/html5/thumbnails/3.jpg)
Methodology Desk research
−Literature review
−Comparing frameworks
5 Practical Enquiry Days
−Combination of collaborative reflection, report back from team, and guest plenaries
−Launch day, 3 Planet workshops, developers' day
![Page 4: formative e-assessment: cases, patterns and scenarios](https://reader036.vdocuments.us/reader036/viewer/2022062307/555bb455d8b42a0d0f8b4b21/html5/thumbnails/4.jpg)
What is formative e-assessment? No consistent view in the literature
−From “anything test before the final” to “synonymous with learning”
The use of digital means to support formative assessment
Formative features of assessment, which are afforded by specific features of digital media
![Page 5: formative e-assessment: cases, patterns and scenarios](https://reader036.vdocuments.us/reader036/viewer/2022062307/555bb455d8b42a0d0f8b4b21/html5/thumbnails/5.jpg)
OK, so what is Formative Assessment?
“An assessment functions formatively when evidence about student achievement elicited by the assessment is interpreted and used to make decisions about the next steps in instruction that are likely to be better, or better founded, than the decisions that would have been made in the absence of that evidence”
(Dylan Wiliam)
![Page 6: formative e-assessment: cases, patterns and scenarios](https://reader036.vdocuments.us/reader036/viewer/2022062307/555bb455d8b42a0d0f8b4b21/html5/thumbnails/6.jpg)
Formative = feedback + moments of contingency
"... These create "moments of contingency," in which the direction of the instruction will depend on student responses. Teachers provide feedback that engages students, make time in class for students to work on improvement, and activate students as instructional resources for one another."
(Leahy, Lyon, Thompson, and Wiliam 2005)
![Page 7: formative e-assessment: cases, patterns and scenarios](https://reader036.vdocuments.us/reader036/viewer/2022062307/555bb455d8b42a0d0f8b4b21/html5/thumbnails/7.jpg)
Teacher
Learner
Peer
Instruction
Tasks
Actions
Tasks
Actions
![Page 8: formative e-assessment: cases, patterns and scenarios](https://reader036.vdocuments.us/reader036/viewer/2022062307/555bb455d8b42a0d0f8b4b21/html5/thumbnails/8.jpg)
Wiliam's 5 stratagies
![Page 9: formative e-assessment: cases, patterns and scenarios](https://reader036.vdocuments.us/reader036/viewer/2022062307/555bb455d8b42a0d0f8b4b21/html5/thumbnails/9.jpg)
Conversational Framework (Laurillard)
![Page 10: formative e-assessment: cases, patterns and scenarios](https://reader036.vdocuments.us/reader036/viewer/2022062307/555bb455d8b42a0d0f8b4b21/html5/thumbnails/10.jpg)
Evidence Centred Design (Mislevy) Highly developed
Pattern based
Oriented to large scale, automated systems
Measurement centric
Light on theory
Less suitable for open activity designs
![Page 11: formative e-assessment: cases, patterns and scenarios](https://reader036.vdocuments.us/reader036/viewer/2022062307/555bb455d8b42a0d0f8b4b21/html5/thumbnails/11.jpg)
A few cases
Creature of the week
CoMo
Post 16 String Comparison
Open Mentor
...
![Page 12: formative e-assessment: cases, patterns and scenarios](https://reader036.vdocuments.us/reader036/viewer/2022062307/555bb455d8b42a0d0f8b4b21/html5/thumbnails/12.jpg)
Creature of the week (Judy Robertson) Situation
large class (138), first and second year computer science students. assignment: create a virtual pet in Second Life.
Task
Engage and motivate the students
show examples of good work which others could learn from
show students their work is valued.
build a sense of community.
http://purl.org/planet/Cases/creatureoftheweek
![Page 13: formative e-assessment: cases, patterns and scenarios](https://reader036.vdocuments.us/reader036/viewer/2022062307/555bb455d8b42a0d0f8b4b21/html5/thumbnails/13.jpg)
![Page 14: formative e-assessment: cases, patterns and scenarios](https://reader036.vdocuments.us/reader036/viewer/2022062307/555bb455d8b42a0d0f8b4b21/html5/thumbnails/14.jpg)
CoMo (Niall Winters, Yishay Mor) Situation
Royal Vet College.
Hospital rotations as part of their training.
Task
Allow students to capture critical incidents in text and image.
Support sharing of clinical experiences and co-reflection.
http://purl.org/planet/Cases/CoMo
![Page 15: formative e-assessment: cases, patterns and scenarios](https://reader036.vdocuments.us/reader036/viewer/2022062307/555bb455d8b42a0d0f8b4b21/html5/thumbnails/15.jpg)
![Page 16: formative e-assessment: cases, patterns and scenarios](https://reader036.vdocuments.us/reader036/viewer/2022062307/555bb455d8b42a0d0f8b4b21/html5/thumbnails/16.jpg)
Post 16 string comparison (Aliy Fowler) Situation
Grammar school been piloting the ‘string comparison’ approach to language teaching at post-16 for AS and A2 level students.
Sixth Form level, grammatical consolidation and whole-sentence translation.
Task
Allow students to practise written language independently and receive feedback on errors in order to improve their language skills.
http://purl.org/planet/Cases/Post16stringcomparison
![Page 17: formative e-assessment: cases, patterns and scenarios](https://reader036.vdocuments.us/reader036/viewer/2022062307/555bb455d8b42a0d0f8b4b21/html5/thumbnails/17.jpg)
Solution
A bespoke string (sequence) comparator was designed; uses fine-granularity sequence comparison to compare correct language strings to a user’s answer. Students answer questions and the comparator marks up errors in their input using colour coding (and font style) to highlight the different types of error. If an answer contains errors the student is given a second attempt in which to correct the submission based on the feedback received.
![Page 18: formative e-assessment: cases, patterns and scenarios](https://reader036.vdocuments.us/reader036/viewer/2022062307/555bb455d8b42a0d0f8b4b21/html5/thumbnails/18.jpg)
Open mentor (Denise Whitelock)
http://purl.org/planet/Cases/OpenMentor
![Page 19: formative e-assessment: cases, patterns and scenarios](https://reader036.vdocuments.us/reader036/viewer/2022062307/555bb455d8b42a0d0f8b4b21/html5/thumbnails/19.jpg)
A few patterns..
Try Once, Refine Once
Feedback on Feedback
Classroom display
![Page 20: formative e-assessment: cases, patterns and scenarios](https://reader036.vdocuments.us/reader036/viewer/2022062307/555bb455d8b42a0d0f8b4b21/html5/thumbnails/20.jpg)
Try Once, Refine Once
(Aliy Fowler)
http://pul.org/planet/Patterns/TryOnceRefineOnce
![Page 21: formative e-assessment: cases, patterns and scenarios](https://reader036.vdocuments.us/reader036/viewer/2022062307/555bb455d8b42a0d0f8b4b21/html5/thumbnails/21.jpg)
ProblemLack of immediate feedback for students leads to fossilisation of errors and misconceptions
providing immediate feedback in an iterative fashion can also hinder effective learning since students are able to "grope their way" step-by-step to a correct solution without necessarily having to think about each answer as a whole.
![Page 22: formative e-assessment: cases, patterns and scenarios](https://reader036.vdocuments.us/reader036/viewer/2022062307/555bb455d8b42a0d0f8b4b21/html5/thumbnails/22.jpg)
Context Class size
Large (30-300)
Content
Skills \ facts
Mode of instruction
Blended / on-line. Computer tested.
![Page 23: formative e-assessment: cases, patterns and scenarios](https://reader036.vdocuments.us/reader036/viewer/2022062307/555bb455d8b42a0d0f8b4b21/html5/thumbnails/23.jpg)
Solution
![Page 24: formative e-assessment: cases, patterns and scenarios](https://reader036.vdocuments.us/reader036/viewer/2022062307/555bb455d8b42a0d0f8b4b21/html5/thumbnails/24.jpg)
Feedback on Feedback
(Linda McGuigan)
http://purl.org/planet/Patterns/FeedbackonFeedback
![Page 25: formative e-assessment: cases, patterns and scenarios](https://reader036.vdocuments.us/reader036/viewer/2022062307/555bb455d8b42a0d0f8b4b21/html5/thumbnails/25.jpg)
Good feedback should -
Alert learners to their weaknesses.Diagnose the causes and dynamics of these.Include operational suggestions to improve the learning experience.Address socio-emotive factors.
Tutors know this, but are pressed for time. Or not aware of their feedback strategies
Large teaching organisations are not equipped to provide tutors with personal feedback on their teaching
Problem
![Page 26: formative e-assessment: cases, patterns and scenarios](https://reader036.vdocuments.us/reader036/viewer/2022062307/555bb455d8b42a0d0f8b4b21/html5/thumbnails/26.jpg)
Context
Large scale, technology supported, graded courses
many tutors instructing many students.
Feedback is mediated by technology that allows it to be captured and processed in real time
Topic of study is subject to both grading and formative feedback.
![Page 27: formative e-assessment: cases, patterns and scenarios](https://reader036.vdocuments.us/reader036/viewer/2022062307/555bb455d8b42a0d0f8b4b21/html5/thumbnails/27.jpg)
SolutionEmbed a mechanism in the learning and teaching system
that regularly captures tutor feedback, analyses it, and presents them with graphical representation of the types of feedback they have given. Ideally, this should also include constructive advice as to how to shift from less to more effective forms.
In computer supported environments (e.g. VLEs), this mechanism could be integrated into the system, providing tutors with immediate analysis of their feedback, as well as long-term aggregates.
![Page 28: formative e-assessment: cases, patterns and scenarios](https://reader036.vdocuments.us/reader036/viewer/2022062307/555bb455d8b42a0d0f8b4b21/html5/thumbnails/28.jpg)
Classroom Display
http://purl.org/planet/Patterns/Classroomdisplay
![Page 29: formative e-assessment: cases, patterns and scenarios](https://reader036.vdocuments.us/reader036/viewer/2022062307/555bb455d8b42a0d0f8b4b21/html5/thumbnails/29.jpg)
Problem
Rewards participation.Relates to learner's personal experiences.Window on student conceptions.
Using learner generated Using learner generated content..content..
Needs to collate works in a single easy to access location.Learners uncomfortable about presenting their work in publicLegal or other restrictions on sharing work.
![Page 30: formative e-assessment: cases, patterns and scenarios](https://reader036.vdocuments.us/reader036/viewer/2022062307/555bb455d8b42a0d0f8b4b21/html5/thumbnails/30.jpg)
Context Class size:
Small / medium (6-60)
Mode of instruction:
Blended (preferable)
Time frame
Continuous, over a period
Pedagogy
Involves construction / media production
![Page 31: formative e-assessment: cases, patterns and scenarios](https://reader036.vdocuments.us/reader036/viewer/2022062307/555bb455d8b42a0d0f8b4b21/html5/thumbnails/31.jpg)
Solution
![Page 32: formative e-assessment: cases, patterns and scenarios](https://reader036.vdocuments.us/reader036/viewer/2022062307/555bb455d8b42a0d0f8b4b21/html5/thumbnails/32.jpg)
Augmented domain map
![Page 33: formative e-assessment: cases, patterns and scenarios](https://reader036.vdocuments.us/reader036/viewer/2022062307/555bb455d8b42a0d0f8b4b21/html5/thumbnails/33.jpg)
Example scenario When using Try Once Refine OnceTry Once Refine Once, there is a risk that
high-achievers do not receive feedback.
So -
Use Showcase Learning Showcase Learning to celebrate students’ work and provoke feedback from peers and tutors.
Use Feedback on Feedback Feedback on Feedback to alert tutors to the problem.
![Page 34: formative e-assessment: cases, patterns and scenarios](https://reader036.vdocuments.us/reader036/viewer/2022062307/555bb455d8b42a0d0f8b4b21/html5/thumbnails/34.jpg)
Conclusions Tip of the iceberg
Practitioners (educational / software) acknowledge the value of patterns, when served with side dishes of cases + scenarios
Collaborative elicitation of patterns from cases could be a potent form of professional development.
![Page 35: formative e-assessment: cases, patterns and scenarios](https://reader036.vdocuments.us/reader036/viewer/2022062307/555bb455d8b42a0d0f8b4b21/html5/thumbnails/35.jpg)
35
Thank you
The Formative e-Assessment project:
http://projects.lkl.ac.uk/feasst
Final report
http://telearn.noe-kaleidoscope.org/open-archive/browse?resource=1875
Yishay Mor
http://people.lkl.ac.uk/yishay
This presentationhttp://www.slideshare.net/yish/formative-eassessment-cases-patterns-and-scenarios