assessment for learning: the benefits of generating feedback david nicol professor of higher...
TRANSCRIPT
![Page 1: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/1.jpg)
Assessment for learning: the benefits of generating feedback
David Nicol Professor of Higher Education
Centre for Academic Practice and Learning Enhancement (CAPLE]
Director, Peer project (www.reap.ac.uk)University of Strathclyde, Scotland
University of Cadiz, March 9-11, 2011
![Page 2: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/2.jpg)
Plan
BackgroundRe-engineering Assessment Practices (REAP) project www.reap.ac.uk/reap/index.html Concepts and example of practiceInstitutional considerationsPEER project www.reap.ac.uk/peer.aspx Discussion
![Page 3: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/3.jpg)
Background
Departments and faculties: educational improvement projects, including REAP projectPolicy/strategy: development of educational policy and principles of assessment & feedback (based on REAP)Students: ‘Feedback as dialogue’ campaignQuality procedures:– redesigning course documentation and review proceduresPEER project – developing students ability to evaluate the quality of each other’s work
See www.reap.ac.uk
![Page 4: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/4.jpg)
REAP : Re-engineering Assessment Practices
Scottish Funding Council for Universities (£1m)3 Universities - Strathclyde, Glasgow & Glasgow CaledonianLarge 1st year classes (160-600 students)A range of disciplines (19 modules ~6000 students)Many technologies: online tests, simulations, discussion boards, e-portfolios, e-voting, peer/feedback software, VLE, online-offlineLearning quality and teaching efficienciesAssessment for learner self-regulationwww.reap.ac.uk
![Page 5: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/5.jpg)
Background (1)
Gibbs, G. & Simpson, C (2004) Conditions under which assessment supports students’ learning, Learning and Teaching in Higher Education, 1, 3-31.
See:
Formative Assessment in Science Teaching (FAST) project at: http://www.open.ac.uk/science/fdtl
![Page 6: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/6.jpg)
Gibbs and Simpson (2004)
Assessment tasks [Conditions 1-4]1. Capture sufficient study time (in and out of
class)2. Are spread out evenly across timeline of study3. Lead to productive activity (deep vs surface)4. Communicate clear and high expectations
i.e concern here is with ‘time on task’ how much work students do - their active engagement in study
![Page 7: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/7.jpg)
Background (2)Literature Review
Nicol, D. & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 34 (1), 199-218
BackgroundStudent Enhanced Learning through Effective Feedback [SENLEF] project funded by HE AcademyREAP project: www.reap.ac.uk
![Page 8: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/8.jpg)
Rethinking assessment and feedback1. Consider self and peers as much as the teacher as sources of assessment and feedback
Tap into different qualities than teacher can provide Saves time Provides considerable learning benefits (lifelong
learning)
2. Focus on every step of the cycle: Understanding the task criteriaApplying what was learned in action
3. Not just written feedback: Also verbal, computer, vicarious, formal and informal
![Page 9: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/9.jpg)
Seven principles of good feedback
Good feedback:1. Clarifies what good performance is (goals, criteria,
standards).2. Facilitates the development of reflection and self-
assessment in learning 3. Delivers high quality information to students: that
enables them to self-correct4. Encourages student-teacher and peer dialogue
around learning5. Encourages positive motivational beliefs & self
esteem6. Provides opportunities to act on feedback7. Provides information to teachers that can be used
to help shape their teaching (making learning visible)
Source: Nicol and Macfarlane-Dick (2006)
![Page 10: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/10.jpg)
EMPOWERMENT/SELF-REGULATION
ENGAGEMENT
Principle 1: Clarify what good performance is (the context of dialogue)
• Students create criteria• Students add own criteria • Students identify criteria
from samples of work• Exemplars of different
performance levels provided
• Students rephrase criteria in own words
• Provide document with criteria
![Page 11: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/11.jpg)
EMPOWERMENT/SELF-REGULATION
ENGAGEMENT
Principle 2: Facilitate reflection and self-assessment in learning
• Students create MCQs including feedback for right and wrong (distractors) answers
• Students self-assess using MCQs and confidence ratings
• Students self-assess own performance using online MCQs
![Page 12: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/12.jpg)
Two meta principles
1. Meta-PRINCIPLE 1: time and effort on task (structured engagement) i.e. steers on how much work to do and when – Gibbs and Simpson 4 conditions
2. Meta-PRINCIPLE 2: developing learner self-regulation (empowerment/self-regulation) i.e steers to encourage ownership of learning – the seven principles discussed above.
Key task for teacher is to balance 1 and 2
![Page 13: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/13.jpg)
Example:Psychology
![Page 14: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/14.jpg)
Psychology
560 first year students6 topic areas (e.g. personality, classical conditioning), 48 lectures, 4 tutorials, 12 practicalsAssessment; 2 x MCQs (25%), tutorial attendance (4%), taking part in experiment (5%), essay exam (66%)
![Page 15: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/15.jpg)
Problems identified
No practice in writing skills but required in the examMore detail provided in lectures than mentioned in exams (not enough independent reading)No feedback except on Multiple Choice Questions (percent correct)Didn’t want to increase staff workload
![Page 16: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/16.jpg)
Psychology Redesign
Discussion board in Learning Management SystemStudents in 85 discussion groups of 7-8, same groups throughout yearAlso open discussion board for classFriday lectures cancelled – discover material themselvesSeries of online tasks
![Page 17: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/17.jpg)
Structure of group tasks
6 cycles of 3 weeks (one cycle x major course topic)
• First week: ‘light’ written task (e.g. define terms) = 7 short answers (all answer)
• Second week = guided reading • Week three: ‘deep’ written task: students
collaborate in writing a 700-800 word essay on the same topic.
Within each week:• The Monday lecture – introducing material• Immediately after lecture, task posted online –
for delivery the following Monday• Model answers (selected from students) posted
for previous week’s task
![Page 18: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/18.jpg)
The teaching role
Participation in the discussions was compulsory but not marked (in subsequent years there was 2% mark for participation)Course leader provided general feedback to the whole class – often motivational He encouraged students to give each other feedbackThe group discussions were not moderated but monitored for participation
![Page 19: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/19.jpg)
An example of ‘deep’ task
The Task – 800 word essay:
Assess the strengths and weaknesses of Freud’s and Eysenck’s theories of personality. Are the theories incompatible?readings suggestedquestions provided which all students should try
![Page 20: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/20.jpg)
Relation to the Gibbs & Simpson’s four assessment conditions
1. Tasks require significant study out of class (condition 1)
2. Tasks are distributed across topics and weeks (condition 2)
3. They move students progressively to deeper levels of understanding (condition 3)
4. There are explicit goals and progressive increase in challenge (condition 4)
![Page 21: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/21.jpg)
Relation to 7 feedback principles
1. Standard format and model answers provide progressive clarification of expectations (principle 1)
2. Students encouraged to self-assess against model answer (principle 2)
3. Course leader provides motivational and meta-level feedback and selects model answers (principle 3)
4. Online peer discussion aimed at reaching consensus is core feature of design about response (principle 4)
5. Focus on learning not just marks, sense of control/challenge enhanced motivation (principle5)
6. Repeated cycle of topics and tasks provide opportunities to act on feedback (principle 6)
7. VLE captures all interactions allowing course leader to monitor progress and adapt teaching (principle 7)
![Page 22: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/22.jpg)
Benefits
Students worked exceptionally hard Written responses of exceedingly high standard Students took responsibility for learningHigh levels of motivation: atmosphere in class improvedOnline interactions showed powerful ‘scaffolding’ and community buildingFeedback with 560 students through peer and self-feedback (model answers)Easy for tutors to monitor participationImproved mean exam performance (up from 51-59%, p<0.01) weaker students benefit most
![Page 23: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/23.jpg)
Has it worked?
I read more about Psychology and read it earlier in each semester than I would have done without
the online projects
0
10
20
30
40
50
60
1 2 3 4 5
agreed-disagreed
Nu
mb
ers
![Page 24: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/24.jpg)
I found that reading other people's contributions helped me understand Psychology
010203040
50607080
1 2 3 4 5
agreed-disagreed
nu
mb
ers
Series1
![Page 25: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/25.jpg)
I learned more in Psychology because of online projects than I did in my other subjects
0
10
20
30
40
50
60
1 2 3 4 5
agreed-disagreed
nu
mb
ers
Series1
![Page 26: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/26.jpg)
Guidelines for Implementation
1. A single principle or many?2. Tight-loose – maintain fidelity to the principles
(tight) but encourage disciplines to develop their own techniques of implementation (loose)
3. Balance teacher feedback with peer and self-generated feedback
4. Focus on developing students’ own ability for critical evaluation
5. Create opportunities for ‘learning communities’ to emerge
6. The more actively engaged students are, the better the course design
![Page 27: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/27.jpg)
Developments since REAP
1. Principles of Assessment and Feedback approved by University Senate and embedded in policy (2008)
2. Use of principles to inform curriculum renewal and Quality Assurance processes
3. ‘Feedback as Dialogue’ campaign to gain commitment of students
4. PEER Project (Peer Evaluation in Education Review)5. Work on the way we document our courses and
programmes
![Page 28: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/28.jpg)
![Page 29: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/29.jpg)
![Page 30: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/30.jpg)
Peer Review in Education Evaluation [PEER]
The aims of the PEER project are to:Review evidence base for peer reviewDevelop educational designs for peer review (and self-review)Identify software support for peer reviewPilot implementations of peer review with large student numbersProduce guidelines for higher education – why do it, how to do it, pitfalls and solutions and software possibilities.
see http://www.reap.ac.uk/peer.aspx
![Page 31: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/31.jpg)
PEER Project: Core Ideas
All graduate development in higher education requires that:
1. Students learn to evaluate critically the quality and impact of their own work (e.g. academic texts, problem solutions, designs)
2. Students learn to evaluate critically the work of others (e.g. peer review and peer feedback)
Ref:Nicol, D (2010) The foundation for graduate attributes: developing self-regulation through self and peer assessment. www.reap.ac.uk/peer.aspx
![Page 32: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/32.jpg)
The research on peer feedback
Mainly about peers marking each others workWhere peer feedback is the focus it is usually about peers augmenting teacher feedback by increasing the quantity and variety of feedback
![Page 33: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/33.jpg)
The focus of PEER project
Scenarios where students make evaluative judgements about the work of peers and provide a feedback commentary, usually writtenNot talking about .....collaborative tasks where students give each other informal feedback.....scenarios where the focus is on students evaluating each other’s performance in group working.....scenarios where the focus is on students grading/marking each other’s work although some rating of performance might be part of the peer design
![Page 34: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/34.jpg)
Benefits of feedback construction (1)
Constructivist rather than transmission perspective
Producing is cognitively more demanding than reading feedback from others: students cannot be passiveStudents actively exercise assessment criteria (process/reprocess) from many different perspectivesSee how others tackle assignments and learn that quality can be produced in different waysDevelop writing skills through commentariesDeepens critical thinking in the discipline
![Page 35: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/35.jpg)
Benefits of feedback construction (2)
Learn to assess own work, to monitor and evaluate their own productions - same skills involvedDevelops self-regulation – for life beyond university.Changes the power relationship in class (shares responsibility across students).Can develop learning communities
![Page 36: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/36.jpg)
Giving and receiving feedback
Giving and receiving feedback within same domain enriches the whole process.Students are able to compare the feedback they receive with the feedback they produceAnd learn how different reviewers perceive their work – a single source of feedback is never satisfactoryCan develop a sense of shared responsibility across students for learning
![Page 37: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/37.jpg)
Example: peer feedback
• Students write essay on one topic from three • Each student provides feedback on three essays in another topic
anonymously using rubric • The rubric: write a short summary of the essay, comment on and
rate (four point scale) the structure, arguments, evidence, writing, suggest ways of improving the essay.
• Students receive peer reviews of their own essays• Then review, comment on and rate their own essay using same
rubric.• Graded: for participating in the task, for their own essay and for
their review of it.• Finally students rate 3 reviews (on others’ work) and comment on
how useful they think they would be to author.• Software used to support administration
![Page 38: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/38.jpg)
Six learning opportunities in peer review
1. Producing the target assignment2. Reviewing and producing peer feedback3. Receiving feedback from peers4. Responding to peer reviews (on own reviews of
those given to others)5. Self-reviewing one’s own work (e.g. after peer
review)6. Receiving feedback from the teacher – e.g. on
the assignment, peer review or self-review
1-3 comprise the essentials of peer review while 4-6 are further options.
![Page 39: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/39.jpg)
Principles of effective peer reviewPeer review should
1. Engage students in active use of criteria and standards 2. Involve students in constructing commentaries in relation
to peer judgements, not just marks3. Lead to practice in both analytic (componential) and
holistic (configurational) judgements about quality4. Facilitate dialogue around the object and quality of the
review5. Ensure an atmosphere of mutual trust and respect6. Integrate self-review activities within peer review designs7. Ensure that making peer judgements is a regular activity
and not a one-off event
www.reap.ac.uk/PEER.aspx
![Page 40: Assessment for learning: the benefits of generating feedback David Nicol Professor of Higher Education Centre for Academic Practice and Learning Enhancement](https://reader035.vdocuments.us/reader035/viewer/2022062421/56649c3a5503460f948e33f1/html5/thumbnails/40.jpg)
Some of my PublicationsNicol, D (2010) The foundation for graduate attributes: developing self-regulation thorugh self and peer assessment, QAA Enhancement Themes, ScotlandNicol, D (2010) From monologue to dialogue: Improving written feedback in mass higher education. Assessment and Evaluation in Higher Education. 35(5), 501 -517 Nicol, D and Draper, S (2010) A blueprint for transformational organisational change in HE: REAP as a case study, Published by the Higher Education Academy, UK (see reap.ac.uk website)Nicol, D (2009) Transforming assessment and feedback: Enhancing integration and empowerment in the first year, Published by Quality Assurance Agency, Scotland (http://www.enhancementthemes.ac.uk/documents/firstyear/FirstYear_TransformingAssess.pdfNicol, D (2009), Assessment for learner self-regulation: Enhancing achievement in the first year using learning technologies, Assessment and Evaluation in Higher Education, 34(3), 335-352Nicol, D (2007), Laying the foundation for lifelong learning: cases studies of technology supported assessment processes in large first year classes, British Journal of Educational Technology, 38(4), 668-678Nicol, D (2007) E-assessment by design: using multiple-choice tests to good effect, Journal of Further and Higher Education.31(1), 53-64.Nicol, D. & Milligan, C. (2006), Rethinking technology-supported assessment in relation to the seven principles of good feedback practice. In C. Bryan and K. Clegg, Innovations in Assessment, Routledge.Nicol, D, J. & Macfarlane-Dick (2006), Formative assessment and self-regulated learning: A model and seven principles of good feedback practice, Studies in Higher Education, 31(2), 199-218.See www.reap.ac.uk/resources.aspx for copies.