Brookes Assessment Compact OCSLD and ASKe Centre for Excellence
Chris Rust and Greg Benfield
Assessment Standards Knowledge exchange & OCSLD
Definitions
Assessment
All judgements made about the work of a student and/or their skills, abilities and progress, and the associated provision of feedback.
Compact
A formal agreement or covenant, indicating intent, but not enforceable by law (cf contract)
Assessment Standards Knowledge exchange & OCSLD
Starting thoughts
What do want your assessment and feedback processes to achieve?
Are you achieving these goals, and if not what are the barriers that are preventing them?
Youtube – The Five Minute University
(at http://www.youtube.com/watch?v=kO8x8eoU3L4&feature=related)
Assessment Standards Knowledge exchange & OCSLD
Tenet 1 - central to learning
“Assessment is at the heart of the student experience”(Brown, S & Knight, P., 1994)
“From our students’ point of view, assessment always defines the actual curriculum”(Ramsden, P.,1992)
“Assessment defines what students regard as important, how they spend their time and how they come to see themselves as students and then as graduates.........If you want to change student learning then change the methods of assessment”(Brown, G et al, 1997)
Assessment Standards Knowledge exchange & OCSLD
Tenet 2 - relational
Feeling valued is often important to the way individuals respond to each other. FDTL Feedback project found that a sense of being valued by staff meant that students were more likely to engage with their feedback, to read it and use it.
One student commented, "You can go through the whole semester and the teacher still doesn't know your name". In this case, she said, she was unlikely to bother to read the feedback that "teacher" gave her.
Creating opportunities for dialogue between staff and students is likely to support this relational dimension of assessment and feedback.
Assessment Standards Knowledge exchange & OCSLD
Tenets 3 & 4
3. Joint responsibility, i.e. students must also take responsibility
4. The skills of self and peer-assessment should be graduate attributes
Students need to understand the assessment standards and criteria to be able to self-evaluate their work in the act of production itself (Sadler,1987)
The ability to make informed judgements on the work of self and others is a key graduate attribute (Boud, 2009)
Assessment Standards Knowledge exchange & OCSLD
Tenet 5 - ‘assessment literate’ communities of assessment practice
‘making sense of the world’ is a social and collaborative activity
(Vygotsky, 1978)
Tacit knowledge is experience-based and can only be revealed
through the sharing of experience – socialisation processes
involving observation, imitation and practice (Nonaka, 1991)
Dialogue and participatory relationships are key elements of
engaging students with assessment feedback (ESwAF FDTL, 2007)
An indispensable condition for improvement in student learning is
that “the student comes to hold a concept of quality roughly similar
to that held by the teacher” (Sadler, 1989)
Assessment Standards Knowledge exchange & OCSLD
Tenet 5 - ‘assessment literate’ communities of assessment practice (contd.)
social constructivism
the social-constructivist view of learning argues that knowledge is shaped and evolves through increasing participation within different communities of practice
the social constructivist process model of assessment argues that students should actively engage with every stage of the assessment process in order that they truly understand the requirements of the process, and the criteria and standards being applied, and should subsequently produce better work
(Rust, O’Donovan & Price, 2005) 30
Assessment Standards Knowledge exchange & OCSLD
Active engagement with feedback
Explicit Criteria
Completion and submission of work
Students Active engagement with criteria
Assessment Standards Knowledge exchange & OCSLD
Active engagement with feedback
Explicit Criteria
Completion and submission of work
Students Active engagement with criteria
Assessment Standards Knowledge exchange & OCSLD
Engaging students with criteria and standards
Get students actively using the criteria through a developmental combination of:
Marking exercisesSelf-assessmentPeer-feedbackPeer-assessmentPossibly creating and negotiating criteria
Assessment Standards Knowledge exchange & OCSLD
Engaging students through early assessment:
(Also engender good study habits; diagnose areas of individual difficulty; clarifies expectations)
“Setting students assignments as soon as they arrive at university could help cut dropout rates………integrate students into university life as quickly as possible. This involves making them aware of the quality and quantity of work expected from them…..The freedom…is too much for some and they probably need more structure in the first year.”
Higher, 24/1/03
Reporting Student Transition and Retention (STAR) project
(in 5 universities)
Assessment Standards Knowledge exchange & OCSLD
Expectations and notions of ‘quality’ can remain more or less the same during a programme of
study... The learning journey of a programme can be seen as: A series of modular stepping stones of similar difficulty, involving: Incremental movement between zones of proximal development
(Vygotsky, 1978)
Phase 1 Phase 2 Phase 3
Lea
rnin
g de
velo
pmen
t
Assessment Standards Knowledge exchange & OCSLD
Radical movement involving zones of discomfort, ‘threshold concepts and troublesome knowledge’ (Meyer and Land, 2006)
Changes in ‘epistemology and knowledge structures’ (Basil Bernstein in Moore et al, 2006)
Phase 1 Phase 2 Phase 3
Lea
rnin
g de
velo
pmen
t….or change radically
Assessment Standards Knowledge exchange & OCSLD
Activity
1.Skim read the ASKe leaflets
2.Discuss: to what extent do you already and/or could you use methods like these to engage students with the assessment criteria and standards?
3.Are there key points, besides entry, that such methods could support student understanding and progression?
20 + 30 then coffee
Assessment Standards Knowledge exchange & OCSLD
Active engagement with feedback
Explicit Criteria
Completion and submission of work
Students Active engagement with criteria
Assessment Standards Knowledge exchange & OCSLD
Enhancing student learning through effective feedback
Feedback is the most powerful single pedagogic influence that makes a difference to student achievementHattie (1987) - in a comprehensive review of 87 meta-analyses of studies
Feedback has extraordinarily high and consistently positive effects on learning compared with other aspects of teaching or other interventions designed to improve learningBlack and Wiliam (1998) - in a comprehensive review of formative assessment
Students are hungry for feedback to develop their learning(Higgins et al, 2002)
Assessment Standards Knowledge exchange & OCSLD
Feedback problems Unhelpful feedback (Maclellan, 2001) Too vague (Higgins, 2000) Subject to interpretation (Ridsdale, 2003) Not understood (e.g. Lea and Street, 1998) Don’t read it (Hounsell, 1987) Damage self-efficacy (Wotjas, 1998) Has no effect (Fritz et al, 2000) Seen to be too subjective (Holmes & Smith, 2003)
It is not enough to improve feedback as a monologue; we must make it a dialogue (Nicol, 2009)
Improving feedback - prepare students Aligning expectations (of staff & students, & between teams of
markers)
- often a mismatch of expectations e.g correcting errors, advice for the future, diagnosis of general problems, comments specific only to that piece of work. These mismatches occur frequently with no particular pattern about who holds which view/perspective but problems arise when the the two don't coincide. Purpose of feedback may vary from assignment to assignment so would need to be clarified each time. (Price et al, in print; Freeman & Lewis, 1998)
Identifying all feedback available (especially oral) Encourage the application of feedback
- e.g. in a subsequent piece of work the student is required to show how they have used prior feedback to try to improve their work (and possibly some marks allocated for this).
Require and develop self-assessment
Assessment Standards Knowledge exchange & OCSLD
Activity
Discuss:
How do you currently prepare students to make use of feedback and to what extent could you do more through the introduction/development of some of these ideas?
20+20
Assessment Standards Knowledge exchange & OCSLD
Improving feedback - ensure it is fit for purpose Ensure students have MOM - Motive, Opportunity, Means (Angelo, 2007) Draft-plus-rework - feedback effort (for markers and students) is located at
the draft stage, and possibly only a summative grade is given for the final submission
Improve the linkage of assessment strategies across programmes and between modules/units
Increase opportunities for dialogue - in-class discussion of exemplars, peer-review discussions supported by tutors, learning-sets, etc.
Identify what is feasible in a given assessment context - written feedback can often do little more than ‘diagnose’ development issues and then direct students to other resources for help and support
Consider the role of marks - they obscure feedback Ensure it is timely - ‘quick and dirty’ generic feedback, feedback on a draft,
MCQs & quizzes, etc. Using technology – see http://tinyurl.com/tfaproject Reduce over-emphasis on written feedback - oral can be more effective
(McCune, 2004). See the Sounds Good website at: http://sites.google.com/site/soundsgooduk/
Review resource allocations (N.B. OU 60%)
Assessment Standards Knowledge exchange & OCSLD
Activity
Discuss:
What are you already doing? Which of these ideas could you introduce/develop to help students engage with their feedback?
20+20 then lunch
Assessment Standards Knowledge exchange & OCSLD
Implementation 2.6, 2.7, 2.9 - ‘assessment literate’ communities of assessment
practice Programmes to include activities (e.g. marking exercises, self and peer-
assessment, etc.) specifically designed to: involve students in the assessment process encourage dialogue between students and their tutors encourage dialogue between students and their peers, and ultimately develop their abilities to make their own informed judgements
(assessment literacy as a graduate skill)
Staff peer discussion of assessment at course design stage, moderation, and staff development
Student involvement to include contribution to the development of assessment policy at course and programme level through the established processes and student representative system (3.4)
Assessment Standards Knowledge exchange & OCSLD
Implementation 2.6, 2.7, 2.9 - ‘assessment literate’ communities of assessment
practice Programmes to include activities (e.g. marking exercises, self and
peer-assessment, etc.) specifically designed to: involve students in the assessment process encourage dialogue between students and their tutors encourage dialogue between students and their peers, and ultimately develop their abilities to make their own informed judgements
(assessment literacy as a graduate skill)
Staff peer discussion of assessment at course design stage, moderation, and staff development
Student involvement to include contribution to the development of assessment policy at course and programme level through the established processes and student representative system (3.4)
Assessment Standards Knowledge exchange & OCSLD
Active engagement with feedback
Explicit Criteria
Completion and submission of work
Students Active engagement with criteria
Assessment design & development of explicit criteria
Tutor discussion of criteria
Marking and moderation
Staff Assessment guidance to staff
Assessment Standards Knowledge exchange & OCSLD
Active engagement with feedback
Explicit Criteria
Completion and submission of work
Students Active engagement with criteria
Assessment design & development of explicit criteria
Tutor discussion of criteria
Marking and moderation
Staff Assessment guidance to staff
Rust C.,O’Donovan B & Price., M (2005)
Assessment Standards Knowledge exchange & OCSLD
Implementation: 2.1-2.4: course design
Constructive alignment
3-stage course design:
What are “desired” outcomes? What teaching methods require students to behave in
ways that are likely to achieve those outcomes? What assessment tasks will tell us if the actual outcomes
match those that are intended or desired?
This is the essence of ‘constructive alignment’ (Biggs, 1999)
Assessment Standards Knowledge exchange & OCSLD
Implementation 2.1-2.4: course design (contd.)
Emphasis on programme outcomes
Slowly learnt academic literacies require rehearsal and practice throughout a programme (Knight & Yorke, 2004)
The achievement of high-level learning requires integrated and coherent progression (based on programme outcomes)
Where there is a greater sense of the holistic programme students are likely to achieve higher standards than on more fragmented programmes (Havnes, p. 2007)
Assessment Standards Knowledge exchange & OCSLD
Implementation 2.1-2.4 & 2.7: course design (contd.)
Assessment for learning
Redress the balance from summative to formative assessment:
“…students become more interested in the mark and less interested in the subject over the course of their studies.” (Newstead 2002, p2)
Design out plagiarism and ensure authenticity Avoid bias and unnecessary disadvantage Programmes to produce assessment schedules of
summative assessment, and make every effort to avoid the concentration of assessment deadlines (API requirement)
Assessment Standards Knowledge exchange & OCSLD
Activity
Discuss:
How could you redesign your programme to redress the balance from summative to formative, and focus the emphasis on the assessment of programme outcomes?
20+20
Assessment Standards Knowledge exchange & OCSLD
Programme Assessment Task List Take a programme-based approach to assessment (2.1.; 2.2; 2.3; 2.7)• Develop a constructively aligned rationale for the programme assessment• Clarify & agree the programme’s expectations of students and approach to learning and
assessment• Develop an assessment schedule showing assessment timing, load and type and
feedforward opportunities
Develop assessment for learning (2.4; 2.5; 2.6)• Increase formative assessment particularly in Yrs. 1 & 2• Communicate clear and high expectations and foster students’ self-evaluative ability
through the programme• Determine those areas/literacies that are commonly ‘slowly learnt’ and allow for slow
learning• Provide opportunities for dialogue on student learning
Create a learning environment that fosters involvement & engagement (2.6; 2.9)
• Value, develop & prioritise processes that build relationships and involvement
Assessment Standards Knowledge exchange & OCSLD
Activity
Looking at the programme assessment task list, identify:
Aspects that your programme is already doing well
Gaps that your programme needs to address
How will you take this forward?
40
Assessment Standards Knowledge exchange & OCSLD