Assessment for Learning – the story so far

Challenging the Ascendancy of Summative Assessment

Val Brooks

Traditional approach• “The giving of marks and the grading

functions are over-emphasised, while the giving of useful advice and the learning function are under-emphasised” (Black and Wiliam, 1998, p. 6)

• With little opportunity or incentive to develop formative assessment, many teachers have modelled their own classroom assessment on summative assessment

TRADITIONAL APPROACH

PLAN TEACH ASSESS

(Traditionally, assessment treated as a terminal activity)

Mid-1990s onwards

• Upsurge of interest in AfL in England, initially within the academic community

• National government convinced of the importance of research findings

• Teachers spurred on by policy initiatives and by their own positive experiences of AfL

Aims

To identify

• the principles underpinning AfL and how these principles differentiate formative from summative assessment

• studies which have been instrumental in bringing about change

• key ways in which teachers are changing how they teach and assess

Note on Terminology

• Assessment for Learning (AfL) = Formative Assessment

• Assessment of Learning (AoL) = Summative Assessment

• Terms used interchangeably but with AfL and AoL representing popular usage

Principles underpinning Practice: What differentiates AfL from AoL?

1. Using assessment for different purposes– SA measures the learning that has taken

place – The goal of AfL is the improvement of

learning

FORMATIVE ASSESSMENT• Takes place during the learning process and

is integral to it. The main beneficiaries are the teacher and pupil because they are provided with FEEDBACK about areas of strength and weaknesses. Formative assessment should have a FEEDFORWARD function, informing decisions about how teaching should be adapted and how learning can be taken forward.

Quotation from a child

“If I only do what I did before then I’ll only get what I got before”

(Using feedback for feedforward purposes is essential if real learning gains are to be made)

Principles underpinning Practice: What differentiates AfL from AoL?

1. Focus on improving teaching and learning

2. Pupils actively involved (e.g. through self and peer assessment)

Principles underpinning Practice: What differentiates AfL from AoL?

1. Focus on improving teaching and learning

2. Pupils actively involved (e.g. through self and peer assessment)

3. Timing: as well as taking periodic snapshots of attainment, assessment forms an ongoing, integral element of teaching

Using assessment to inform planning

“The most important single factor influencing learning is what the learner already knows: ascertain this and teach him [her] accordingly.”

(Ausubel 1968)

Principles underpinning Practice: What differentiates AfL from AoL?

1. Focus on teaching and learning

2. Pupils actively involved

3. Timing: assessment forms an ongoing, integral element of teaching

4. Rich, detailed feedback replaces summary information

Principles underpinning Practice: What differentiates AfL from AoL?

5. Effective communication with a small, intimate audience – not the broader audience interested in the results of summative assessment

Distinguishing AfL from AoL

FORMATIVE• TIMING (ongoing and

continuous feature of teaching and learning)

• PURPOSE (Enhancing teaching and learning)

• FORM (Written/verbal feedback + feedforward)

• AUDIENCE (Pupils, teachers and parents)

SUMMATIVE• TIMING (Snapshot taken

at particular point in time)

• PURPOSE (Measuring the learning that has taken place)

• FORM (Summary e.g. a grade, level, mark or percentage)

• AUDIENCE (Other interested parties: FE and HE, government, employers)

TRADITIONAL APPROACH

PLAN TEACH ASSESS

(Traditionally, assessment was a terminal activity i.e.summative)

INNOVATIVE APPROACH

PLAN TEACH ASSESS

ASSESS ASSESS

(Assessment is an ongoing and integral part of teaching and learning i.e. formative)

Teaching/learning/assessment:

a cyclical process

Plan

Teach

Assess Assess

Assess

Research which was instrumental in stimulating change

Version 1 (for the academic community)Black, P. and Wiliam, D. (1998) “Assessment and

Classroom Learning” Assessment in Education 5 (1), pp. 7-78.

Version 2 (for policy makers and practitioners)Black, P. and Wiliam, D. (1998) Inside the Black

Box: Raising Standards through Classroom Assessment (London, King’s College).

Distinctive Features of Black & Wiliam’s Research

• Focus - neglected topic of formative assessment

• Literature review (aim: to ensure that policy and practice are evidence-based)

• Scale - in-depth survey of 250 separate studies

• Scope - international/ covers learners of all ages and types/different subjects

Distinctive Features of Black & Wiliam’s Research

• Emphasis on quasi-experimental studies which produced measures of effect

• Well-suited to a climate where there is a preoccupation with measurable outcomes in the form of improved test and examination results.

Principal findings of Black and Wiliam (1998)

• No evidence of an adverse effect on attainment• Positive effect sizes ranging between 0.4 and 0.7• Compares favourably with effect sizes produced

by other initiatives designed to raise attainment• AfL reduces the spread of attainment whilst

raising it overall

Issues Arising from the Research• Some of the studies on which conclusions are

based did not take place as part of normal classroom life

• “It is hard to see how any innovation in formative assessment can be treated as a marginal change in classroom work” (Black and Wiliam 1998, p. 16)

• “… formative assessment is not well understood by teachers and is weak in practice” (Black and Wiliam, 1998)

• So for them the question was not “Does it work?” but “How do we get it to happen?”

Follow-up Study

• KMOFAP (Kings, Medway, Oxfordshire Formative Assessment Project)

• Empirical research involving 6 secondary schools and teachers of 3 subjects (Science, Maths and English)

Research focus

Original• Questioning• Feedback• Sharing criteria• Self-assessment

Revised• Questioning• Feedback through

marking• Peer and self-

assessment• The formative use of

summative tests

Main Publications arising from the Research

• Black, P., Harrison, C., Lee, C., Marshall, B. and Wiliam, D (2003) Assessment for Learning: Putting it into Practice (Open University Press)

• Black, P. et al. (2002) Working Inside the Black Box: Assessment for Learning in the Classroom (London, King’s College).

AfL and Planning: Research Evidence

• Large-scale study involving 800+ American kindergarten children

• Children came primarily from economically disadvantaged homes

• 8-week experimental programme– 29 teachers in experimental group trained to use

assessment at the planning stage to match provision to needs

– 27 teachers in the control group adopted their usual approach

Findings• Pre- and post-tests (reading ability,

mathematics and science) used to measure effects

• Children in the experimental group achieved significantly better results in post-tests than those in control group

• Incidence of special education referrals– Control group: 1 in every 3.7 – Experimental group: 1 in 17

AfL and Planning: Putting it into Practice

• Creation of flexible plans containing space to respond to feedback

• Baseline assessment of a topic that is about to be introduced– Use of end-of-topic test as a pre-test– Concept-mapping

AfL and Feedback: Research EvidenceIsraeli study investigating the effects of

different types of feedback on pupils’ motivation and attainment

• Comment only (individually composed comment on the level of match between a child’s work and the assessment criteria which were explained to all beforehand)

• Individual comment plus grade• Grade only

Group Performance Motivation

Comment only Raised and the improvement was sustained over sequence of tasks

Influenced by ability:

High achievers maintained a high level of interest irrespective of feedback type

• Low achievers who received grades quickly lost interest

Grade + comment

Steady decline across tasks

Grade only Initial improvement which was not sustained

Note: both feedback types which used grades were associated with a deterioration in performance and motivation.

AfL and Feedback: Putting it into PracticeComment-only marking• Task-involving feedback focuses on knowledge,

skills and concepts necessary to be successful• Detailed guidance on what pupils are doing

well, what they need to improve and how to make the improvements

• Feedback should “scaffold” improvement• Opportunities must be built into lessons for

pupils to read feedback and respond• Feedback needs to be regular and rapid

AfL and Questioning: Research Evidence

• Teachers ask too many questions to be able to give serious thought to the quality of their questioning

• Pupils who are over-questioned tend to become passive and teacher-dependent

• The goal: fewer, better questions

Encouraging Thoughtful Behaviour• One study found an average wait time of

0.9 of a second before teachers re-worded a question or answered it

• Requiring an almost instant response only works well with knowledge/recall questions

• Some teachers are increasing their wait time, to allow students to formulate more thoughtful answers

Effects of increasing the wait time after a question:

• In one study the average wait time was about a second but: ‘where a longer silence was left – even as short as three seconds – the quality and extent of pupils’ responses improved dramatically… not only longer but also more thoughtful’ (Woods 1998:176).

• More pupils offer answers• The number of ‘I don’t know’ responses decreases• The number of hypothetical answers increases• The frequency of answers from less able students

increases• Students more likely to challenge and/or improve

each other’s answers

Bloom et al’s Taxonomy: A Hierarchy of Thinking Skills

Higher order

• Evaluation

• Synthesis

• Analysis

Lower order

• Application

• Comprehension

• Knowledge

Engaging more learners

‘No hands’ policy

• Pupils are trained not to raise their hands to answer a question

• Works well with increased thinking time/questions requiring more thought

• Everyone is expected to provide an answer if called upon

Other Strategies for Obtaining fuller feedback

• Response partners/groups• Pupil votes• Answers written on hand-held dry-wipe

boards• True/false cards• Traffic lights (green = fully understood,

amber = partial understanding, red = not understood)

• Thumbs (up = confident; horizontal = limited confidence; down = not confident)

Peer and Self-Assessment: Research Evidence• Portugese study• Pupils (N= 354) taught by teachers in the

experimental group were trained to use self-assessment on a regular, usually daily, basis in Mathematics

• Pupils in the control group (N=313) did not use self-assessment in Mathematics

• Pre- and post-tests of performance: children in the experimental group made almost double the progress of pupils in the control group

Peer and Self-Assessment: Issues• Self-assessment is “essential” rather than

a “luxury” (Black and Wiliam 1998)

• Some teachers – Confuse self-assessment with self-marking– worry that students cannot be trusted to mark

their own work accurately and honestly

• A difficult and demanding skill which can take a long time to master

Requirements• Assessment criteria shared with pupils in

an accessible language and format• Pupils must develop a ‘nose’ for standards

so that they can recognise how well work meets criteria (Exemplification material de-mystifies assessment providing concrete examples of what success looks like)

• Best completed during rather than at the end of an activity

Use of traffic light icons• Pupils use them to

– Indicate how well they have understood a topic (Green = completely understood; amber = partial understanding; red = not understood)

– To formulate a revision strategy ( e.g. pupils who have coded a topic green or amber can work together leaving the teacher free to work intensively with the ‘reds’)

Review of Aims

To identify

• the principles underpinning AfL and how these principles differentiate formative from summative assessment

• studies which have been instrumental in bringing about change

• key ways in which teachers are changing how they teach and assess

Sources of Information• Black, P. and Wiliam, D. (1998) Inside the Black

Box: Raising Standards through Classroom Assessment (London, King’s College)

• Black, P. et al. (2002) Working Inside the Black Box: Assessment for Learning in the Classroom (London, King’s College)

• Black, P., Harrison, C., Lee, C., Marshall, B. and Wiliam, D (2003) Assessment for Learning: Putting it into Practice (Open University Press)

• Centre for Educational Research and Innovation (2005) Formative Assessment: Improving Learning in Secondary Classrooms (Paris, OECD).

Useful websites• King’s College Assessment for Learning Group

www.kcl.ac.uk• Association for Achievement and Improvement

through Assessment www.aaia.org.uk• Suffolk Learning and Management Network

www.slamnet.org.uk• Qualifications and Curriculum Authority

www.qca.org.uk• Scotland: Assessment is for Learning

www.ltscotland.org.uk