Dylan Wiliam (@dylanwiliam)

Formative Assessment: The most effective way to build learning

www.dylanwiliamcenter.com

Outline: four questions

• Why we need to raise achievement

• Why we have to focus our efforts

• Why formative assessment should be the priority

• Why formative assessment requires new models of professional development

2

Why we need to raise achievement3

• In advanced economies, over the next 20 to 30 years

– Between a quarter and a third of jobs could be offshored (Blinder, 2011)

– About half of all jobs could be done by machines (Frey & Osborne, 2013)

• The choice for young people

– Wait for someone else to invent a new job for you

– Create your own

Change 1969-1999

+14%

+8%

–5%

–8%

–3%

The world of work is changing

Skill category

A Complex communication

B Expert thinking/problem solving

C Non-routine manual

D Routine cognitive

E Routine manual

Which kinds of skill are disappearing fastest from the workplace?

4

Autor, Levy and Murnane (2003)

What kinds of schools do we need?

School model Ethos Key process

Talent refineries School must provide opportunities for students to show what they can do

Ensuring good teaching and syllabus coverage

Talent incubators All students students can learn, but not all students can achieve at high levels

Drawing out what is within the student

Talent factories All students can achieve at high levels

“Whatever it takes”

Why every school should do pareto analysis6

• Vilfredo Pareto (1848-1923)

– Economist, philosopher, and sociologist,

associated with the 80:20 rule

• Pareto improvement

– A change that can make at least one person

(e.g., a student) better off without making

anyone else (e.g., a teacher) worse off.

• Pareto efficiency/Pareto optimality

– An allocation (e.g., of resources) is Pareto efficient or Pareto

optimal when there are no more Pareto improvements

Using research evidence

7

Relevant studies

• Fuchs & Fuchs (1986)

• Natriello (1987)

• Crooks (1988)

• Bangert-Drowns et al. (1991)

• Dempster (1991)

• Dempster (1992)

• Elshout-Mohr (1994)

• Kluger & DeNisi (1996)

• Black & Wiliam (1998)

• Nyquist (2003)

• Brookhart (2004)

• Allal & Lopez (2005)

• Köller (2005)

• Brookhart (2007)

• Wiliam (2007)

• Hattie & Timperley (2007)

• Shute (2008)

• Kingston & Nash (2011, 2015)

8

Formative Assessment: A contested term

Span

Length

Impact

Long-cycle Medium-cycle Short-cycle

Across terms, teaching units

Four weeks toone year

Monitoring, curriculum alignment

Within and between lessons

Minute-by-minute and day-by-day

Engagement, responsiveness

Within and between

teaching units

One to four weeks

Student-involved

assessment

9

Where the learner is going

Where the learneris now

How to get the learner there

Teacher

Peer

Student

Unpacking Formative Assessment

Clarifying, sharing, and

understanding learning

intentions

Eliciting evidence of learning

Providing feedback that

moves learners forward

Activating students as learningresources for one another

Activating students asowners of their own learning

10

Where the learner is going

Where the learneris now

How to get the learner there

Teacher

Peer

Student

Unpacking Formative Assessment11

Using evidence of achievement to adapt what

happens in classrooms to meet learner needs

An educational positioning system

• A good teacher:

– Establishes where the students are in their learning

– Identifies the learning destination

– Carefully plans a route

– Begins the learning journey

– Makes regular checks on progress on the way

– Makes adjustments to the course as conditions dictate

12

Educational Endowment Foundation toolkit13

Intervention Cost Quality of evidence

Extra months of learning

Feedback ££ +8

Metacognition and self-regulation ££ +8

Peer tutoring ££ +6

Early years intervention £££££ +6

One to one tuition ££££ +5

Homework (secondary) £ +5

Collaborative learning £ +5

Phonics £ +4

Small group tuition £££ +4

Behaviour interventions £££ +4

Digital technology ££££ +4

Social and emotional learning £ +4

Educational Endowment Foundation toolkit14

Intervention Cost Quality of evidence

Extra months of learning

Parental involvement £££ +3

Reducing class size £££££ +3

Summer schools £££ +3

Sports participation £££ +2

Arts participation ££ +2

Extended school time £££ +2

Individualized instruction £ +2

After school programmes ££££ +2

Learning styles £ +2

Mentoring £££ +1

Homework (primary) £ +1

Educational Endowment Foundation toolkit15

Intervention Cost Quality of evidence

Extra months of learning

Teaching assistants ££££ 0

Performance pay ££ 0

Aspiration interventions £££ 0

Block scheduling £ 0

School uniform £ 0

Physical environment ££ 0

Ability grouping £ -1

Educational Endowment Foundation (2015)

Where the learner is going

Where the learneris now

How to get the learner there

Teacher

Peer

Student

Unpacking Formative Assessment

Clarifying, sharing, and

understanding learning

intentions

Engineering effective discussions, tasks, and activities that elicit evidence of

learning

Providing feedbackthat moves learners

forward

Activating students asresources for one another

Activating students asowners of their own learning

16

Strategies and practical techniques for classroom formative assessment

17

Clarifying, sharing and understanding learning intentions

18

Share learning intentions

• Explain learning intentions at start of lesson/unit:– Learning intentions

– Success criteria

• Consider providing learning intentions and success criteria in students’ language

• Use posters of key words to talk about learning:– E.g., describe, explain, evaluate

• Use planning and writing frames judiciously

• Use annotated examples of different standards to “flesh out” assessment rubrics (e.g., lab reports).

• Provide opportunities for students to design their own tests.

19

Engineering effective discussions, activities, and classroom tasks that elicit evidence of learning

20

Eliciting evidence

• Key idea: questioning should

– cause thinking

– provide data that informs teaching

• Improving teacher questioning

– generating questions with colleagues

– low-order vs. high-order not closed vs. open

– appropriate wait-time

• Getting away from I-R-E (initiation-response-evaluation)

– basketball rather than serial table-tennis

– ‘No hands up’ (except to ask a question)

– ‘Hot Seat’ questioning

• All-student response systems

– ABCD cards, “show-me” boards, exit passes

21

Providing feedback that moves learners forward

22

Effects of feedback

• Kluger & DeNisi (1996) review of 3000 research reports

• Excluding those:

– without adequate controls

– with poor design

– with fewer than 10 participants

– where performance was not measured

– without details of effect sizes

• left 131 reports, 607 effect sizes, involving 12652 individuals

• On average, feedback increases achievement

– Effect sizes highly variable

– 38% (231 out of 607) of effect sizes were negative

23

Getting feedback right is hard

Response type Feedback indicates performance…

falls short of goal exceeds goal

Change behavior Increase effort Exert less effort

Change goal Reduce aspiration Increase aspiration

Abandon goal Decide goal is too hard Decide goal is too easy

Reject feedback Feedback is ignored Feedback is ignored

24

Provide feedback that moves learning on

• Key idea: feedback should:

– Cause thinking

– Provide guidance on how to improve

• Comment-only marking

• Focused marking

• Explicit reference to mark-schemes/rubrics

• Suggestions on how to improve:

– Not giving complete solutions

• Re-timing assessment:

– E.g., three-fourths-of-the-way-through-a-unit test

25

Activating students as learning resources for one another

26

Help students be learning resources

• Students assessing their peers’ work:

– “Pre-flight checklist”

– “Two stars and a wish”

– Choose-swap-choose

– Daily sign-in

• Training students to pose questions/identifying group weaknesses

• End-of-lesson students’ review

27

Activating students as owners of their own learning

28

Help students own their own learning

• Students assessing their own work:

– With rubrics

– With exemplars

• Self-assessment of understanding:

– Learning portfolio

– Traffic lights

– Red/green discs

– Coloured cups

– Plus/minus/interesting

29

So much for the easy bit

A model for teacher learning

• Content, then process

• Content (what we want teachers to change):

– Evidence

– Ideas (strategies and techniques)

• Process (how to go about change):

– Choice

– Flexibility

– Small steps

– Accountability

– Support

31

Supportive accountability

• What is needed from teachers:

– A commitment to:• The continual improvement of practice

• Focus on those things that make a difference to students

• What is needed from leaders:

– A commitment to engineer effective learning environments for teachers by:• Creating expectations for continually improving practice

• Keeping the focus on the things that make a difference to students

• Providing the time, space, dispensation, and support for innovation

• Supporting risk-taking

32

To find out more…

www.dylanwiliam.net

www.dylanwiliamcenter.com

33