great teachers change lives – but can we prove it?
TRANSCRIPT
Pearson Morning for English Teachers of Cambridge Exams
Great teachers change lives – but can we prove it?
Ian Wood
It’s all about outcomesWhere every journey starts
12
What do we value?
From the values exercise, Ian identified his core values as follows: •Making a meaningful difference
What difference do we make?
What difference does Pearson make?
“By 2018, we will report publicly on our impact on learner outcomes, in a rigorous and externally audited way, across Pearson’s whole business. We will be setting ourselves challenging targets and holding ourselves publicly to account for delivering on them.”
John Fallon, Pearson CEO
ef fi ca cy‧ ‧ ‧ [uncountable + of] formal the ability of something to produce the right result [= effectiveness]
How good is my English?And other awkward questions
26
How good is my English?
What do I need to learn?
Am I making progress?
How much longer?How many more $$
$s?
You’re intermediate.An intermediate book.Of course you’re learning!!One of our courses/books…
Qual
ity
Quantity
Two basic dimensionsHow many
different tasks?
How effective?How efficient?
Language development
Combining quantity and quality
A slippery slope
C2
C1
B2
B1
A2
A1
Below A1
The Council of Europe scale
Prof
icie
nt
Use
r C2
Can understand with ease virtually everything heard or read. Can express him/herself spontaneously, very fluently and precisely.
C1 Can use language flexibly and effectively for social, academic and professional purposes.
Inde
pend
ent
Use
r
B2 Can understand the main ideas of complex text on both concrete and abstract topics, including technical discussions in his/her field of specialisation.
B1 Can understand the main points of clear standard input on familiar matters regularly encountered in work, school, leisure, etc.
Bas
ic
Use
r A2 Can understand sentences and frequently used expressions related to areas of most immediate relevance.
A1 Can interact in a simple way provided the other person talks slowly and clearly and is prepared to help.
Asp
irin
g
Use
r
<A1 Can use and understand some isolated words. Communication is hardly possible.
Basic Characteristics of the CEF ‘Levels’
Need for formal instruction extremely rare
Learner becomes autonomous
Most learners require instruction and guidance
Continuous relevant input required to even start learning
CEF
C2
C1
B2
B1
A2
A1
• Disagreement on what it means to be “at a level”
• Levels are too wide to be of practical use
• All learners start below A1
• Limited set of descriptors
Disadvantages of the CEF
Start learning “Perfect”
Learner Descriptor
<A11
32
CEF boundaries
A1A2
B1
B2
C1C2
Back to the slippery slope
Increasing language proficiency and CEF descriptors
Measuring progressThe CEF – the original data
DIFFICULTY ORIGINAL CFR LABEL CEFR
3.90 - Mastery C2
2,80 3.90 Operational efficiency C1
1.74 2.80 Vantage Plus B2+
0.72 1.74 Vantage B2
-0.26 0.72 Threshold Plus B1+
-1.23 -0.26 Threshold B1
-2.21 -1.23 Waystage Plus A2+
-3.23 -2.21 Waystage A2
-4.29 -3.23 Breakthrough A1
-5.39 -4.29 Below A1 <A1
…ut you’ll get there later!
CEFR CEFR LABEL CEFR
C2 MasteryProficient User
C1 Operational efficiency
B2 VantageIndependent User
B1 Threshold
A2 WaystageBasic User
A1 Breakthrough
Common European Framework of Reference for Languages: Learning, Teaching and Assessment
, Cambridge University Press, 2001
Measuring progressThe CEF – the original data
DIFFICULTY ORIGINAL CFR LABEL CEFR
3.90 - Mastery C2
2,80 3.90 Operational efficiency C1
1.74 2.80 Vantage Plus B2+
0.72 1.74 Vantage B2
-0.26 0.72 Threshold Plus B1+
-1.23 -0.26 Threshold B1
-2.21 -1.23 Waystage Plus A2+
-3.23 -2.21 Waystage A2
-4.29 -3.23 Breakthrough A1
-5.39 -4.29 Below A1 <A1
…ut you’ll get there later!
CEFR CEFR LABEL CEFR
C2 MasteryProficient User
C1 Operational efficiency
B2 VantageIndependent User
B1 Threshold
A2 WaystageBasic User
A1 Breakthrough
So, how many levels?
Measuring progressThe CEFR – the original data
DIFFICULTY ORIGINAL CFR LABEL CEFR
3.90 - Mastery C2
2,80 3.90 Operational efficiency C1
1.74 2.80 Vantage Plus B2+
0.72 1.74 Vantage B2
-0.26 0.72 Threshold Plus B1+
-1.23 -0.26 Threshold B1
-2.21 -1.23 Waystage Plus A2+
-3.23 -2.21 Waystage A2
-4.29 -3.23 Breakthrough A1
-5.39 -4.29 Below A1 <A1
…ut you’ll get there later!
DIFFICULTY ORIGINAL CFR LABEL CEFR
3.90 - Mastery C2
2,80 3.90 Operational efficiency C1
1.74 2.80 Vantage Plus B2+
0.72 1.74 Vantage B2
-0.26 0.72 Threshold Plus B1+
-1.23 -0.26 Threshold B1
-2.21 -1.23 Waystage Plus A2+
-3.23 -2.21 Waystage A2
-4.29 -3.23 Breakthrough A1
-5.39 -4.29 Below A1 <A1
CEFR CEFR LABEL CEFR
C2 MasteryProficient User
C1 Operational efficiency
B2 Vantage
Independent User
B1 Threshold
A2 WaystageBasic User
A1 Breakthrough
So, how many levels?
Ten even, or six uneven ones – your choice – but either way
Measuring progressThe CEFR – the original data
…ut you’ll get there later!
So, how many levels?
Ten even, or six uneven ones – your choice – but either way
the ‘intermediate plateau’ = 4 whole levels of difficulty.
CEFR CEFR LABEL CEFR
C2 MasteryProficient User
C1 Operational efficiency
B2 Vantage
Independent User
B1 Threshold
A2 WaystageBasic User
A1 Breakthrough
Measuring progressThe CEFR – the original data
DIFFICULTY ORIGINAL CFR LABEL CEFR
3.90 - Mastery C2
2,80 3.90 Operational efficiency C1
1.74 2.80 Vantage Plus B2+
0.72 1.74 Vantage B2
-0.26 0.72 Threshold Plus B1+
-1.23 -0.26 Threshold B1
-2.21 -1.23 Waystage Plus A2+
-3.23 -2.21 Waystage A2
-4.29 -3.23 Breakthrough A1
-5.39 -4.29 Below A1 <A1
Glo
bal
Sca
le o
f Englis
h
Measuring progressThe Global Scale of English
30
43
59
76
85
90
22
13
10
C2
C1
B2+
B2
B1+
B1
A2+
A2
A1
T
Glo
bal
Sca
le o
f Englis
h
DIFFICULTY ORIGINAL CFR LABEL CEFR
3.90 - Mastery C2
2,80 3.90 Operational efficiency C1
1.74 2.80 Vantage Plus B2+
0.72 1.74 Vantage B2
-0.26 0.72 Threshold Plus B1+
-1.23 -0.26 Threshold B1
-2.21 -1.23 Waystage Plus A2+
-3.23 -2.21 Waystage A2
-4.29 -3.23 Breakthrough A1
-5.39 -4.29 Below A1 <A1
90
10
● Finer grained scale from 10 to 90● A psychometric definition of what
is means to be “at a level”● Starts below A1
CEF
C2
C1
B2
B1
A2
A1
A granular Global Scale of English
0%10%20%30%40%50%60%70%80%90%
100%
10 20 30 40 50 60 70 80 90
Likelihood Correct Performance
GSE Task Difficulty
A learner at 25 on GSE
0%10%20%30%40%50%60%70%80%90%
100%
10 20 30 40 50 60 70 80 90
Likelihood Correct Performance
GSE Task Difficulty
A learner at 61 on GSE
0%10%20%30%40%50%60%70%80%90%100%
10 20 30 40 50 60 70 80 90
Likelihood Correct Performance
GSE Task Difficulty
A learner at 80 on GSE
GSE90
10
B1 MATURA
GSE 59
PEARSON Courses
C2
C1
B2
B1
A2
A1
90
10
90
10
Progress
B2
43
3013 to go
GSE 43B1
PTEAcademic
59= 87 TOEFL= 6.5 IELTS
CEF
C2
C1
B2
B1
A2
A1
CEF
C2
C1
B2
B1
A2
A1
Map the inputsWhat do I need to learn?
327
‘The Sea’Proficient User
C1-C2
‘The Delta’Independent User
B1-B2+
‘The River’Basic User
A1-A2+
‘The Stream’‘Tourist’Below A1
How realistic are we really about the ‘width’ of the CEF levels and how many hours it takes to progress from one to the next?
Measuring outcomes with the CEF
Matching the old world and the new…
…ut you’ll get there later!
20th century levels
Beginner
Elementary
Pre-intermediate
Intermediate
Upper-intermediate
Advanced
21st century levels
A1
A2
B1
B2
C1
C2
Publishers traditionally used six level names and the CEFR reduced its nine equal levels into six – so a simple match, right?
…or the 6 into 5 problem…
…ut you’ll get there later!
20th century levels
Beginner
Elementary
Pre-intermediate
Intermediate
Upper-intermediate
Advanced
21st century levels
A1
A2
B1
B2
C1
Everyone agrees A1 = Beginner, A2 = elementary & C1 = Advanced, but…
three ‘intermediate’ levels don’t go into two CEFR levels.
…or the 6 into 5 problem…
…ut you’ll get there later!
In reality, very few courses cater for genuine beginners or genuine C1-C2 level learners. Consider:
Look at the following CEFR can-do statement. Which level is it?
Can use the language fluently, accurately and effectively on a wide range of general, academic, vocational or leisure topics, marking clearly the relationships between ideas. Can communicate spontaneously with good grammatical control without much sign of having to restrict what he/she wants to say, adopting a level of formality appropriate to the circumstances.
● Original CEF descriptors – with values
● 200+ new descriptors – rated by teachers
● Starts below A1
● Triple the descriptors
Available now at:
english.com/gse
Global Scale of English: Learning Objectives
GSE Learning Objectives: 43-50 Spoken Interaction
Global Scale of English: Learning Objectives
● General Adult (Available now)
● Academic (Late 2015)
● Professional (Late 2015)
● Young Learners (Late 2015)
Available at:
english.com/gse
Global Scale of English: Learning Objectives
More accurate book mapping
More accurate book mapping
Measure the outputsWhat have I learnt?
439
40
The measuring toolProgress test wish list
What would we want from a progress test?
• Accuracy
• Reliability
• Mapped to CEFR
• All four skills
• Grammar and vocabulary
• Quick results
• Convenience
• Affordability
Progress
Access / ExperienceThe measuring tool
Introducing Progress
42
online & adaptive
integrated skills & course agnostic: to measure real language ability rather than course coverage
ready-made & standardised: for time-saving and consistent testing
robustly linked to the Global Scale of English and the CEFR: to accurately measure small amounts of progress within a level
3 tests taken at start, middle and end of a course
personal score report: highlights areas for focus to inform teaching & learning
automatically and objectively scored: delivers rapid results
institutionally administered: can be conveniently taken at school or at home
The measuring toolIntroducing Progress
43
The measuring toolProgress item types
44
45
46
47
Gradebook /
GSE score
Individual
Skills
Intervention
The measuring toolFeedback
The measuring toolFeedback
Can generally identify the topic of discussion around them when conducted slowly and clearly
• Outcomes need to be clear, understood and relevant to the learner
• Outcomes are more likely to be improved if they can be articulated,
made meaningful and measured
• Expectations of progress need to be realistically achievable in the
time available
• Learners and teachers need regular feedback on progress to allow for
timely intervention
• Inputs and outputs need to be calibrated to the same scale
In summaryHow NOT to set ourselves up to fail
The Global scale of English
Thank you