improving education in turkey final print
TRANSCRIPT
CONFIDENTIAL AND PROPRIETARYAny use of this material without specific permission of McKinsey & Company is strictly prohibited
Improving the EducationSystem and its Outcomesin Turkey
September 6th 2011
1
Objectives and Context for today
Objectives for today
▪ Share various best practices from around the world on:– the role and daily conduct of employment agencies
(1st session) – how to improve the education system and
outcomes (2nd session)
▪ Discuss relevance and need for adaptation for Turkey
1
Basis of our exposure
▪ McKinsey is the leading management consulting Firm with 99 offices in 55 countries
▪ We have conducted public sector work in 60 different countries on education and employment policy, system design and institutional transformation
▪ Unique macroeconomic think thank (McKinsey Global Institute) and research capability
2
2
Discussion topics
Our perspective on improving the quality of schooling
Selected themes for Turkey
Questions to move forward
3
Despite high spending increases, education systems’ outcomes have stagnated over the past decades
1 Real expenditure, corrected for the Baumol effect using a price index of government goods and service2 Math and Science
SOURCE: UNESCO, EFA Global Monitoring Report 2005, Pritchett (2004), Woessmann (2002). McKinsey analysis
%
Increase in real expenditure per student1
(1970-1994)
Belgium
United Kingdom
Japan
Germany
Italy
France
New Zealand
Australia
223
270
212
126
108
103
77
65
-10
-2
-7
1
-5
2
-8
-5
Increase in student achievement2
(1970-1994)
4
“Variations in teacher quality completely dominate any effect of altered class size”
Popular reforms have not improved student outcomes: Class size
SOURCE: Hanushek The Evidence on Class Size, Akerhielm Does class size matter, McKinsey analysis
14
89
9
Significant negative effectof reduced class size
No significant effectof reduced class sizes
Significant positive effectof reduced class sizes
Of 112 studies that have examined the effects of class size on student achievement …
5SOURCE: McKinsey & Company
We identified 20 improved systems across the world from which to learn
Each of these systems has achieved an increase in student achievement that is
� Significant � Widespread �Sustained
Sustained improvers1. Aspire Public Schools, USA2. Boston/Mass, USA3. England4. Hong Kong5. Latvia6. Lithuania7. Long Beach, CA, USA8. Ontario, Canada9. Poland10. Saxony, Germany11. Singapore12. Slovenia13. South Korea
Promising starts14. Armenia15. Chile16. Ghana17. Jordan18. Madhya Pradesh, India19. Minas Gerais, Brazil20. Western Cape, South Africa
6
Our sample represents a continuum of improvement frompoor to fair to good to great
SOURCE: TIMSS, PISA, NAEP, national and provincial assessments; McKinsey & Company interventions database
Poor2
Fair2
Good2
Great
1985 1990 20001995 2005
Systems
Systems withSpecial Assumptions
2010
Ontario, Canada
Saxony, Germany
England
South Korea
Hong Kong
Latvia
Lithuania
Slovenia
Poland
Long Beach, CA, USA
Boston/MA, USA
Chile
Western Cape, SA
Singapore
Ghana
Aspire Public Schools (USA)
Madhya Pradesh, India
Minas Gerais, Brazil
Armenia
Jordan
7
Our research highlights nine lessons about school system improvement
SOURCE: McKinsey & Company
A system can make significant gains from wherever it starts – and these gains can be achieved in as short as six years1
Each stage of the school system improvement journey is associated with a unique set of interventions 2
While ‘structure’ and ‘resource’ interventions dominate the debate, ‘process’ deserves as much attention3
A system’s context might not determine what needs to be done, but it does determine how it is done 4
Leadership continuity is essential – the median tenure of new strategic leaders is six years and that of new political leaders is seven years 5
Prescribe adequacy and unleash greatness, with collaborative practice as the engine of continuous improvement 6
Six interventions occur equally at every performance stage for all systems but manifest differently7
Ignition occurs due to socio-economic crisis, a critical report about system performance, or the installation of a new political or strategic leader 8
The middle layer plays a crucial role in delivering and sustaining improvement – compliance, communication, collaboration, and insulation9
8
Lesson I
A system can make significant gains from wherever it starts –and these gains can be achieved in as short as six years
SOURCE: McKinsey & Company
9
Systems at all performance levels can improveoutcomes substantially in as short as six years
2006
2000
SOURCE: PISA, McKinsey & Company interventions database
PISA scores, average1; 2000–06
1 Average across math, science, and reading PISA scores2 One school-year-equivalent (SYE) corresponds to 38 points on the PISA scale
InitialPerformance
+75% SYE2
Poor
+65% SYE2
Fair
+75% SYE2
Good
+25% SYE2
Chile
412
440
Latvia
460
485
Saxony
497
525
Hong Kong
533542
Great
10
Singapore narrowed the achievement gap between its ethnic groups
65
70
75
80
85
90
95
100
Overall
Indian
Chinese
Malay
20060504030201009998979695949392919089881987
SOURCE: Singapore Ministry of Education
% of pupils who sat the Primary School Leaving Examand achieved eligibility for secondary school by ethnicity
11
Each stage of the school system improvement journey is associated with a unique set of interventions, from poor to fair to good to great to excellent
Lesson II
SOURCE: McKinsey & Company
12
Our research finds that a unique ‘intervention cluster’ exists for each improvement journey
SOURCE: McKinsey & Company
Fair to good Good to greatPoor to fairGreat to excellent
Journey
Shaping the professional
Improving through peers and innovation
Achieving the basics of literacy andnumeracy
Getting the foundations in place
Intervention cluster theme
Common across all journeys
Six interventions1] Revising curriculum and standards2] Reviewing remunerations
structure3] Building technical skills
4] Assessing student5] Utilizing student learning data 6] Revising policy or education
laws
13
Fair to Good journeys focus on getting the system foundations in place
SOURCE: McKinsey & Company
Theme
Data
Getting all schools to minimum quality standard
Intervention types
▪ Data transparency at school and system level▪ Improvement area identification
▪ School network structure ▪ System size and decision rights▪ System and school funding model
▪ School model/streaming▪ Language of instruction
Pedagogy
Systems included
Aspire (2002-2003)Boston (2003-2005)Chile (2006+)Hong Kong (1983-1988)Jordan (1999+)
LBUSD (2002-2005)Latvia (1995-2000)Lithuania (1995-2000)Poland (2000-2002)Singapore (1983-1987)Slovenia (1995-2005)
14
Good to great journeys emphasize shaping the professional
SOURCE: McKinsey & Company
Theme Intervention types
Raising calibre of entering teachers and principals
▪ Recruiting▪ Preparation and induction
Raising calibre of existing teachers and principals
▪ Professional development ▪ Coaching on practice▪ Career pathways
▪ Self-evaluation▪ Curriculum flexibility
School-based decision-making
Systems included
Aspire (2003+)Boston (2006+)England (1995+)Hong Kong (1989–99)Long Beach (2005+)Latvia (2001+)
Lithuania (2001+)Poland (2003+)Saxony (2000–05)Singapore (1988–98)Slovenia (2006+)South Korea (1983–98)
15
Math
Reading
Good to Great example: Boston improved student outcomes substantially between 1998 and 2004 with a focus on improving instruction
1998 2004 1998 2004
1 Massachusetts state assessment exam
SOURCE: Boston Public Schools, McKinsey analysis
% of Boston students meetings the target standard in Grade 10 MCAS1
43
25
7774
16
The engine of sustained and continuous improvement from ‘good’ performance onwards is collaborative professional practice
SOURCE: system interviews, McKinsey analysis
ExamplesCollaborative practice is school professionals working with each other to improve their practice
Hong Kong
Boston, MA, USA
Ontario, Canada
Aspire Public Schools, USA
���� Study groups, professional learning communities using research and data
���� Teachers visiting each other's class rooms
���� Teachers doing demonstration lessons together and joint-lesson-planning
���� Teachers mentoring and coaching each other and working with specialist coaches and principals on instructional practice
���� Schools, subject groups, or system developing a consensus model of good practice
���� Teacher and leaders reviewing student performance data together and jointly developing solutions
17
Capability building vs. formal accountability Share of professional development & training interventions relative to accountability interventions
The balance of capability-building and accountability system intervention shifts as systems improve their performance
SOURCE: McKinsey & Company interventions database
67
Fair to Good
54
Great to Excellent
32
78
Good to Great
100% =
50
51
55
Poor to Fair
26
Accountability
Professional development and training
18
Lesson III
While ‘structure’ and ‘resource’ interventions dominate the debate, ‘process’ deserves as much attention
SOURCE: McKinsey & Company
19
Process is the most prevalent intervention type relative to structure and resource
SOURCE: McKinsey & Company interventions database
Intervention type
Structures –organizational, financial, and instructional configuration/ shape of the system
Processes –practices, activities, rights and responsibilities in the system
Resources –Level and allocation of financial and human resources to fuel the system
Share of all interventions(Percent,100% = 573)
70
15
15
Content
Policy
Focus of process intervention
Total processreforms
Policy & strategy
Learning Model
Total deliveryinterventions
ProfessionalDevelopment
Management& leadership
Communications
Change Authorities& Responsibilities
AccountabilityDelivery
Share oftotal process interventions Percent, 100% = 400
100
26
12
8
11
13
15
72
15
20
Our research highlights nine lessons about school system improvement
SOURCE: McKinsey & Company
A system can make significant gains from wherever it starts – and these gains can be achieved in as short as six years1
Each stage of the school system improvement journey is associated with a unique set of interventions 2
While ‘structure’ and ‘resource’ interventions dominate the debate, ‘process’ deserves as much attention3
A system’s context might not determine what needs to be done, but it does determine how it is done 4
Leadership continuity is essential – the median tenure of new strategic leaders is six years and that of new political leaders is seven years 5
Prescribe adequacy and unleash greatness, with collaborative practice as the engine of continuous improvement 6
Six interventions occur equally at every performance stage for all systems but manifest differently7
Ignition occurs due to socio-economic crisis, a critical report about system performance, or the installation of a new political or strategic leader 8
The middle layer plays a crucial role in delivering and sustaining improvement – compliance, communication, collaboration, and insulation9
21
Discussion topics
Our perspective on improving the quality of schooling
Selected themes for Turkey
Questions to move forward
22
1 Systems were categorized across time as low, fair, good, or great based on their average performance across test instrument, subject, and age group in a particular year on the universal scale. See earlier page on the methodology of creating the universal scale.
2 No directly comparable assessment data to link these systems to international assessments exists, so special assumptions were made in placing them on the scale (see following pages)
The overall quality of Turkey’s school system has movedfrom poor to fair, and now aspires to move from ‘fair to good’
SOURCE: TIMSS, PISA, NAEP, national and provincial assessments; team analysis
Poor
Fair
Good
Great
1985 1990 20001995 2005
Ontario, Canada
Saxony, Germany
England
Slovenia
Poland
Latvia
Lithuania
Chile
Western Cape, SA
Singapore
Hong Kong
Korea
Long Beach, CA, USA
Boston/MA, USA
Jordan
Armenia
Ghana
2010
Systems1
Turkey
23
Across PISA-participating countries, Turkey has among the highest degrees of variance between schools and lowest degrees of variance within schools
SOURCE: OECD PISA
81
26
43
59
70
92
Lowest
1st Quartile
Median
3rd Quartile
Turkey
Highest
80
67
53
57
75
91
Lowest
Turkey
1st Quartile
Median
3rd Quartile
Highest
Standard deviation of student performance among 36 OECD countries2
Points1, 2009
1 PISA scores range is from 0 (min) to 600 (max)2 Based on readily available data. Specific percentile may vary with inclusion of all 66 participating countries but overall pattern should not
Between schools Within schools
24
There are significant regional differences in schooling across Turkey
SOURCE: MEB, McKinsey analysis
0
5
10
15
20
25
30
35
40
45
50
55
60
65
70
75
80
85
0 0.0005 0.0010 0.0015 0.0020 0.0025 0.0030 0.0035 0.0040 0.0045
Secondary education schooling ratePercent
Bilecik
IspartaKarabük
Bolu
RizeKırklareliEskişehir
Ankara
School to populationbetween 15-19 years old, ratio
Bitlis
Muş
Şırnak
Ağrı
Van
Mardin
BatmanDiyarbakır
Şanlıurfa
Turkeyaverage:58.5
Turkeyaverage:0.0014
2008
25
Improving education for all Turkish students may therefore require a differentiated strategy based on the different starting performance levels within Turkey
SOURCE: McKinsey & Company
Fair to good Good to greatPoor to fairGreat to excellent
Journey
Shaping the professional
Improving through peers and innovation
Achieving the basics of literacy andnumeracy
Getting the foundations in place
Intervention cluster theme
…with the core of Turkey going from fair-to-good…
Some parts of the system still moving from poor-to-fair…
…and some parts already moving from good-to-great
Turkey’s journey
26
We have seen systems employ differentiated strategies in response to the different needs within their systems
Ontario, Canada
Good-to-great –School improvement strategies focussed on building “collective capacity”
Poor-to-fair –An interventionist failing schools strategy
Western Cape, South Africa
Poor-to-fair –A guided literacy and numeracy strategy focused on the low performing schools
Good-to-great –More latitude gives high-performing schools to follow own methods/ program
Dominant strategy Secondary strategy
27
From our analysis and conversations on Turkish schooling,a few key themes stand out
Theme
I Cultivating great teaching and school leadership at scale
II Creating student pathways to success and employment
III Equipping Turkish students with English proficiency
IVImproving the role of “middle layer” – between schools and central governance
28
From our analysis and conversations on Turkish schooling,a few key themes stand out
Theme
I Cultivating great teaching and school leadership at scale
II Creating student pathways to success and employment
III Equipping Turkish students with English proficiency
IVImproving the role of “middle layer” – between schools and central governance
29
We know teaching quality is the most important determinant of student outcomes
Student performance100th percentile
50th percentile
Age8 11
Students with high-performing teachers1
Students with low-performing teachers2
90th percentile
37th percentile
53 percentile points difference
1 Among top 20% teachers2 Among bottom 20% teachers
SOURCE: Sanders and Rivers ‘Cumulative and residual effects on future student academic achievement’
Two students with same performance
I
30
We also know that effective school leadership is also critical to student achievement
40
45
50
55
60
65
70
75
40 45 50 55 60 65 70 75 80 85 90 95 100
Student achievement in school
Leadership in school
Outstanding principal
High-performing principal
Average principal
Replacing an ‘average’ principal with an outstanding principal in an ‘average’ school could increase student achieve-ment by over 20 percentile points
1 For leadership and student achievement, percentile implies the relative placement within the distribution SOURCE: A ‘meta-analysis’ of 69 studies of school leadership conducted between 1978 and 2001, involving an estimated
14,000 teachers and 1.4 million students, Marzano, Robert J., Timothy Waters, and Brian A. McNulty, 2005
Percentile1
Effective school leadership includes three roles
▪ Instructional focuses on improving current instructional practice
▪ Change focuses on altering the instructional practices themselves and their supporting systems
▪ Administrative focuses on maintaining an orderly environment, budgeting, and support functions
I
31
Support and develop
How can systems build system-wide excellence in teaching practice and school leadership?
SOURCE: McKinsey & Company
Prepare and place
Manage perfor-mance
3 4 5Attract and recruit
21
Define professional standards
Recruiting top talent into schools in leading systems
Coaching on curriculum in South Africa & Bahrain
Developing teachers through collaborative practice in Boston, Ontario, and Shanghai
Establishing the E5 model of effective instruction in Victoria, and linking it to standards
Preparation that is increasingly school-based and practical
3 4 521
Teacher performance management practices across the OECD
I
32
Several top performing systems attract high caliber candidates into the teaching profession
Teacher pool composition (share of cohort)
Country
▪ The top 5 percent▪ Korea
▪ The top 10 percent▪ Finland
▪ Singapore/Hong Kong
▪ The top 30 percent
Candidates undergo a rigorous selection process, have flexible career tracks, and receive a competitive starting salary relative to other professions (flatter salary structure with targeted increases in years 3 and 5 for cost-effectiveness)
SOURCE: How the World’ Best School Systems end up on Top, McKinsey & Company
1
33
Top-performing systems made it difficult to become a teacher: Finland
▪ Check for minimum qualifications:– Academically, applicants should be in the top 10% of
their age cohort– Applicants should have completed relevant school and
university education
▪ Check skills– Applicants must have a high level of literacy and
numeracy
▪ Check attitude, aptitude and personailty:– Conducted by a panel of experienced headmasters– May include practical tests or activities
▪ Check attitude, aptitude and personailty:– Teachers are monitored during their initial teacher
training for suitability as teachers– A small number of candidates who do not demonstrate
the required standards are removed from the course
Source: Interviews, McKinsey
Only 1 in 10 applicants is accepted to become a teacher
Interviews
Assessment tests
Monitoring at university
CV ScreenCV Screen
1
34
Making it difficult to enter the teaching profession can elevate it status and attractiveness to talented candidates
▪ 1 in 5 applicants accepted into the National Institute of Education’s Initial Teacher Training programs
▪ Attracts high caliber candidates into teaching (from the top 30% of any academic cohort)
Singapore
Selected international strategies to elevate teacher status
▪ Compulsory registration of teachers in Victoria by the Victorian Institute of Teaching (VIT), which stipulates standards teachers must fulfill
Australia
▪ 1 in 5 applicants acceptedTeach First
▪ All teachers must have Qualified Teacher Status (QTS) to work in the U.K.
U.K.
SOURCE: Team interviews; Teacher Development Agency; Victorian Institute of Teaching (VIT); Teach First
EXAMPLES
“After we started rejecting candidates from top universities like Oxford and Cambridge, the number of applications from these universities soared. Students started to think, ‘wow, if my friends got rejected from this program, it must be pretty good”
“Setting standards that teachers must meet is a powerful tool for conveying the quality of the profession to the general public”
“We wanted teaching to have the same status as other professions like doctors and lawyers. The first step to achieving this is ensuring that all teachers meet the same high standards as in these other professions”
1
35
England’s campaign to improve the attractiveness of teaching improved it from the 92nd most desirable job to the 1st over 4 years
Source: Training and Development Agency For SchoolsSource: Training and Development Agency For Schools
▪ Teaching careers moved from the 92nd position of “most desirable next job” for 25 - 35 year olds to first place over four years
▪ Teacher applications increased by 35% over three years
1
36SOURCE: Teach for All web site; interview; McKinsey & Company
The Teach for All Network is expanding and adapting the Teach for America model of recruiting top talent into teaching across the globe
In 2008 . . .
▪ In the U.S. 25,000 students applied to Teach for America, including 5–10% of Harvard, Yale and Princeton
▪ In the U.K., Teach First was ranked the 9th most prestigious employer
▪ Now the network is expanding and adapting globally
1
37
Top-performing systems offer good salaries, but not great salaries
Source: OECD Education at a glance, McKinsey analysis
OECD Finland
Starting salary
OECD Finland OECD Finland
Salary after 15 years Maximum salary
95
112 112
95
131
159
Primary teacher salary as a % of GDP per capita
1
38
Victoria has developed the E5 Instructional Model as a guide for good teaching practice
Engage
▪ Prompts inquiry▪ Structures inquiry▪ Maintains
session momentum
Explore
▪ Presents new content
▪ Develops language and literacy
▪ Strengthens connections
Explain
▪ Facilitates substantive conversation
▪ Cultivates higher order thinking
▪ Monitors progress
Elaborate
▪ Assesses performance against standards
▪ Facilitates student self assessment
Evaluate
▪ Develops shared norms
▪ Determines readiness for learning
▪ Establishes learning goals
▪ Develops meta-cognitive capacity
SOURCE: Victoria Department of Education and Early Childhood Development
▪ E5 is a model of the pattern of good instruction, including capabilities, performance indicators, quality criteria
▪ E5 provides a model for collaborative enquiry and professional development among teachers
2
39
…and defined standards for excellence in teaching practice and school leadership to underpin both development and performance management
SOURCE: Breakthrough Leadership: A Way Forward, Department of Education Victoria, Australia; “The Developmental Learning Framework for School Leaders,” 2007
Standards of professional practice for teachers, in three domains▪ Professional knowledge▪ Professional practice▪ Professional engagement
Developmental Learning Framework for School Leaders
3
Victoria, Australia has a clear framework of professional standards and competencies of effective teachers and school leaders
40
Teacher preparation is increasingly school-based and practical
SOURCE: OECD Report; UNESCO Report; New Zealand Teachers Council; Ofsted; Ministry of Education, Finland; Ministry of Education, Singapore, Teacher Development Agency; NIE
Pre-service practical component
NIE, Singapore▪ 40% of credit in PGDE is
practical▪ Enhanced Partnership
Model with schools and Ministry of Education
Stanford University, USA▪ ~40% of time for 12 months▪ Partnerships with Palo Alto
area schools to place trainees
University of Melbourne, Australia▪ 40% practical for 18 months▪ A network of 30 close partner
schools
National regulation, England▪ A minimum of two thirds
(66%) of teacher training is in practicum
▪ A network of partner schools for every university
“A significant practice componentis now seen as an essential element to teacher preparation”
– OECD
3
41
Several systems use induction periods to transition new teachers and principals into the profession
Teacher inductionprogramCountry
LengthYears
1 At discretion of schools2 Pilot
SOURCE: OECD (Teachers Matter, 2004); Wong et al What the World Can Teach Us About New Teacher Induction (2005)
New teachers usually receive mentors who provide 1-2.5 hours of in-class coaching per week (mentors work with 10-15 teaches at a time)
Induction period yields a retention rate of ~90% in the first 3 years
3
�Sweden 1
�Australia <1
1
�Canada (Quebec) 1-2
�Estonia 1
�2
Ireland 1
�Japan 1
�Korea 0.5
�New Zealand 2
�Shanghai (China) –
�U.K. 1
�Boston (US) 1-3
�Singapore 1
42SOURCE: Teacher Advancement Program website and press; SEDL; Dufour Learning by Doing: A Handbook for Building Professional Learning Communities (2006); California department of Education
Teachers work together to analyze student progress, develop and trial lessons, and share successful practices. This is sometimes facilitated by a Student Achievement Officer from the Literacy and Numeracy Secretariat
Ontario, Canada
All teachers are required to visit and observe at least eight lessons by a colleague each term
Shanghai, China
Teachers were timetabled to have common planning time. This time was used for analysis of assessment data, whereby teachers looked at results from assessments and using them to inform teaching plans. Session facilitated by the principal or a coach.
Boston, USA
“Teachers who attend more professional development, especially in a co-operative context, are more likely to be involved in co-operative teaching.” - TALIS, 2009
“Teachers who exchange ideas and information and co-ordinate their practices with other teachers also report more positive teacher-student relations at their schools” - TALIS, 2009
4 In-service training is increasingly collaborative, data-driven, and facilitated, all with a focus on classroom practice
43
60
70
65
75
55
02009-10
2006-07
2004-05
2002-03
2000-01
2004-05
2009-10
2006-07
72
70
68
66
64
62
60
58
56
54
2002-03
02000-01
Good to Great example: Ontario’s proficiency levels improved consistently and dramatically at both 3rd and 6th grade levels since 2002
SOURCE: Ontario Education Quality and Accountability Office; IELD Ontario Case Study Report 2007
Proportion of 3rd grade students proficient in reading, writing, and mathematics
Proportion of 6th grade students proficient in reading, writing, and mathematics
In 2003, a new Premier and education team entered office in Ontario and launched school system reforms
Education reforms Education reforms
MathematicsWritingReading
44
Intentional instruction
Teachers analyse student data to understand needs
Teachers work together to plan lessons and student support
Teachers build a shared range of effective instructional strategies
Assess impact
Share methods and impact
Analyze student data
Plan instruction
Teach
Teacher assesses impact on student learning
Teacher uses the planned lesson and strategies
“We’re are precise but not prescriptive…we are trying to cultivate intentional teaching, where a teacher can tell you why she is using a particular strategy for a particular student need”
– Student Achievement Officer
The core of collaborative practice is intentional instruction
SOURCE: Interviews
4
45
Developing collaborative practice requireddeliberate design and facilitation …
BASED ON BOSTON EXPERIENCE
4
We made the time for it We used real and meaningful data
We interjected expertiseWe taught teachers howto collaborate
▪ Boston scheduled Common Planning Time,
▪ Built it into the school schedule (who meets, how fits in day, etc)
▪ This took significant leadership time (principal, teacher leader, coach)
▪ It was someone’s role to prepare analysis beyond a single class (e.g. 4th grade math trends) – principal, analyst, teacher leader
▪ Helped teachers prioritise what data to use (e.g. student achievement data)
▪ Sought to create data-rich school environments (e.g. data in teacher lounges)
▪ It had to be facilitated and modelled by a capable principal, coach, teacher leader, or administrator…it was a new behaviour!
▪ Session agendas were linked to a school improvement objectives (e.g. improving math)
▪ Built root cause problem solving skills
▪ Followed up on strategies from one meeting to next
▪ Teachers came across issues for which solutions were not apparent
▪ Encourage teacher research if teachers have experience and skills to do it,
▪ Otherwise the administration, coach, or principal may need to source and interject expertise
“We found that in-school collaborative practice required greater support, not less”
Effective teacher collaboration
46
… and took time to embed as a new culture4 BASED ON BOSTON EXPERIENCE
The destination may be clear …
▪ Schools professional capable and in the habit of working together to analyse student data, and develop instructional strategies
▪ Teachers spending time in each others’ classrooms, and sharing strategies that work
▪ A shared sense of responsibility for the practice and performance in their school and school system
▪ A school culture that is focused on evidence of student learning
… but it takes time to get there
Getting started
▪ Teachers need to understand why this is important and how it will help
▪ Consider starting with enthusiasts (anxieties can create resistance from others until value is clear)
▪ Piloting new practice with a few schools will build comfort and allow the system to adapt to culture
Stay the course
▪ Sustain the structures of collaboration…we saw too often that when facilitation stopped. teachers revert to old behaviours
▪ Increase the opportunities for collaboration over time (in-class observation, sharing effective practice across schools, etc)
From structure to culture
▪ The goal is a culture of collaboration with minimal facilitation
▪ However, this takes time!
47
Creating meaningful career pathways is a powerful way tostrengthen professionalism and collaborative practice (1/2) Lithuanian example
1 Includes Junior teachers. Percentages as of 1995/19962 Lithuanian Teachers Qualification Institute. Nomination comes from principal and then approved by municipality first.
SOURCE: Interviews
Junior Teacher
Classroom Teacher(37%1)
Professional designations
Senior Teacher(50%)
▪ Coach other teachers in the school
Methodist(12%)
▪ Coach other teachers in the district
Expert(1%)
▪ Coach other teachers at national level
▪ Contribute to writing national curriculum
▪ +1 year▪ Principal’s
decision▪ +10% extra
salary increase
▪ +2 years▪ Principal &
municipality decision
▪ +10% extra salary increase
▪ +5 years▪ Municipality’s
decision▪ +10% extra
salary increase
▪ +7 years▪ National
Council2 makes decision
▪ +10% extra salary increase
Success is in the details. The impact of potential pathways can be undermined if they become a mechanism only for promotion and salary increase, or if the promotion decision loses credibility among professionals
4
48SOURCE: Interviews
1 These teachers maintain a “workshop” or office at their schools focusing on curriculum development, teacher mentoring and development
Backbone Teacher
Lead subject Specialist
District Level
Expert Panels are formed comprising experienced peers and professors to assess teachers
Professional Evaluation Committee sends evaluators to assess teachers
Increasing responsibility for curriculum development
▪ Separate teacher designations at district, municipal and national levels
▪ National level designations are linked to subject specialization at district and municipal levels– E.g. achieving Grade 1
Teacher designation is a pre-requisite for recognition as a district level Backbone Teacher
▪ Increasing recognition of status brings greater responsibility for mentoring and curriculum development
Evaluation every 3 years
Municipal or Provincial Level
New Graduates
Grade 2 Teacher
National Level
Champion Teacher
Evaluation every 5 years
Grade 1 Teacher
Senior Teacher
Master Teacher1Famous Teacher1
Top Talent
Prerequisite: Advanced teacher
Prerequisite: Grade 1 Teacher
SHANGHAI EXAMPLECreating meaningful career pathways is a powerful way tostrengthen professionalism and collaborative practice (2/2)
4
49
A surprising number of teachers across OECD systems do not receive regular appraisal or feedback
31
23
36
More than onceper year
Once per year
Never or Once every two or more years
49
13
38
22
13
65
SOURCE: Teaching and Learning International Survey (TALIS) first results, OECD, 2009,
1 TALIS Survey was conducted in 23 OECD and partner countries: [OECD] – Australia, Austria, Flemish Belgium, Denmark, Hungary, Iceland, Ireland, Italy, Korea, Mexico, Norway, Poland, Portugal, Slovak Republic, Spain, and Turkey; [Partner countries] – Brazil, Bulgaria, Estonia, Lithuania, Malaysia, Malta, Slovenia
From principalsFrom peers or senior teachers
From external individual or body
5
Teacher reported frequency of appraisal and/or feedbackPercent (TALIS average1)
50
However, appraisal and feedback usually increases teacher job satisfaction
Impact of appraisal and feedback on teachers’ job satisfaction
Percentage of teachers who reported the following change to job satisfaction following the appraisal and/or feedback they received in their school, TALIS1
7
41
52
Decreased
Did not change
Increased
SOURCE: Teaching and Learning International Survey (TALIS) first results, OECD, 2009,
1 TALIS Survey was conducted in 23 OECD and partner countries: [OECD] – Australia, Austria, Flemish Belgium, Denmark, Hungary, Iceland, Ireland, Italy, Korea, Mexico, Norway, Poland, Portugal, Slovak Republic, Spain, and Turkey; [Partner countries] – Brazil, Bulgaria, Estonia, Lithuania, Malaysia, Malta, Slovenia
5
Appraisal and feedback also had a positive impact on teachers performance, with ~40% of teachers reporting that it made significant differences in their teaching
51
Victoria and Singapore have annual appraisal cycles markedby three formal steps
SOURCE: NIE; Victoria Department of Education and Early Childhood Development; McKinsey & Company
Mid-cycle review to discuss the teacher’s progress and any needed support strategies
Performance plans for Leading teachers and Classroom teachers prepared with the Principal
Evaluation of the teacher’s performance against the standards, led by Principal, with decision on salary progression
Victoria
1. Self-evaluation and meeting with Reporting Officer (RO)
2. Mid-cycle review/ evaluation
3. Final evaluation
Formal annual evaluation cycle
Informal coaching and feedback expected throughout the year
1. Initial planning
Singapore Self-evaluation and meeting with Reporting Officer (RO) to establish targets, expected results, competencies and professional development needs
RO evaluates the performance and potential of the teacher, and a school panel ranks teacher
Performance coaching and mid-year evaluation (portfolio evaluations, summaries of relevant teacher-RO discussions, and evaluative narratives from teacher and RO)
5
52
From our analysis and conversations on Turkish schooling,a few key themes stand out
Theme
I Cultivating great teaching and school leadership at scale
II Creating student pathways to success and employment
III Equipping Turkish students with English proficiency
IVImproving the role of “middle layer” – between schools and central governance
53
0
10
20
30
40
50
60
70
Ca
na
da
Au
str
ia
Sp
ain
No
rwa
y
Un
ite
d K
ingd
om
Ne
w Z
ea
lan
d
Ja
pa
n
Tu
rke
y
Me
xico
Po
lan
d
Lu
xem
bou
rg
Cze
ch
Re
pu
blic
Ge
rma
ny
Hu
nga
ry
Fra
nce
Ne
the
rla
nds
Slo
vak R
ep
ub
lic
Be
lgiu
m
Fin
lan
d
De
nm
ark
Sw
ed
en
Gre
ece
Ice
lan
d
Un
ite
d S
tate
s
Ita
ly
Au
str
ia
Sw
itze
rla
nd
Po
rtu
ga
l
Ire
lan
d
Turkey has the highest proportion of out-of-school non-working 15-19 years olds among OECD countries
Percentage of 15-to-19-year-olds not in education (unemployed, not in the labor force, or working)
1 Umeployed refers to people not in education and not working, but activeiy seeking a jobNote: Missing bars refer to cells below reliability thresholds
SOURCE: OECD
Not in education and not in the labor force
Not in education and unemployed1
Not in education (total)
Major struggle against declared vision of becoming a top 10 economy in 2023 (translates ~8% annual growth rate, requires increase in labor pool and productivity)
2008
Working
Doing nothing!
Looking for a job
II
54
Turkey is not alone. Youth unemployment is a global challenge
1 In addition 152m young people are employed, but still living in poverty (i.e., living in households surviving on less than US$ 1.25 per capita per day)
SOURCE: ILO, Global Employment Trends 2011 (January 2011)
Unemployment rate for youth between the ages of 15 and 24, 2010
8.39.5
12.312.614.2
15.218.218.9
23.725.1
Middle East
North Africa
Non-EU Europe,
CIS
Developed countries
Latin America
SE Asia Global average
Sub-Saharan
Africa
South Asia
East Asia
78 million young people between 15 and 24 who are able to work and looking for a job, but can’t get one1
II
55SOURCE: CEOs on Strategy and Social Issues (McKinsey Quarterly, 2007); Feb 2007 McKinsey survey of 391 CEOs
whose companies participate in UN Global compact
Education for employment is not only a public concern. Global business leaders recognize education and talent as their #1 issue to address for the future
1 Respondents were allowed to select up to 3 issues
None of these
Other
HIV/AIDS and other public health issues (e.g., malaria, nutrition, tuberculosis)
Access to clean water, sanitation
Security of energy supply
Making globalization's benefits accessible to the poor (e.g., microfinance)
Climate change
Poor public governance (e.g., weak states, conflict zones, corruption)
Educational systems, talent constraints
Which of the following global environmental, social, and political issues are the most critical to address for the future success of your business? Percent of respondents citing as Top 3 concern1
2
7
8
12
35
36
38
44
50
Education is also vital to addressing many “non-education” issues on this list
II
56
Across Europe, jobs demanding ‘medium’ qualifications (secondary and post-secondary vocational) are highest
SOURCE: CEDEFOP, Skills Supply and Demand in Europe to 2020 (released 2010)
1 Low = below post-secondary (ISCED 0-2); medium = secondary or post-secondary vocational (ISCED 3,4); high = tertiary (ISCED 5, 6)2 Expansion and replacement demand
Total demand by qualification level Total job openings by qualification level, 2010-2020 2
100% = 80 million openings
Projected demand by qualification level 2, 2010-2020, EU-27 countriesMillions of jobs, Percent
44High
Medium48
Low
8
Low HighMed
2010
2020
66
114
46
82
118
34
Largest number expected in technical and associate professional occupations
II
57
The breaking points in education for employment (E4E) often lie in the relationships between youth, providers, and employers
Education providers
Youth
Employers
What institutions and degree programs can provide youth with skills relevant to the job market?
How can youth identify jobs/careerpaths that the market needs?
How can employers identify and recruitcapable graduates?
How can employers and education providers collaborateon programs to produce skilled candidates?
SOURCE: McKinsey & Company
II
When these relationships are ineffective, several issue arise:• Quantity or mix of graduates does not match employers needs• Relevance or quality or graduates’ are not appropriate for jobs• Unrealistic or unclear expectations among youth
58SOURCE: OECD, UNESCO
We see systems are increasing the mix of education for employment offerings to improve capacity and quality
Example trends in provision/mix
Upper
secondary
▪ Increasing upper secondary pathways – both formal (e.g., dual systems) and informal (e.g., career pathways)
▪ Overall, focus on flexibility of pathways and balancing general and technical content, to support lifelong learning
Post-secondary
▪ Creating new models that support E4E goals (e.g., polytechnics)
▪ Increasing recognition of “E4E” concerns in traditional university education (e.g., skills to support management, drive innovation in companies, alternate degree offerings)
Workforce readiness
▪ Often not part of formal education, but a focus on bridges to post-secondary education or the labor market
▪ Often short-term, practical skills (e.g., resume building, interviewing) or remediation of critical labor market skills (e.g., English) not obtained in formal education
II
59SOURCE: Team analysis
Private education providers and private employers are also introducing successful approached to addressing education for employment
Proton – Talent pipeline through education partnership
▪ Partnership with Malaysian Automotive Institute
▪ Work-base program (70% work, 30% theory; 18 – 24 months); 2,500 staff already trained
Saudi Japanese Automobile High Institute: Transnational public private partnership
▪ Initiative to create a pipeline of skilled local labourers in Saudi to work for Japanese OEMs
▪ Teaches technical and general skills, delivered by Saudi instructors and Japanese experts
Dr. Reddy’s foundation –Cost-effective training and placement for masses
▪ “Interest inventory” test to align candidates with courses
▪ Curriculum developed with private sector partners
▪ 2009: 46,000 students trained, 37,000 of them placed
SELECTED EXAMPLES
Tennessee Tech Centers –Tailoring job-training to local industries
▪ 27 independent centers tailor curriculums to local needs
▪ Curriculum developed with local industry leaders
▪ High graduation rates (85%)
II
60
From our analysis and conversations on Turkish schooling,a few key themes stand out
Theme
I Cultivating great teaching and school leadership at scale
II Creating flexible student pathways to success
III Equipping Turkish students with English proficiency
IVImproving the role of “middle layer” – between schools and central governance
61
1,000 native English teachers were recruited to enable to conversion to English instruction
Faced with too few proficient English teachers to meet its intention of rapidly expanding English instruction, one middle eastern country recruited international teachers
“The English skills of many of the teachers are not yet good enough to teach English successfully. They make mistakes in spoken and written English, which are copied and perpetuated by the students.”
The instructional intention: Deep fluency for all
The educator reality – Even among English teachers, only 13% of scored at a good level on the IELTS exam
IELTS exam results for English Teachers, % of participants, 2007
29%
58%
13%Good User: “Has operational command of the language…” (Level 7)
Competent User: “Has generally effective command of the language despite some inaccuracies...” (Level 6)
Modest User: “Has a partial command of the language…” (Level 5)
CLIENT EXAMPLE
III
Convert to English as a language of instruction for all core subjects in all schools
SOURCE: Confidential
62
In establishing an English language strategy there are at least 3 broad questions to answer
1 What is the scale of the English learning we intend and need? ▪ Depth – do students need deep fluency (e.g. all subjects in English)
or 2nd language proficiency (e.g. English as a subject)▪ Breadth – Is the need the same for all families / students?
2 What scale of English instruction can we deliver?▪ What is educators’ current English proficiency?▪ Where are English-able teachers allocated today (e.g. Secondary
vs. Primary levels)▪ Can we attract enough new English-proficient teachers to meet the
gap, either locally or internationally? Is that politically possible?▪ Would phasing-in English instruction – starting with primary school –
create the time to build up the number of capable teachers needed▪ How can we supplement the instruction our teachers provide to
support students’ learning (e.g. encouraging tutoring, through ICT)
3 At what grades and how intensively should we introduce English to match what we know about how children learn languages?
SOURCE: National Association for Bilingual Education; Team
III
63
From our analysis and conversations on Turkish schooling,a few key themes stand out
Theme
I Cultivating great teaching and school leadership at scale
II Creating student pathways to success and employment
III Equipping Turkish students with English proficiency
IVImproving the role of “middle layer” – between schools and central governance
64
System improvement requires integration and coordination across every level
SOURCE: McKinsey & Company
▪ Deliver classroom instruction▪ Collaborate with peers to develop, test and share pedagogical practices
that raise student outcomes▪ Engage parents as needed to advance student performance
▪ Define and drive school improvement strategy, consistent with direction from middle/center
▪ Provide instructional and administrative leadership for the school▪ Involve school community to achieve school improvement goals
▪ Provide targeted support to schools and monitor compliance▪ Facilitate communication between schools and the center▪ Encourage inter-school collaboration ▪ Buffer community resistance to change
▪ Set system strategy for improvement ▪ Create support and accountability mechanisms to achieve system goals▪ Establish decision rights across all system entities and levels▪ Build up skills and leadership capacity at all system levels
Teachers
Leaders
The ‘middle layer’
The centre
Sch
oo
ls
Role in system improvement
65
The middle layer comes in multiple forms and plays four importantroles in supporting system improvement
SOURCE: McKinsey & Company
School clusters
▪ School networks to coordinate resource allocation and facilitate learning across schools and up to the center (e.g. Singapore, Boston)
Geographic
▪ Almost all systems have a geographical middle to which they decentralise authorities around administration, finance, and pedagogy to middle (all systems)
Pedagogical
Subject-based
▪ A network of subject leaders to develop and standardize practice in that subject across the system. (e.g. Shanghai)
Level-based
▪ A middle layer organised by level schooling (e.g. primary, middle, and secondary) (e.g. Long Beach USD)
▪ Targeted support to school– Adapt reforms to local needs of schools
and communities, including – A channel for delivery, especially in
large systems– Examples: Korea, Chile, Ontario
▪ Two-way communication– Communicate from centre to schools &
from schools to centre. – Examples: Singapore, Boston
▪ Encourage collaboration between schools– Facilitate learning across schools and
up to the centre– Examples: Long Beach, Shanghai,
Western Cape▪ Buffer noise
– Reduce the volume of noise of resistance to changes
– Examples: Latvia, Poland
The middle layer comes in multiple forms
4 roles of the middle
66
By customizing support and intervention to schools’ needs, the middle layer’s can play a valuable role in a differentiated improvement strategy
Ontario, Canada
Western Cape, South Africa
South Korea▪ District offices offer training for teachers based on needs of
schools within district
▪ Implementation of reform often cascaded through provincial offices (e.g. KEDI’s ICT reforms in the 1990s)
▪ 2009 changes to Education Act increase School Boards’ responsibility for student achievement (in addition to administrative and financial responsibilities)
▪ Student Achievement Officers facilitate professional learning communities for Principals within School Boards
▪ Districts are disaggregated into cross-functional circuit teams that provide improvement support to schools (literacy advisers and curriculum coordinators, as well as administrative support)
▪ Districts/circuit identify locally-specific issues and develop locally-tailored solutions: (e.g. lobbying wine farmers association to allow farm workers leave to visit their children’s schools)
SOURCE: System interviews
Description
67
Discussion topics
Our perspective on improving the quality of schooling
Selected themes for Turkey
Questions to move forward
68
In four years we believe Turkey can make significant improvement in the quality of schooling
• Significant increases in Turkey’s student performance in reading and mathematicson PISA from bottom third among participating countries in 2009 to top half or top third in 2015, while also narrowing the achievement gap
• All new students entering secondary school with basic English proficiency
• A teaching profession that is in the Top 10 of most desirable professions for graduates in Turkey
• Teachers and principals who feel both supported by the education system and accountable for student outcomes; e.g.
− A credible instructional coach for every teacher− A school improvement partner for every principal− A consistent and meaningful cycle of performance reviews
• A track record of preparing and placing youth in jobs through vocational education partnerships with the private sector. Employers who express increased satisfactionwith the candidates they hire
• A public that recognizes Turkey’s achievements in education and expresses confidence in the system’s leadership
1 Based on 20-30 pt gain (50%-75% school year equivalent) on PISA
Students gains
Teacher and principal gains
Youth in jobs
Public confidence
ILLUSTRATIVE ONLY –TO SUPPORT DISCUSSION
69
An effective strategy would both (a) build public confidence through visible victories and (b) improve the fundamentals of sustained student gains
Build public confidence through visible victories that support student gains
Improve the fundamentals to achieve sustained student gains, from fair-to-good-to-great
Two interdependent tracks
EXAMPLES ONLY – NOT A COMPLETE STRATEGY
K-12 schooling Education for employment
• Deliver ‘announcables’ from plan:
− Expanded English instructionin primary schools
− Every child with a learning plan
− A new principals academy
− An extended school day
• Develop & implement a holistic school improvement strategythat delivers gains on 2015 PISA assessment
• Implement a campaign that increases the caliber of new teachers (Teaching as a Top 10 profession)
• Develop & implement a strategy to improve education for employment
−Private partnerships
−Flexible & well-supported pathways for students
−Improved information & matching for students
−Standards & quality assurance
−Financing mechanisms
• Implement a series of sector-based partnerships with employers that lead to jobs for youth
• Define standards for selected vocational programmes (e.g. starting with 10-20, then expand)
• Publically commit Turkey to specific goals for education
• Communicate a compelling strategy for improvement
and
• Same
• Same
70
Discussion questions
• Priorities: What are your current top priorities for Turkey’s education system, and what would be a set of compelling goals for education 4 years from now?
• Delivering on your goals:
− Is there a clear, holistic, and convincing strategy in place for improvement?
− Is there the capacity to drive and support implementation (especially at a local level)?
− Do you have the support of the public and professionals in the system to undertake change, so does that support need to be cultivated?
− What has been effective in achieving the gains that Turkey has made to date, and what can be taken and extended from those?
• Differentiating the approach:
− How can Turkey best segment its schools’ or regions’ to appropriately customise improvement support to the needs of those schools?
− Would a sequenced approach to concentrate efforts and demonstrate success be viable (either by region or starting with a subset of low and high performing local areas)?
• Education for Employment: What vocations or sectors would be appropriate to start with in developing and piloting programmes in partnership with the private sector to accelerate education for employment?
71
Appendix
72SOURCE: McKinsey & Company interventions database
Poor to fair Fair to good Good to great
Intervention
cluster1
▪ Raising calibre of
entering teachers and
principals– Recruiting programs– Pre-service training– Certification
requirements
▪ Raising calibre of
existing teachers and
principals– In-service training
programs– Coaching on practice– Career tracks– Teacher and
community forums
▪ School-based decision-
making– Self-evaluation– Independent and
specialized schools
▪ Cultivating peer-led learning
for teachers and principals– Collaborative practice– Decentralizing pedagogical
rights to schools & teachers– Rotation and secondment
programs
▪ Creating additional support
mechanisms for professionals– Release professionals from
admin burden by providing additional administrative staff
▪ System-sponsored
experimentation/innovation
across schools– Providing additional funding
for innovation– Sharing innovation from front-
line to all schools
▪ Providing motivation and
scaffolding for low skill
teachers– Scripted teaching materials– Coaching on curriculum– Instructional time on task– School visits by center– Incentives for high
performance
▪ Getting all schools to a
minimum quality level– Outcome targets– Additional support for low
performing schools– School infrastructure
improvement– Provision of textbooks
▪ Getting students in seats– Expand school seats– Fulfil students’ basic needs
to raise attendance
▪ Data and accountability
foundation– Transparency to schools
and/or public on school performance
– School inspections and inspections institutions
▪ Financial and organizational
foundation– Optimization of school and
teacher volumes– Decentralizing financial and
administrative rights– Increasing funding – Funding allocation model– Organizational redesign
▪ Pedagogical foundation – School model/ streaming– Language of instruction
Shaping the professional Improving through peers
and innovation
Achieving the basics of
literacy and numeracy
Getting the foundations in
place
Theme
Great to excellentImprovementjourney
Common
across all
journeys
Six interventions: [1] Revising curriculum and standards; [2] Reviewing reward and remunerations structure; [3] Building technical skills
of teachers and principals, often through group or cascaded training; [4] Assessing student learning; [5] Utilizing student data to guide
delivery, and [ 6] Establishing policy documents and education laws
1 Total number of interventions in each phase: poor to fair, n=103, fair to good, n=226; good to great, n=150; great to excellent, n=94
A unique ‘intervention cluster’ exists for each improvement journey, with six interventions common across all journeys
73
Systems with similar spend have widely ranging levels of performance
SOURCE: World Bank EdStats; IMF; UNESCO; PISA, TIMSS, PIRLS, McKinsey & Company
Public spend per student, PPP USD
580
560
540
520
500
480
460
440
420
400
380
0
464
8,000–9,000
498
522
489
9,000–10,000
470
10,000+
561
7,000–8,000
488
541
6,000–7,000
485
520
5,000–6,000
402
533
4,000–5,000
421
525
3000–4,000
456
508
2,000–3,000
383
498
1,000–2,000
361
478
0–1,000
380
471
Ghana
W. Cape
Brazil
Morocco
Azerbaijan
El Salvador
Algeria
Uruguay
Argentina
Botswana
Saudi Arabia
Kuwait
Jordan
Armenia
SyriaPhilippines
Turkey
Moldova
OmanIran
Bulgaria Malta
Bahrain
GreeceNew Zealand
IsraelCyprus
Norway
LatviaPoland
Lithuania
England
SloveniaGermany4 USA
Slovak Republic
CroatiaCzech
Republic
Hungary
PortugalSpain
France
Italy
NetherlandsIceland
SwedenAustria Luxembourg
Denmark
MalaysiaRomania
ColumbiaMexico
GeorgiaChile
Tunisia
South Korea
Japan Ireland Belgium
Singapore
Hong Kong
Ontario
EstoniaSwitzerland
Finland
Australia
Poor
Fair
Good
Great
Excellent
Universal scale score (maximum, median, minimum), in PISA 2000 units
74
Turkey’s student performance on PISA reflects a wide spread of proficiency
Below Level 1 18%
Level 1 25%
Level 2 25%
Level 3 17%
Level 4 10%
Level 5 4%
Level 6 1%Level 6: Can conceptualise, generalise, and utilise information [in] complex problem situtations… advanced reasoning…can formulate and communicate actions and reflections
High
Low
Level 3: Executes clearly defined procedures [and] develop short communications…
Level 1: Can answer questions involving familiar contexts…carry out routine problems…perform actions that are obvious
Turkish students’ performance by proficiency level on 2009 PISA Mathematics
SOURCE: OECD PISA
75
There are also comparatively large differences acrosssocio-economics groups in Turkey
SOURCE: OECD PISA 2003, “First results from PISA 2003”
▪ Low performance
▪ Large differences by impact of socio-economic status
▪ High performance
▪ Large differences by impact of socio-economic status
▪ Low performance
▪ Low differences by impact of socio-economic status
▪ High performance
▪ Low differences by impact of socio-economic status
PISA 2003 Performance in mathematics and the impact of socio-economic background