international comparisons of science and mathematics education: pisa and timss harrie eijkelhof...

International comparisons of science and mathematics education: PISA and TIMSS

Harrie EijkelhofFreudenthal Institute for Science and Mathematics Education

August 2014 Summerschool

Outline of this session

• Why comparing countries?• PISA

• Age 15• Mathematics and Science

• TIMSS• Grades 4 & 8• Mathematics and Science

• Implications of comparing countries• Learning from other countries

2


Why comparing countries in science and mathematics education?

• Economic importance of science and mathematics education

• Internationalisation: mobility of students• Tool for educational policy: monitoring, reform• Learning from other systems: relative strengths

and weaknesses

3


Programme for International Student Assessment (PISA)

• Created in 1997 by OECD• Aim: to monitor outcomes of educational

systems in terms of student achievement• Age: 15 years• Surveys every 3 years since 2000 in reading,

mathematical and scientific literacy• 2000 reading, math, science [43]• 2003 reading, math, science [41]• 2006 reading, math, science [57]• 2009 reading, math, science [65]• 2012 reading, math, science [65]• 2015 reading, math, science [?? ]http://www.youtube.com/watch?v=q1I9tuScLUA

4


Focus of PISA

• Collaborative effort: OECD + non-member partner economies

• To measure how well students at age 15 are prepared to meet challenges in future life

• Beyond school-based approach: use of knowledge in everyday tasks and challenges

• Based on model of lifelong learning• Not only assessing knowledge: also how to

apply knowledge in real-life issues

5


PISA-definition of Mathematical literacy

An individual’s capacity:

• to identify and understand the role that mathematics plays in the world

• to make well-founded judgements

• to use and engage with mathematics in ways that meet the needs of that individual’s life as a constructive, concerned and reflective citizen.

6


PISA-definition of Scientific literacy

• The ability to engage with science-related issues, and with the ideas of science as a reflective citizen

• This requires the competencies to:

Explain phenomena scientifically

Evaluate and design scientific enquiry

Interpret data and evidence scientifically

7


PISA 2012 Mathematics Framework

8


PISA 2015 Science Framework

9

Summerschool

Contexts for the PISA 2015 science assessment

Personal Local/National Global

Health Maintenance of health, accidents, nutrition

Control of disease, social transmission, food choices, community health

Epidemics, spread of infectious diseases

Natural Resources Personal consumption of materials and energy

Maintenance of human populations, quality of life, security, production and distribution of food, energy supply

Renewable and non-renewable natural systems, population growth, sustainable use of species

Environmental Quality Environmentally friendly actions, use and disposal of materials and devices

Population distribution, disposal of waste, environmental impact

Biodiversity, ecological sustainability, control of pollution, production and loss of soil/biomass

Hazards Natural and human-induced decisions about housing

Rapid changes [earthquakes, severe weather], slow and progressive changes [coastal erosion, sedimentation], risk assessment

Climate change, impact of modern warfare

Frontiers of Science and Technology Scientific aspects of hobbies, personal technology, music and sporting activities

New materials, devices and processes, genetic modifications, health technology, transport

Extinction of species, exploration of space, origin and structure of the Universe

August 2014 10


Scientific knowledge

Of science:• Physical systems• Living systems• Earth and space systemsAbout science:• Procedural knowledge: variables, measurement,

replicability, representing data, control of variables

• Epistemic knowledge: purposes, nature, values, justification, errors, collaboration and critique

11

Attitudes in science (PISA 2015)

• Interest in science:e.g. curiosity, willingness to acquire knowledge, career

aspirations

• Valuing scientific approaches to enquiry:e.g. commitment to evidence, criticism

• Environmental awareness:e.g. concern, disposition to sustainable behaviour

August 2014 Summerschool 12


Sample PISA-items (released)

• Study sample questions• 3 mathematics• 3 science

• Look at • the question type • level of difficulty

• Give your opinion on the nature and level of the questions: feasible in your country?

13

PISA-questions MATHEMATICS EXAMPLES

Question 1: SAVINGS ACCOUNT

1 000 zed is put into a savings account at a bank. There are two choices: one can get an annual rate of 4% OR one can get an immediate 10 zed bonus from the bank, and a 3% annual rate. •Which option is better after one year? •After two years?



Question 2: SCIENCE TESTS

In Mei Lin’s school, her science teacher gives tests that are marked out of 100. Mei Lin has an average of 60 marks on her first four Science tests. On the fifth test she got 80 marks.What is the average of Mei Lin’s marks in Science after all five tests?

Average: ...............................................

15


Question 3: BOOKSHELVES To complete one set of bookshelves a carpenter needs the following components:4 long wooden panels,6 short wooden panels,12 small clips,2 large clips and14 screws.The carpenter has in stock 26 long wooden panels, 33 short wooden panels, 200 small clips, 20 large clips and 510 screws.How many sets of bookshelves can the carpenter make?

Answer: .................................................

16

PISA-questions: Science example (1)

TOBACCO SMOKINGTobacco is smoked in cigarettes, cigars and pipes. Research shows

that tobacco related diseases kill nearly 13 500 people worldwide every day.

It is predicted that, by 2020, tobacco-related diseases will cause 12% of all deaths globally.

Tobacco smoke contains many harmful substances. The most damaging substances are tar, nicotine and carbon monoxide.

Question 1: TOBACCO SMOKING Tobacco smoke is inhaled into the lungs. Tar from the smoke is

deposited in the lungs and this prevents the lungs from working properly.

Which one of the following is a function of the lungs?A To pump oxygenated blood to all parts of your bodyB To transfer some of the oxygen that you breathe to your bloodC To purify your blood by reducing the carbon dioxide content to

zeroD To convert carbon dioxide molecules into oxygen molecules

SummerschoolAugust 2014 17


Question 2: TOBACCO SMOKINGTobacco smoking increases the risk of getting lung cancer

and some other diseases.Is the risk of getting the following diseases increased by

tobacco smoking? Circle “Yes” or “No” in each case.

Is the risk of contracting this disease increased by smoking?

Bronchitis Yes / NoHIV/AIDS Yes / NoChicken pox Yes / No



Question 3: TOBACCO SMOKING Some people use nicotine patches to help them to give up smoking.

The patches are put on skin and release nicotine into the blood. This helps to relieve cravings and withdrawal symptoms when people have stopped smoking.

To study the effectiveness of nicotine patches, a group of 100 smokers who want to give up smoking is chosen randomly. The group is to be studied for six months. The effectiveness of the nicotine patches is to be measured by finding out how many people in the group have not resumed smoking by the end of the study.

Which one of the following is the best experimental design?A All the people in the group wear the patches.B All wear patches except one person who tries to give up

smoking without them.C People choose whether or not they will use patches to help give

up smoking.D Half are randomly chosen to use patches and the other half do

not use them.



Pisa tests in schools

• Selection of schools• Schoolcoordinator appointed• Random selection of 35 15year olds per school• Permission from parents• Tests session organized by test administrator• 13 different booklets with sets of questions• 2 hours for knowledge testing of students• 30 minutes for student questionnaire: personal

background, learning habits, attitudes, motivation• School questionnaire (headmaster): demographic

characteristics, quality of the learning environment

20

Proficiency levels mathematics PISA 2012(500 = OECD average)

OECD %Level 6 > 669 3.3Level 5 607 - 669 9.3Level 4 545 – 607 18.2Level 3 482 – 545 23.7Level 2 420 – 482 22.5Level 1 358 – 420 15.0Below 1 < 358 8.0



PISA 2012 – cognitive performance

Czech

Indo Korea Mexico NL Shang. Spain Turk. USA

Sci 508 382 538 415 522 580 496 463 497

Math 499 375 554 413 523 613 484 448 481

Read 493 396 536 424 511 570 488 475 498

22

Motivation to learn mathematics(PISA 2012)

Czech

Indo Korea Mexico NL Shang. Spain Turk. USA

I enjoy reading about M

17.5 76.9 27.2 62.0 12.1 50.1 19.3 56.2 33.8

I look forward to M lessons

33.9 72.3 21.8 70.6 19.8 54.4 25.7 48.9 45.4

I do M because I enjoy it

30.3 78.3 30.7 52.8 32.4 49.3 37.0 52.7 36.6



How to explain PISA-results?

• Which results are striking to you?

• Give at least five reasons which might explain differences between countries in results:1. 2. 3. 4. 5.

24


How to explain PISA-results?

• Cultural factors• Value of knowledge in society• Value of arguing• Science in the media (TV, journals, newspapers)

• Educational factors • Organization of education• Nature of the school curricula• Quality of teaching• Quality of assessment• Investments in education

• PISA-factors• Attitude towards PISA-test by 15-year olds and society• Nature of the PISA-questions

25

Objections raised against PISA

• Escalation of standardized testing• Leads to short-term policies• Narrowing to measurable educational objectives• Bias to economic role of schooling• Alliance with for-profit companies• Decreases autonomy of teachers



PISA-sources

• http://www.pisa.oecd.org/

• http://www.oecd.org/pisa/pisaproducts/

27


Trends in International Mathematics and Science Study (TIMSS)

• Started in 1995 by IEA (International Association for the Evaluation of Educational Achievement)

• Aim: to measure math and science knowledge and skills aligned with curricula in participating countries

• Age: grades 4, 8, 12• Surveys every 4 years in many countries:

Grade 4 Grade 8Grade 12

• 1995 26 41 5• 1999 -- 38 --• 2003 25 46 --• 2007 36 48 10 (2008)

• 2011 50 42 --http://timss.bc.edu/timss2011/index.html

28


TIMSS Frameworks and tests

• Mathematics • Grade 4

Content domains: number, geometric shapes and measures, data display

• Grade 8 Content domains: number, algebra, geometry, data and chance

• Science• Grade 4

Content domains: life science, physical science, earth science

• Grade 8 Content domains: biology, chemistry, physics, earth science

• Mathematics + Science Cognitive domains: knowing, applying, reasoning

29

Content and Cognitive Domains Grade 4

Science

• Content Domains 45% Life Science35% Physical Science20% Earth Science

• Cognitive Domains 40% Knowing40% Applying20% Reasoning

Mathematics

• Content Domains 50% Number35% Geometric Shapes and

Measures15% Data Display

• Cognitive Domains 40% Knowing40% Applying20% Reasoning


Sample of released math items TIMSS grade 4

1. In a car park, 762 cars were parked in 6 equal rows. How many cars were in each row?

2. Subtract 5.3 – 3.8

3. The temperature at 7 a.m. one morning was 12º. It increased by 2ºC every hour until it reached 20ºC at 11 a.m. What was the temperature at 9 a.m.?a. 14ºC b. 15ºC c. 16ºC d. 17ºC

4. Last year there were 92 boys and 83 girls in Fairmont School. This year there are 210 students, and 97 boys. How many more girls are there this year than last year? Show your work.


Sample of released math items TIMSS grade 8


Sample of released science items TIMSS grade 4

• What do birds, bats and butterflies have in common?a. feathersb. hairc. internal skeletond. wings

• Write down one form of energy Earth receives from the sun

• Which of these soil changes is due only to natural causes:a. loss of minerals due to farmingb. deserts forming due to tree cuttingc. flooding due to dam constructiond. minerals washing out due to heavy rain


Released science item TIMSS grade 4


Sample of released science items TIMSS grade 8


Released science item TIMSS Grade 8


TIMSS Results 2011Grades 4 & 8 Math & Science

Grade 4 (N=50)Math Science

Singapore 606 (1) 583 (2)Korea 605 (2) 587 (1)Hong Kong 602 ( 3) 535 (9)Chinese Taipei 591 (4) 552 (6)Japan 585 (5) 559 (4)Belgium (Flemish) 549 (7) 509 (27)Finland 545 (8) 570 (3)England 542 (9) 529 (15)Russian Feder. 542 (10) 552 (5)USA 541 (11) 544 (7)Netherlands 540 (12) 531 (14)Germany 528 (16) 528 (17)Norway 495 (29) 494 (33)Tunisia 359 (47) 346 (48)Morocco 335 (49) 264 (49)Yemen 248 (50) 209 (50)No significant difference with NL

Grade 8 (N=42)Math Science

Korea 613 (1) 587(3)Singapore 611 (2) 590 (1)Chinese Taipei 609 (3) 564 (2)Japan 570 (5) 558 (4)Russian Feder. 539 (6) 542 (7)Finland 514 (8) 552 (5)USA 509 (9) 525 (10)England 507 (10) 533 (9)Norway 475 (20) 494 (19)Tunisia 425 (32) 439 (29)Qatar 410 (33) 419 (37)Indonesia 386 (38) 406 (40)Morocco 371 (40) 376 (41)Ghana 331 (42) 306 (42)

Botswana (G9) 397 404South Africa (G9) 352 332


Trends Between 1995 and 2011 TIMSS Grade 4 Mathematics

Countries Improving• Australia• England• Hong Kong SAR• Iran• Japan• Korea• New Zealand• Norway• Portugal• Singapore• Slovenia• United States

Countries Declining• Austria• Czech Republic• Netherlands


Trends Between 1995 and 2011 TIMSS Grade 4 Science

Countries Improving

• Hong Kong SAR• Hungary• Iran• Japan• Korea• Portugal• Singapore• Slovenia

Countries Declining

• Norway


Progress in USA at local levels

Minnesota:http://www.youtube.com/watch?v=r4Xbej4Ncow

Michigan:http://www.youtube.com/watch?v=AtGlPOp-96M&playnext=1&list=PLF81F896453A0BC40&feature=results_video


Policy Implications of TIMSS & PISA

• Assessment of problems/successes• Looking abroad for solutions• Focus on assessment• Developing new curricula• Policy shifts, e.g.:

• focus on excellence• investing in backward regions or populations

• Acting on comparisons at system level


The Learning CurveLessons in country performance in

educationPearson,2012

http://thelearningcurve.pearson.com/videos http://thelearningcurve.pearson.com

•40 countries compared•Index cognitive skills & educational attainment•Top-countries Finland and South Korea•NL in subtop (7)

•www.thelearningcurve.pearson.com


School responsibility & autonomy compared


Country profiles


SouthKorea

Finland Nether-lands

Indonesia

TIMSS Grade 4 561 585 529 392PISA Grade 8 541 543 519 385HE-graduation 63% 49% 42% 13%% education in GNP 16% 12% 12% 17%Teachers’s salaries compared to mean

1,6 0,9 1,3 0,8

44

Global index of cognitive skills and educational attainment


Comparing educational systems

South Korea

• exam driven• rigid curriculum• reproduction oriented• frontal teaching• private crammer schools

(hagwans)• large classes• rel. high salaries teachers• importance of teacher

training• learning seen as moral

duty to family and society

Finland

• late start in schooling• less lessons• no homework• only some frontal teaching• focus on applying

knowledge• small classes• rel. low salaries teachers• importance of teacher

training• education seen as act of

social justice



Strong Performers and Successful Reformers in Education

A video series profiling policies and practices of education systems that demonstrate high or improving performance in the PISA tests.

Belgium (Flanders), Brazil, Canada (Ontario), Finland, Germany, Japan, Korea, Netherlands, Poland, Portugal, Shanghai, Singapore

http://www.pearsonfoundation.org/oecd/

47

Strong Performers and Successful Reformers in Education: Lessons from PISA for the United States

• Chapter 1 Introduction• Chapter 2 Viewing Education in the United States Through the Prism of PISA• Chapter 3 Ontario, Canada: Reform to Support High Achievement in a

Diverse Context• Chapter 4 Shanghai and Hong Kong: Two Distinct Examples of Education

Reform in China• Chapter 5 Finland: Slow and Steady Reform for Consistently High Results• Chapter 6 Japan: A story of Sustained Excellence• Chapter 7 Singapore: Rapid Improvement Followed by Strong Performance• Chapter 8 Brazil: Encouraging Lessons from a Large Federal System• Chapter 9 Germany: Once Weak International Standing Prompts Strong

Nationwide Reforms for Rapid Improvement• Chapter 10 Vignettes on education reforms: England and Poland• Chapter 11 Lessons for the United States

http://www.oecd.org/pisa/46623978.pdf


international comparisons of science and mathematics education: pisa and timss harrie eijkelhof...

Documents

mathematics education

summerschool pisadefinition

science framework

pisa age

ideas of science

science implications

mathematics framework

summerschool contexts