22 april 2009 benefits and limitations of international comparative research in science and...
TRANSCRIPT
![Page 1: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/1.jpg)
22 April 2009
Benefits and limitations of international
comparative research in science and
mathematics education
AMSTEL – SeminarPauline Vos
Universiteit van Amsterdam
AMSTEL Instituut
![Page 2: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/2.jpg)
2
International comparison Use identical instruments in different countries and look at differences between countries.
My own international experience: Went to school in NL, D and F worked in NL, Zimbabwe, Mozambique &
South Africa Worked at the Dutch research centre for
TIMSS-1999 (University of Twente) PhD on mathematics curriculum in NL using
TIMSS-instruments
![Page 3: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/3.jpg)
3
Many variations
Bilateral studies
Affect in science: ROSE (Relevance of Science Education), 40 countries
Teacher competencies: TEDS(M), 17 countries
![Page 4: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/4.jpg)
4
Two abbreviations…PISA = Project for International Student Assessment
organised by OECD(Organisation for Economic Cooperation and Development)
TIMSS = Trends in International Maths and Science Studypreviously: Third Internat. Maths & Science Study
organised by IEA (Internat. Assoc. for the Evaluation of Educational Achievement)
![Page 5: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/5.jpg)
5
Translate and define
Mathematics Basic calculations and further
Science physics, chemistry, biology, physical
geography, astronomy, ecology, etc
Students all students (not just pre-university)
![Page 6: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/6.jpg)
6
NL USA Fla Jap Maroc Russ.
38 49 38 47 16 37
![Page 7: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/7.jpg)
7
NL USA Fla Jap Maroc Russ.
45 44 46 31 16 38
![Page 8: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/8.jpg)
8
NL USA Fla Jap Maroc Russ.
82 76 76 77 33 74
![Page 9: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/9.jpg)
9
NL USA Fla Jap Maroc Russ.
44 36 66 80 31 55
![Page 10: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/10.jpg)
10
NL 36
USA 19
Fla 21
Jap 44
Maroc
2
Russ. 9
![Page 11: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/11.jpg)
11
NL USA Fla Jap Maroc Russ.
81 74 71 78 45 76
![Page 12: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/12.jpg)
12
Some history…1964 A comparison of 12 countries, Husén & Bloom1971 FISS1980-82 SIMS 1984 SISS
1995 TIMSS (later known as “TIMSS-1995” )
1999 TIMSS-Repeat (later known as “TIMSS-1999”)
2003 TIMSS-2003
2007 TIMSS-20072008 TIMSS Advanced
TIMSS-2011
2000 PISA-2000
2003 PISA-2003
2006 PISA-2006
PISA-2009
![Page 13: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/13.jpg)
22 April 2009
To what extent do countries differ in the education of science and mathematics?
How can you do research on the question:
![Page 14: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/14.jpg)
14
Hours of instruction for science (grd 4)
![Page 15: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/15.jpg)
15
Teacher characteristics math (grd 8)
![Page 16: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/16.jpg)
22 April 2009
Both studies focus on a test
![Page 17: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/17.jpg)
17
Complex studies ..Assessment frameworkLarge number of itemsLong design process for instrumentsLarge random sampleQuality control trend items (retained) coding and re-coding excluding countries when criteria were
not met
![Page 18: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/18.jpg)
18
Pisa framework for science
![Page 19: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/19.jpg)
19
Pisa framework for science competencies
![Page 20: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/20.jpg)
20
Pisa framework for science
![Page 21: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/21.jpg)
21
Pisa framework for science
![Page 22: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/22.jpg)
22
Pisa science item ‘Acid Rain’
![Page 23: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/23.jpg)
23
Pisa science item ‘Acid Rain’
![Page 24: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/24.jpg)
24
Pisa science item ‘Acid Rain’
![Page 25: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/25.jpg)
25
Complex studies… (1)Not just measuring country scores in one discipline, but also: Sub-disciplines (algebra, ecology, ..) Sub-competencies
(reproduce, explain, communicate, ..) Background variables
Not just measuring students’ scores, but also trying to explain: Differences between countries Differences between schools Differences between students
![Page 26: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/26.jpg)
26
Complex studies… (2)PISA: 85 exercises distributed over 13 test booklets
PISA-2000 Main topic: Literacy (reading) Additional: science, maths
PISA-2003: Main topic: Mathematical Literacy Additional: literacy, science, problem solving
PISA-2006 Main topic: Science Additional: literacy, maths
![Page 27: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/27.jpg)
27
Complex studies… (3)TIMSS: 300 exercises distributed over 12 booklets
TIMSS-95: grades 3, 4, grades 7, 8 Performance Assessment (= test in a lab-like environment) grade 12: math/science literacy; advanced maths/science video study (USA, Jap, Germ.)
TIMSS-99: grade 8 TIMSS-R Video study (USA, Jap, NL, Cz, Switz, Austr, HongK.)
TIMSS-2003 grade 4, grade 8
TIMSS-2007 grade 4, grade 8
TIMSS-2008 grade 12
![Page 28: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/28.jpg)
28
Percentage Distribution Math Items PISA - TIMSS
15-year-olds
PISAGrade 8
TIMSSNumber 22% 33%
Measurement 18% 14%
Geometry 12% 18%
Data 40% 11%Algebra 11% 23%Classified to multiple strands 2% 0%
![Page 29: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/29.jpg)
29
Math Item Formats
Multiple Choice
Short Constructed
Response
Extended Constructed
Response
TIMSS 71% 16% 13%
PISA* 33% 42% 25%
*PISA definitions:Multiple Choice includes regular and “complex” multiple-choice items.Short Constructed Response includes “short answer” and “closed response” items.Extended Constructed Response includes “open response” items.
![Page 30: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/30.jpg)
30
Mathematical Complexity
LowModerat
eHigh
TIMSS 51% 46% 3%
PISA 29% 64% 7%
![Page 31: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/31.jpg)
31
PISA-2006 science country ranking
![Page 32: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/32.jpg)
32
PISA-2006 science country ranking
![Page 33: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/33.jpg)
33
TIMSS-2007country ranking
Maths grade 4
![Page 34: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/34.jpg)
34
TIMSS-2007 multi-country comparison
![Page 35: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/35.jpg)
35
Horse race - thinkingboth in PISA and TIMSS
Ranking is simplisticSample: intrinsically inaccurateScores of some countries are very close. With s.e.>3 are small differences not significantNL position in 2003: math: on shared positions 2-7 in PISA
and shared positions 6-9 in TIMSS science: on shared positions 4-8 in PISA
and shared positions 7-11 in TIMSS
![Page 36: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/36.jpg)
36
Differences PISA and TIMSSPopulation:
PISA: age cohort: 15-year olds (indep. of grade) TIMSS: grade (indep. of age)
Countries: PISA: economically developed (OECD) TIMSS: whole world (incl. Saudi Arabia, Botswana)
Cyclus: PISA: 3-yearly with different thematic foci TIMSS: 4-yearly with different populations
Content: PISA: applicable knowledge (situations): reading, math.
literacy, scientific literacy TIMSS: school knowledge (US curriculum): mathematics,
scienceJustification of the study:
PISA: knowledge has economical importance TIMSS: knowledge about education is needed
![Page 37: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/37.jpg)
37
Country score and GDP
![Page 38: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/38.jpg)
38
![Page 39: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/39.jpg)
39
TIMSS-1999 Video StudyGrade 8seven countries: Australia, Hong Kong, Japan, NL, Czech Rep, USA, SwitzerlandAim: describe patterns in lessonsquestion: what is typical for the math lessons in a certain country?
in each country 85 random lesson filmed, transcribed & coded
Points of analysis:•Structure of the lesson: e.g. who speaks? How many exercises?•Mathematical content: complexity, procedures•Char of exercise: symbols, contexts
![Page 40: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/40.jpg)
40
TIMSS-Video study: use of materials:Percentage lessons in which a calculator is used
![Page 41: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/41.jpg)
41
TIMSS-Video study: structure of lessonsPercentage lessons in which a goal is stated
![Page 42: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/42.jpg)
42
TIMSS-Video study: structure of lessonsPercentage lessons, in which a summary statement is made
![Page 43: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/43.jpg)
43
TIMSS-Video study: exercisesPercentage exercises per lesson with a context of symbols only
![Page 44: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/44.jpg)
44
TIMSS-1999 Video Study
USA
video USA
Grade 8 (14 yrs) Heterogeneous
non-problem segment:the teacher shows the class examples of angles formed by secants and tangents intersecting inside, on, or outside a circle.
![Page 45: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/45.jpg)
45
TIMSS-1999 Video Study
Netherlands
video NL
Grade 8 (14 yrs) Homogeneous (pre-university)
non-problem segment:the teacher briefly presents historical background related to the Pythagorean theorem (i.e., Pythagoras was a Greek gentleman). She also connects ideas learned previously (i.e., the definitions of various types of triangles) to the present lesson.
![Page 46: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/46.jpg)
46
TIMSS-1999 Video Study
Hong Kong: grd 8, heterogeneousExplanation of a new concept.
video HongKong
Pre-knowledge: students can solve linear equations
What preceded the scene:Solving2x+10=2(x+5)2x+10=2x+10 0 = 0Then different values were insterted for x .
left hand sideright hand side“expanded form”“factorized form”
![Page 47: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/47.jpg)
47
Next in this theatre…
TEDS(M)report in December 2009
TIMSS-2008 Advancedreport in December 2009
Pisa 2009report in 2010
![Page 48: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/48.jpg)
48
References for PISA and TIMSSwww.pisa.oecd.org
http://timss.bc.edu
![Page 49: 22 April 2009 Benefits and limitations of international comparative research in science and mathematics education AMSTEL – Seminar Pauline Vos Universiteit](https://reader035.vdocuments.us/reader035/viewer/2022070306/5519d4a9550346443e8b4bbb/html5/thumbnails/49.jpg)
49
remains the question….
what are the benefits and limitations?