The emerging paradigm The emerging paradigm for lifelong learning, for lifelong learning,

quantitative and quantitative and qualitative data from:qualitative data from:

The IEA Second International The IEA Second International Technology in Education Technology in Education

Study (SITES) Study (SITES)

byby

Hans Pelgrum, University Hans Pelgrum, University TwenteTwente

Content of presentation Professional background Background of current ICT studies

Role of ICT in education The information society, education and ICT Need for (curriculum) indicators and deeper

insight into innovations SITES Modules 1,2,3: overview SITESM2006 and EU-policies Conclusions

Professional background: assessment, monitoring and ICT

IEA Dutch Mathematics and Science assessments-1980-1985 Computers in Education Study-1989-1993 SITES-school survey: 1997-1999 SITES-case studies: 1999-2002 SITES-school, teacher, student surveys: planning/funding phase TIMSS-sec. Analyses (2002): high ICT uselower scores?

EU European network Assessment, Effectiveness, Innovation: 1995-1999 Multimedia technologies in schools: Euro-parliament: 1998 Methods of Educational monitoring: 1996

OECD Paper on possible directions international assessments ICT: 2001 Pilot testing ‘ICT and quality of learning’: 2000

Sundries Teacher training ‘Using the Web in Education’: Hungary, Netherlands, Poland.

Extension: Bulgaria, Latvia, Lithuania, Slovak Republic: 2001-2002

Background/Context of current international ICT studies

Role of ICT in education

The information society, education and ICT

Need for (curriculum) indicators and insight in realistic possibilities for innovation

SITES Modules (see also: www.iea.nl)

Module-1 (quantitative + little qualitative): a snapshot picture (school-survey) of the situation regarding ICT in education (1998-1999);

Module-2 (qualitative+little quantitative): observations of innovative practices in selected schools (1999-2002);

Module-3 : a repeat of Module-1 to determine changes across time and an assessment of readiness of teachers (2004-2007).

Countries in M1 and/or M2 Africa: South Africa Asia & Pacific: Australia, China Hong Kong, Chinese Taipei,

Japan, Korea, New Zealand, Singapore, Thailand Europe- Central and East: Bulgaria, Czech Republic, Hungary,

Latvia, Lithuania, Russian Federation, Slovenia, Slovak Republic Europe-West: Belgium-French, Cyprus, Denmark, Finland,

France, Germany, Iceland, Italy, Luxembourg, Netherlands, Norway, Portugal, Spain, United Kingdom

Middle-East: Israel North America: Canada, USA South America: Chile

Only M1

Topics for indicators in Module-1

Curriculum :ICT-related objectives of the school, Presence of types of teaching and learning practices, ICT attainment targets, Realization of ICT-related objectives, Use of e-mail/WWW for instructional purposes, Percentage of students/teachers using WWW, Internet-related activities of students. Use of technology applications by students

Infrastructure: Needs and priorities, Perceived obstacles, Expenditures, Software, Maintenance, Number and types of computers, Operating systems, Processor types, Hardware- Access to e-mail/WWW, Existence and content of Web home page, Number of computers not in use, Availability of peripherals, Availability of software types, Availability of software for school subjects, Hardware- and software-related obstacles

Staff Development: Prescriptions regarding training of teacher in the school, Attendance by teachers, Expenditures on staff development, Types of internal information exchange, Availability of in-house/external training courses, Self-assessment of ICT skills

Management and organization: Existence of written policies on ICT, Priorities for external support, ICT-related policy measures, Principal attitudes towards ICT, Use of ICT for administration/monitoring, Technical support infrastructure, Priorities for external support

Innovative practices: Most satisfying ICT-related learning activities experienced

Module-2: overview

Participants and international consortium Case selection process (after national

formation and nomination of cases) Case study process (protocols, procedures,

formats) Resulted in 174 cases: all coded on general

characteristics Selection, in-depth analysis and write up

SITES2006: overview

Conceptual: focus on pedagogical practices (see initial framework)

International coordination consortium: Univ. Twente (ICC), University HongKong, IEA-DPC

School surveys (n=400) and teacher surveys (math and science: n=1600)

Try-out online data collection Currently: start-up phase Expected participation: ~20 countries

External technology provision

School level

National/provin-cial district

policies on ICT in education, curriculum

External regulations

Software/ content

Teacher competencies

T/L practice ICT-use

Infrastructure School vision

Teacher vision

Objectives

Teacher level

assessment content time

location materials grouping

Pressure from community and

parents

SITESM3 and EU policies

Can IEA serve EU information needs? What do the current EU reports say? What can SITES contribute? What can other international studies

contribute?

EU can influence SITES2006

ConclusionsConclusions

• Case reports: examples of best practices

• Short term: also use PISA, TIMSS, PIRLS

• Optimize M3 to inform EU policies regarding, e.g.:– Infrastructure– Curriculum reform and quality– Training– Equity issues

Addendum: Quantitative indicators of innovative practices in Module-1 M1: Literature review operationalisations (extent to which present): Students developing abilities to undertake independent learning Providing weaker students with additional instruction Organizing teaching and learning so that differences in entrance

level, learning pace, and learning route are taken into account Students learning to search for information, process data, and

present information Students being largely responsible for controlling their own

learning progress Students learning and/or working during lessons at their own

pace Students involved in cooperative and/or project-based learning Students determining for themselves when to take a test Students learning by doing Combining parts of school subjects with one another

(multidisciplinary approach)

M1 conclusions

“Before summarizing the statistics regarding the curriculum indicators it should be pointed out that, at this stage of SITES, one needs to be cautious with regard to the interpretation of the indicators of pedagogical paradigms. These indicators seem to have face validity, and factor analyses showed the empirical tenability of the distinction between the emerging and traditionally important practices. However, the construct validity of these indicators has not yet been investigated in depth. Therefore it is not known yet if, for example, the indicator of emerging pedagogical practices really reflects the extent to which schools have implemented a curriculum that is focused on student-centered, active, and autonomous learning. (Pelgrum, 1999, page 91)”

Addendum: Questions

How can M2 help in understanding validity of M1-indicator?

Is it possible to increase the measurement range by adding items?

M2 case positions in M1 M2 case positions in M1 distributionsdistributions

0

10

20

30

40

50

60

70

80

90

100

Canad

a *

China H

ong K

ong

Czech

Rep

ublic

Denmark

Finlan

d *

France

Israe

l *Ita

ly *

Lithua

nia

Norway

Singap

ore

South

Africa

*

Thaila

nd

German

y

Netherl

ands

Phillip

ines

Portug

alSpa

in

%

0

10

20

30

40

50

60

70

80

90

100

ca-m

2ca

-m1

hk-m

2

hk-m

1

cz-m

2cz

-m1

dk-m

2

dk-m

1

fi-m2fi-

m1fr-

m2fr-

m1is-

m2is-

m1it-

m2it-

m1lt-

m2lt-

m1no

-m2

no-m

1

si-m2si-

m1sa

-m2

sa-m

1th-

m2th-

m1ge

-m2

ge-m

1nl-

m2nl-

m1ph

-m2

ph-m

1

po-m

2

po-m

1

sp-m

2

sp-m

1

m2

vernieuwend traditioneel 62.50

M1-M2 emerging M1-M2 emerging paradigm paradigm

indicator:longitudinalindicator:longitudinal

30.00

40.00

50.00

60.00

70.00

80.00

90.00

100.00

M1 M2

Addendum: New items Students learn search strategies to find diverse types of relevant information Students learn to assemble, organize and integrate information Students learn to critically evaluate the validity and worth of information obtained from

their searches Students present work using several forms of presentation, e.g., text, visual, verbal, and

electronic Students involved in collaborative activities where the outcomes are based upon

interdependent work Students assigned projects that require several persons working together for an

extended period of time Students have some authority to decide what topics to study Teachers assign problems where student selects types of evidence and appropriate

reasoning for the solution Students engage in intellectual discourse to exchange information and jointly solve

problems. Students learn to critically evaluate the bases of knowledge and the logical structure of

deductions and inferences made about this knowledge Students are encouraged to engage in self-reflection about the consequences of the

research strategies they use

Addendum: ConclusionsAddendum: Conclusions

• Many innovative cases available for analysis

• This set of cases will provide insight in best practices from Europe, Asia and North America

• M2 very valuable for providing input to M3