workshop of ict indicators in education ocde
DESCRIPTION
Open a file Play computer games Start a computer game Save a computer document or file Delete a computer document or file Draw pictures using a mouse Print a computer document or file Scroll a document up and down a screen Create/edit a document Move files from one place to another on a computer Copy a file from a floppy diskTRANSCRIPT
111111Workshop of ICT Indicators in
EducationSanto Domingo, March 4th, 2008
ICT Indicators: a work in progress
Bernard HugonnierOECD Deputy Director for Education
222Outline
What do we know?What do we need to know?How to proceed?
333Outline
What do we know?What we need to know?How to proceed?
444 Access to computers at school has increased rapidly between PISA 2000 and PISA 2003…
0
25
50
75
100
0
25
50
75
100
Den
mar
k
Liec
hten
stein
Aus
tralia
Cana
da
New
Zea
land
Hun
gary
Swed
en
Finlan
d
Unite
d St
ates
Thailand
Czec
h Re
publ
ic
Switze
rlan
d
Germ
any
Belgium
Latv
ia
Irelan
d
Mex
ico
Russ
ian
Fede
ration
Unite
d Ki
ngdo
m1
At home
At school
PISA 2003 PISA 2000
Source: OECD (2005) Source: OECD (2005) Are students ready for a technologyAre students ready for a technology--rich world? What PISA studies tell usrich world? What PISA studies tell us, Figure 2.5, p.22., Figure 2.5, p.22.1. Response rate too low to ensure comparability.
555 But in some countries students still have only limited opportunity to use computers at school.Number of computers per student (PISA 2003)
0.0
0.1
0.2
0.3
0.4
Liec
hten
stei
nU
nite
d S
tate
sA
ustr
alia
Kore
aH
unga
ryN
ew Z
eala
ndH
ong
Kong
-Chi
naA
ustr
iaCa
nada
Japa
nD
enm
ark
Luxe
mbo
urg
Icel
and
Nor
way
Sw
itze
rlan
dFi
nlan
dS
wed
enBe
lgiu
mN
ethe
rlan
dsIt
aly
Mac
ao-C
hina
Czec
h Re
publ
icIr
elan
dM
exic
oG
reec
eS
pain
Ger
man
yPo
rtug
alS
lova
k Re
publ
icPo
land
Latv
iaT
haila
ndU
rugu
ayIn
done
sia
Tur
key
Ser
bia
Russ
ian
Fede
rati
onBr
azil
Tun
isia
Uni
ted
King
dom
1
More than 10 students per computer
Source: OECD (2005) Source: OECD (2005) Are students ready for a technologyAre students ready for a technology--rich world? What PISA studies tell usrich world? What PISA studies tell us, Figure 2.8, p.27., Figure 2.8, p.27.
5 or fewer students per computer
1. Response rate too low to ensure comparability.
666 School principals’ reports on shortage of computers for instruction
0
10
20
30
40
50
60
70
80
Nor
way
Mex
ico
Spai
n
Pola
nd
Port
ugal
Swed
en
Irel
and
Gree
ce
Den
mar
k
Cana
da
Belg
ium
Czec
h Re
publ
ic
New
Zea
land
Japa
n
Finl
and
Net
herl
ands
1
Aus
tria
Aus
tral
ia
Germ
any
Icel
and
Ital
y
Hun
gary
Uni
ted
Stat
es
Luxe
mbo
urg
Swit
zerl
and
Kore
a
Percentage of students in schools whose principals report that instruction is hindered by a shortage of computers for instruction
PISA 2003 PISA 2000
Source: OECD (2005) Source: OECD (2005) Are students ready for a technologyAre students ready for a technology--rich world? What PISA studies tell usrich world? What PISA studies tell us, Figure 2.9, p.29., Figure 2.9, p.29.
1. Response rate too low to ensure comparability for PISA 2000.
2. Response rate too low to ensure comparability for PISA 2003.
777Canada
IcelandSweden
Liechtenstein
Australia
Korea
Denmark
Belgium
United St
Germany
Switzerland
Austria
New ZealandFinland
PortugalItalyCzech RepublicHungarySlovak Republic
Ireland
Poland
Uruguay
Greece
Tunisia
Serbia
Latvia
Mexico
Turkey
Russian Federation
JapanThailand
United Kingdom1
Percentage of students using a computer at least a few times each week
100%
0%
At home
At school
Percentage of students reporting they use computers “Almost
every day” or “A few times each week”:
Source: OECD (2005) Source: OECD (2005) Are students ready for a technologyAre students ready for a technology--rich world? What PISA studies tell usrich world? What PISA studies tell us, Figure 3.2, p.37., Figure 3.2, p.37.
1. Response rate too low to ensure comparability.
888What do students use computers to do?
PISA asked students how often they used:The Internet to look up information about people things or ideasGames on a computerWord processing (e.g. <Microsoft Word® or WordPerfect®>)The Internet to collaborate with a group or teamSpreadsheets (e.g. <Lotus 1 2 3® or Microsoft Excel®>)The Internet to download software (including games)Drawing, painting or graphics programs on a computerEducational software such as mathematics programsThe computer to help learn school materialThe Internet to download musicThe computer for programmingA computer for electronic communication (e.g. e-mail or “chat rooms”)
Students could choose from the following answers:Almost every day, A few times each week, Between once a week and once a month, Less than once a month, Never
999
1. Response rate too low to ensure comparability.
0
20
40
60
80
Cana
da
Uni
ted
Stat
es
Aus
tral
ia
Icel
and
Denm
ark
New
Zea
land
Swed
en
Aus
tria
Belg
ium
Kore
a
Port
ugal
Swit
zerl
and
OEC
D av
erag
e
Czec
h Re
publ
ic
Ital
y
Germ
any
Mex
ico
Gree
ce
Pola
nd
Hun
gary
Finl
and
Irel
and
Turk
ey
Slov
ak R
epub
lic
Japa
n
Uni
ted
King
dom
1
The Internet to look up information about people, things or ideas.Games on a computer.Word processing (e.g. <Word® or WordPerfect®>)
Access to computers at school is more universal than access to computers at home, but students report using
computers much more frequently at home
Percentage of students reporting they use the following “Almost every day” or “A few times each week”:
Source: OECD (2005) Source: OECD (2005) Are students ready for a technologyAre students ready for a technology--rich world? What PISA studies tell usrich world? What PISA studies tell us, ,
Figures 3.3 and 3.4, pp.39 and 41.Figures 3.3 and 3.4, pp.39 and 41.
Games = 53% on average
Internet research = 55%
on average
Word processing =
48% on average
101010
1. Response rate too low to ensure comparability.
0
20
40
60
80
Port
ugal
Uru
guay
Den
mar
k
Mex
ico
Ital
y
Tun
isia
Icel
and
Tha
iland
Uni
ted
Sta
tes
Aus
tral
ia
Tur
key
Slo
vak
Repu
blic
Aus
tria
Hun
gary
New
Zea
land
OEC
D a
vera
ge
Cana
da
Ser
bia
Ger
man
y
Czec
h Re
publ
ic
Pola
nd
Latv
ia
Belg
ium
Sw
eden
Gre
ece
Russ
ian
Fede
rati
on
Liec
hten
stei
n
Sw
itze
rlan
d
Kore
a
Finl
and
Irel
and
Japa
n
Uni
ted
King
dom
1
Educational software such as mathematics programs
The computer to help learn school material
A minority of students frequently use educational software on computers
Percentage of students reporting they use the following “Almost every day” or “A few times each week”:
Source: OECD (2005) Source: OECD (2005) Are students ready for a technologyAre students ready for a technology--rich world? What PISA studies tell usrich world? What PISA studies tell us, ,
Figure 3.4, p.41.Figure 3.4, p.41.
To learn school material = 30% on
average
Educational software = 13%
on average
111111 In general, students are confident in performing routine and Internet tasks on computers.
Routine tasks on a computer –percentage of students who are confident (OECD average)
Open a file 90 7Play computer games 90 7Start a computer game 86 10Save a computer document or file 88 8Delete a computer document or file 88 8Draw pictures using a mouse 85 10Print a computer document or file 86 9Scroll a document up and down a screen 87 8Create/edit a document 80 13Move files from one place to another on a computer 76 17Copy a file from a floppy disk 75 16
I can do this…
By myself
With help
Source: OECD (2005) Source: OECD (2005) Are students ready for a technologyAre students ready for a technology--rich world? What PISA studies tell usrich world? What PISA studies tell us, Table 3.9, p.110., Table 3.9, p.110.
121212 Internet tasks on a computer –percentage of students who are confident (OECD average)
I can do this…
By myself
With help
At least 90% of students report confidence in these tasks in Australia, Canada, Iceland, Korea, New Zealand, Sweden and the United States.
Get onto the Internet 88 7
Write and send e-mails 79 12Copy or download files from the Internet 70 19
Download music from the Internet 66 21
Attach a file to an e-mail message 58 24
Source: OECD (2005) Source: OECD (2005) Are students ready for a technologyAre students ready for a technology--rich world? What PISA studies tell usrich world? What PISA studies tell us, Table 3.11, p.112., Table 3.11, p.112.
131313 In general, 15-year-old boys report higher confidence than girls do in performing computing tasks and these
differences are particularly apparent for the more demanding computing tasks...
High-level tasks on a computer –percentage of students who are confident to perform
these tasks by themselves or with help (OECD average)
Source: OECD (2005) Source: OECD (2005) Are students ready for a technologyAre students ready for a technology--rich world? What PISA studies tell usrich world? What PISA studies tell us, Table 3.14, p.115., Table 3.14, p.115.
Boys Girls
Use software to find and get rid of computer viruses 79 54
Create a multi-media presentation (with sound, pictures, video) 77 62
Create a computer program (e.g. in Logo, Pascal, Basic) 63 48
Construct a Web page 71 61
Create a presentation (e.g. using <Microsoft® PowerPoint® > 79 70
Use a spreadsheet to plot a graph 79 70
Use a database to produce a list of addresses 85 79
141414 Average performanceof 15-year-olds in mathematics
High mathematics performance
Low mathematics performanceGreece
Russian Federation
Liechtenstein
Korea
Hong Kong-China
Finland
Netherlands
CanadaMacao-China Switzerland
New Zealand
BelgiumJapan
Australia
IcelandCzech Republic
SwedenFranceDenmark
Ireland GermanyAustria
Slovak Republic
LuxembourgPoland Hungary
Norway
SpainUnited StatesLatvia
PortugalItaly
460
480
500
520
540The 37% of students having more than 5 years of computer
experience scored at 532 points
On average across OECD countries
The 27% of students having between 3 and 5 years of
computer experience scored at 514 points
The 26% of students having between 1 and 2 years of
computer experience scored at 479 points
The 10% of students having less than 1 year of computer
experience scored at 430 points
Does computing experience counts ?
151515
0
20
40
60
80
100
120
140
Switze
rlan
d
Belgium
Icelan
d
Unite
d St
ates
Germ
any
New
Zea
land
Aus
tria
Den
mar
k
Italy
Kore
a
Hun
gary
Swed
en
Mex
ico
Aus
tralia
Poland
Port
ugal
Czec
h Re
public
Slov
ak R
epub
lic
Turk
ey
Japa
n
Cana
da
Gree
ce
Finlan
d
Irelan
d
between students who reported using computers less than one year and those using computers more than five years
between students who reported using computers less than one year and those using computers three to five years
between students who reported using computers less than one year and those using computers one to three years
… but the performance advantage varies across countries
161616
0
20
40
60
80
100
120
140
Switze
rlan
d
Belgium
Icelan
d
Unite
d St
ates
Germ
any
New
Zea
land
Aus
tria
Den
mar
k
Italy
Kore
a
Hun
gary
Swed
en
Mex
ico
Aus
tralia
Poland
Port
ugal
Czec
h Re
public
Slov
ak R
epub
lic
Turk
ey
Japa
n
Cana
da
Gree
ce
Finlan
d
Irelan
d
between students who reported using computers less than one year and those using computers more than five years
between students who reported using computers less than one year and those using computers three to five years
between students who reported using computers less than one year and those using computers one to three years
… and diminishes somewhat when socio-economicbackground factors are taken into account
171717Outline
What do we know?What do we need to know?How to proceed?
181818 What do we need to know?Much work has been done on the statistics of access and use of ICTs in schools, sometimes with an impressive array of details Most of these attempts have emphasised aspects such as quality of access, teacher preparation and time of actual use. Only a few, such as PISA (2005) have tried to connect these parameters with the desired effects on improving the quality of teaching and learning and educational performance.We therefore need to improve the available information in four specific areas
191919 What do we need to know?
1. The ICT environmentThe environment can be defined as the technical conditions that enable the development and use of ICT. Some of the most relevant might be:
access: how easy is it to get access to ICT both in schools and at home. ICT availability: how much ICT is available to users
2. The ICT user readiness This is the propensity of users to produce/use ICT . This propensity is linked to the level or ICT-related skills competence showed by teachers and pupils.
202020 What do we need to know?
3. The use of ICTThis refers to the actual use of ICT in teaching and learning activities, both by teachers and pupils, as well as the types of ICT used and for what purposes.
4. The effects of the use of ICTAny kind of measurable effect of the use of ICT either in the quality or in the output of the teaching and learning processes could be said to be the impact of using ICT. In addition, it could be reasonable to expect that an intensive use of ICT at large can result also in the learning of competences and skills not accounted for in traditional educational settings.
212121Outline
What do we know?What do we need to know?How to proceed?
222222How to proceed?
The ongoing OECD/CERI project on Digital Learning Resources *(DLR) includes among its final outputs the delivery of a proposal for a system of indicators that will benchmark the development of DLR. This proposal will come as a result of a standing working group in this area.
_________________________________________* DLR are defined in the context of this project as “any
digital resource that is actually used by teachers and learners for the purpose of teaching and learning”.
232323 What do we need to know?Indicators would need to be developed in the four identified areas:
The ICT environmentThe ICT User readinessThe ICT useThe effects of ICE use
242424 How to proceed?
1. The ICT environment access: An indicator merging data such as ratios of pupils per computer connected to the Internet, percentage of networked homes and the like can provide a useful quick look but more detailed indicators can also be considered, such as for example separating the values for teachers, pupils and families;DLR availability: This could be restricted to DLR in the national language (most appropriate for early school years), but even so could only be measured nationally on the basis of proxies such as, for example, the number of DLR available in the top three devoted national websites or the total number of national websites devoted to DLR.
252525 How to proceed?
2. The ICT user readiness Measuring readiness is a difficult task since no available data on this propensity exist (such as the precise answer to the question how ready teachers/pupils are to use/produce DLR). However, to start with it could be somewhat linked to the training or level or competence showed by different groups of users on the basis of already existing data, such as:
– teachers: training hours on ICT related competences by teacher or surveys of ICT-related skills and attitudes;
– pupils: surveys of ICT-related skills and attitudes (such as PISA data);
– families: general population surveys of ICT-related skills and attitudes.
262626 How to proceed?
3 & 4. ICT Use and effectsThere are no proxies available for the actual level of use of DLR at a national level, nor for assessing the effects. Therefore, a methodology has to be designed – to deal with the indicators in this area, – including, for example, different groups of users
(teachers, pupils, parents) and places of use (school, home).
272727 How to get there: two possible strategies
Strategy 1: Questionnaires to a sample of main stakeholders, like government representatives, teacher associations, parent organisations, webmasters and the like.Strategy 2: Questionnaires to a sample of school teachers, pupils and, eventually, to parents.
