predicting academic resilience with ... - university of macau reports/201409_parm.pdfuniversity of...
TRANSCRIPT
Kwok-cheung Cheung, Pou-seong Sit, Man-kai Ieong &
Soi-kei Mak,
Educational Testing and Assessment Research Centre,
University of Macau, Macao
Predicting academic resilience with
mathematics learning and demographic
variables: Comparing Macao, Hong
Kong, Korea, Japan, Canada, Estonia
and Finland
Paper presented at the 2014 European Conference on Educational Research(ECER) in Porto, Portugal, 2-5 September, 2014
Synopsis
• Mathematics is the major domain of literacy
assessment in PISA 2012.
• Based on the impact of the Economic, Social, and
Cultural Status (ESCS) of the student on mathematical
literacy performance, the equity of the participating
education systems is also assessed.
• This study analyzes data of selected mathematics
learning variables for the seven high-performing high-
equity economies (i.e. Macao, Hong Kong, Korea,
Japan, Canada, Estonia, and Finland).
• Special attention is paid to the ESCS-disadvantaged
students who are academically resilient in spite of
being in an unfavorable condition.
• Logistic regression is run to identify the predictive
learning mathematics and demographic variables.
• Across the seven economies under study, demographic
variables like gender, family structure, immigration
status, attending kindergarten and grade repetition, as
well as the four self-regulatory learning mathematics
variables, are found to differentiate within the group of
ESCS-disadvantaged between the academically
resilient and non-resilient students.
• The key finding is that across the seven economies
primary attention needs to be paid to students’
mathematics self-efficacy and familiarity with
mathematics concepts. Another two alterable
variables mathematics self-concept and mathematics
anxiety are also verified to affect particular
economies to different extents.
• After accounting for the effects of the demographic
variables, interesting patterns of effects are
discernible when the high-performing and high-
equity East Asian economies (Macao, Hong Kong,
Japan and Korea) are contrasted with that of the non-
East Asian counterparts (Canada, Estonia and
Finland).
Hypothesis of study
This study examines links between
resilience (and non-resilience) of the
ESCS-disadvantaged students with
pertinent learning mathematics variables
while controlling for the demographic
characteristics of the seven high-
performing high-equity economies in
PISA 2012.
Across the seven high-performing and high-
equity economies in PISA 2012, the self-
regulatory learning mathematics variables (i.e.
familiarity with mathematical concepts,
mathematics self-efficacy, mathematics self-
concept and mathematics anxiety), after
controlling for the effects of the demographic
covariates, have differential effects on the
classification of academic resilience in
mathematical literacy within the group of
ESCS-disadvantaged students.
Method
Sample
• This study utilizes data drawn from PISA 2012 in
which the seven economies Macao, Hong Kong,
Korea, Japan, Canada, Estonia and Finland
ranked amongst the top positions in the league
table of mathematical literacy performance, and at
the same time, bottom positions regarding the
impact of ESCS on student mathematical literacy
performance.
Dependent Variable
• The research interest lies with the academic resilience in
mathematical literacy within the subgroup of ESCS-
disadvantaged students in connection with the resilience
classification by four self-regulatory learning mathematics
variables, using the demographic variables as covariates.
• The difference between disadvantaged non-resilient and
disadvantaged resilient student regarding the effects of the
learning mathematics in the prediction of the resilience
class membership is examined in this study.
• The dependent variable corresponds to the two
classifications of the resilient and non-resilient students in
the subgroup of ESCS-disadvantaged students.
Method
Dependent Variable
• There are three steps in the identification of resilient students.
First, students located at the bottom quarter of the PISA index of
ESCS within their own economies are identified as
disadvantaged students.
• Second, literacy performance scores as assessed in PISA are
regressed on student ESCS across all participating economies to
find out the international ESCS-performance relationship.
• Third, student residual performance is obtained by comparing
the actual performance of each student with the performance
predicted by the international ESCS-performance relationship.
• Resilient student is identified as those whose residual
performance is amongst the top quarter of student residual
performance from all economies.
Method
Identification of disadvantaged resilient (DRS),
disadvantaged non-resilient (DNR) and
non-disadvantaged students (NDS) in Macao
0
200
400
600
800
1000
-4 -3 -2 -1 0 1 2 3
ESCS
Mathematical Literacy Score
Table 1: Percentage of students by resilience
classification
Economy Resilience classification (%) Total sample size
(N) NDS DNR DRS
Macao 75.0 8.0 17.0 5,335
Hong
Kong
75.0 6.7
18.3 4,670
Korea 75.0 12.2 12.8 5,033
Japan 75.0 13.6 11.4 6,351
Canada 75.0 16.6 8.4 21,544
Estonia 75.0 15.3 9.7 4,779
Finland 75.0 16.7 8.3 8,829
Note: NDS=Non-Disadvantaged Student; DNR=Disadvantaged Non-Resilient
student; DRS=Disadvantaged Resilient Student
Independent Variables
• Gender (X1): female=0; male=1.
• Family structure (X2): mixed or single family=0; nuclear
family=1. Students in nuclear family refer to those who are
living with both parents, whereas students in mixed or
single family are not.
• Immigration status (X3): first or second generation=0;
native=1.
• Years of attendance of kindergarten (X4): 0= one year or
less; 1= more than one year.
• Grade repetition (X5): 0=not repeated; 1= has repeated one
time or more in primary or secondary grades.
Method
Independent Variables
• Familiarity with mathematical concepts (X6)
• Mathematics self-efficacy (X7)
• Mathematics self-concept (X8)
• Mathematics anxiety (X9)
Method
Pearson correlation of the learning mathematics variables with
mathematical literacy performance in PISA 2012
Economy Familiarity
with
mathematical
concepts (X6)
Mathematics
self-efficacy
(X7)
Mathematics
self-concept
(X8)
Mathematics
anxiety
(X9)
East Asian economy
Macao .474 .513 .371 -.321
Hong
Kong .405 .553 .355 -.319
Korea .610 .622 .424 -.202
Japan .477 .580 .217 -.206
Non-East Asian economy
Canada .420 .553 .441 -.411
Estonia .320 .524 .439 -.470
Finland .434 .553 .516 -.445
OECD
Average .432 .529 .400 -.366
Analysis Strategies
• First, the sample characteristics (i.e. the demographic,
self-regulatory learning mathematics characteristics)
for the DRS versus DNR in each of the seven
economies are examined.
• Second, logistic regression is carried out for the DRS
vs DNR student classification, as a function of the
demographic and self-regulatory learning
mathematics characteristics.
Results
Sample Characteristics of the Seven
High-performing High-Equity
Economies
Sample characteristics of the DRS vs NDR
classification for Macao, Hong Kong, Korea and Japan Variables Macao Hong Kong Korea Japan
DRS DN
R
Total S.S DRS DN
R
Total S.S DRS DN
R
Total S.S DRS DN
R
Total S.S
Demographic characteristics (%)
Gender (X1)
0 = Female
1 = Male
65.9
68.6
34.1
31.4
100
100
71.7
73.1
28.3
26.9
100
100 *
49.5
52.5
50.5
47.5
100
100 *
40.4
49.4
59.6
50.6
100
100 *
Family structure (X2)
0= mix or single
family
1 = nuclear family
68.2
68.0
31.8
32.0
100
100
74.9
72.6
25.1
27.4
100
100 *
56.5
52.7
43.5
47.3
100
100 *
45.0
46.7
55.0
53.3
100
100 *
Immigration status
(X3)
0= first or second
generation
1 = native
70.8
58.2
29.2
41.8
100
100 *
74.2
70.7
25.8
29.3
100
100 * -- -- --
15.6
45.3
84.4
54.7
100
100 *
Attend kindergarten
(X4)
0 = one year or less
1 = more than one
year
54.1
69.9
45.9
30.1
100
100 *
61.9
73.5
38.1
26.5
100
100 *
45.3
52.6
54.7
47.4
100
100 *
27.1
46.0
72.9
54.0
100
100 *
Grade repetition (X5)
0 = not repeated
1 = repeated in
primary/secondary
85.3
48.1
14.7
51.9
100
100 *
77.3
53.9
22.7
46.1
100
100 *
51.7
30.1
48.3
69.9
100
100 * -- -- --
*p<.05; S.S. = Statistical significance (Chi-square/t-test)
Sample characteristics of the DRS vs NDR
classification for Macao, Hong Kong, Korea and Japan
Variables Macao Hong Kong Korea Japan
DRS DNR Total S.S DRS DNR Total S.S DRS DNR Total S.S DRS DNR Total S.S
Learning mathematics and problem solving characteristics (group mean)
Familiarity with
mathematical
concepts (X6) .751 -.001 .497 * .388 -.334 .192 * .492 -.097 .188 * .352 -.194 .058 *
Mathematics self-
efficacy (X7)
.235 -.580 -.025 * .242 -.820 -.076 * -.362 -1.206 -.765 * -.245 -1.189 -.775 *
Mathematics self-
concepts (X8) .072 -.418 -.086 * -.001 -.415 -.106 * -.175 -.781 -.460 * -.068 -.450 -.274 *
Mathematics anxiety
(X9) .078 .541 .227 * .056 .418 .149 * .285 .546 .409 * .258 .548 .415 *
*p<.05; S.S. = Statistical significance (Chi-square/t-test)
For Macao, Hong Kong, Korea and Japan, the
following findings are summarized:
• The Chi-square test of independence between gender and
resilience classification is statistically significant for Hong
Kong, Korea and Japan.
• The Chi-square test of independence between family structure
and resilience classification is statistically significant for Hong
Kong, Korea and Japan.
• The Chi-square test of independence between immigration status
and resilience classification is statistically significant for Macao,
Hong Kong, and Japan.
• The Chi-square test of independence between attend
kindergarten and resilience classification is statistically
significant for Macao, Hong Kong, Korea and Japan.
• The Chi-square test of independence between grade
repetition and resilience classification is statistically
significant for Macao, Hong Kong, and Korea.
• For familiarity with mathematical concepts,
mathematics efficacy, mathematics self-concepts, and
mathematics anxiety the DRS mean is statistically
significantly different from that of DNR (p<0.05).
Noteworthy is that the mean mathematics anxiety for
DNR is higher than that of DRS, whereas it is the
other way round for the other three learning
mathematics variables.
Sample characteristics of the DRS and NDR classification for
Canada, Estonia and Finland
Variables Canada Estonia Finland
DRS DNR Total S.S DRS DNR Total S.S DRS DNR Total S.S
Demographic characteristics (%)
Gender (X1)
0 = Female
1 = Male 29.8
36.9
70.2
63.1
100
100
* 38.0
39.2
62.0
60.8
100
100
31.4
33.6
68.6
66.4
100
100
*
Family structure (X2)
0= mix or single family
1 = nuclear family 35.4
34.0
64.6
66.0
100
100
* 43.2
38.3
56.8
61.7
100
100
* 31.9
34.8
68.1
65.2
100
100
*
Immigration status (X3)
0= first or second
generation
1 = native
33.0
34.3
67.0
65.7
100
100
* 22.5
40.3
77.5
59.7
100
100
* 13.4
34.2
86.6
65.8
100
100
*
Attend kindergarten (X6)
0 = one year or less
1 = more than one year 32.1
35.7
67.9
64.3
100
100
* 45.3
36.4
54.7
63.6
100
100
* 30.0
35.1
70.0
64.9
100
100
*
Grade repetition (X5)
0 = not repeated
1 = repeated in
primary/secondary
36.8
13.4
63.2
86.6
100
100
* 40.3
9.9
59.7
90.1
100
100
* 35.2
3.7
64.8
96.3
100
100
*
*p<.05; S.S. = Statistical significance (Chi-square/t-test)
Sample characteristics of the DRS and NDR classification for
Canada, Estonia and Finland
Variables Canada Estonia Finland
DRS DNR Total S.S DRS DNR Total S.S DRS DNR Total S.S
Learning mathematics characteristics (group mean)
Familiarity with
mathematical
concepts (X6)
.301 -.288 -.089 * .411 .098 .213 * -.189 -.769 -.580 *
Mathematics self-
efficacy (X7)
.365 -.479 -.203 * .073 -.471 -.260 * -.057 -.833 -.572 *
Mathematics self-
concepts (X8)
.571 -.105 .127 * .251 -.273 -.061 * .409 -.444 -.150 *
Mathematics
anxiety (X9)
-.277 .331 .124 * -.497 .246 -.052 * -.654 -.014 -.232 *
*p<.05; S.S. = Statistical significance (Chi-square/t-test)
For Canada, Estonia, and Finland, the following
may be summarized:
• The Chi-square test of independence between
gender and resilience classification is statistically
significant (p<0.05) for Canada and Finland.
• The Chi-square test of independence between
family structure and resilience classification is
statistically significant (p<0.05) for Canada,
Estonia and Finland.
• The Chi-square test of independence between
immigration status and resilience classification is
statistically significant (p<0.05) for Canada,
Estonia, and Finland.
• The Chi-square test of independence between
attend kindergarten and resilience classification is
statistically significant (p<0.05) for Canada,
Estonia, and Finland.
• The Chi-square test of independence between
grade repetition and resilience classification is
statistically significant (p<0.05) for Canada,
Estonia, and Finland.
• For familiarity with mathematical concepts,
mathematics efficacy, mathematics self-concepts,
and mathematics anxiety the DRS mean is
statistically significantly different from that of DNR
(p<0.05).
• Noteworthy is that the mean mathematics anxiety
for DNR is higher than that of DRS, whereas it is
the other way round for the other three self-
regulatory learning mathematics variables.
Logistic regression analysis of the
resilience classification (i.e. DRS vs
DNR) as a function of the
demographic and learning
mathematics variables
The following findings are
summarized for the five demographic
variables:
• Grade repetition (X5) is found predictive of the
resilience classification (except Japan where this
variable is not applicable to this education system).
• X5 is most predictive for Finland. The logistic
regression coefficient is -2.775 and the odds ratio is
0.062. This shows that when the other predictor
variables are held constant the chance that an ESCS-
disadvantaged student who have repeated in
primary/secondary grades is an academically resilient
student is 0.062 times of those who have not repeat
grades at all.
• Immigration status (X3) is found most predictive
of the resilience classification for Macao, Canada,
Estonia and Finland.
• X3 is most predictive for Finland. The logistic
regression coefficient is 1.182 and the odd ratio is
3.262. This shows that when the other predictor
variables are held constant the chance that an ESCS-
disadvantaged Finnish native student is academically
resilient is 3.262 times of the first or second
generation peers.
• Attend kindergarten (X4) is found predictive for Macao and
Estonia, but in different direction.
• In the case of Macao, The logistic regression coefficient is
0.612 and the odds ratio is 1.844. This shows that when the
other predictor variables are held constant the chance that an
ESCS-disadvantaged Macanese student who has attended more
than one year of kindergarten education is academically
resilient is 1.844 times of peers who attend one year or less.
• In the case of Estonia, The logistic regression coefficient
is -0.471 and the odds ratio is 0.625. This shows that when the
other predictor variables are held constant the chance that an
ESCS-disadvantaged Estonian student who has attended more
than one year of kindergarten education is academically
resilient is only 0.625 times of those who attend one year or
less.
• Gender (X1) is predictive only for Canada.
• The logistic regression coefficient is 0.226 and the
odds ratio is 1.254. This shows that when the other
predictor variables are held constant the chance that
an ESCS-disadvantaged Canadian male student is
academically resilient is 1.254 times of a female
peer.
• Family structure (X2) is not predictive in the
logistic regression analyses of the seven high-
performing high-equity economies of PISA 2012.
The following findings are
summarized for the four learning
mathematics variables:
• Mathematics efficacy (X7) is a powerful predictor of
the resilience classification for all the seven
economies under study.
• X7 is most predictive for Japan. The logistic regression
coefficient is 0.911 and the odds ratio is 2.486. This shows that
when the other predictor variables are held constant the chance
that an ESCS-disadvantaged Japanese student whose
mathematics efficacy increases one scale unit is academically
resilient is 2.486 times of peers who haven’t increased in
mathematics efficacy.
• Across the seven economies, higher level of mathematics
efficacy is found associated with better chance of academic
resilience, and this phenomenon is especially prominent in most
East Asian (i.e. Hong Kong, Korea and Japan; except Macao)
than the Non-East Asian economies (i.e. Canada, Estonia and
Finland).
• Familiarity with mathematics concepts (X6) is also
predicting the resilience classification very well.
The coefficients of X6 are statistically significant
for all the four East Asian and two of the three
East Asian economies (except Estonia).
• X6 is most predictive for Korea. The logistic regression
coefficient is 0.960 and the odds ratio is 2.610. This shows that
when the other predictor variables are held constant, the chance
that an ESCS-disadvantaged Korean student whose familiarity
with mathematics concepts increases one scale unit is
academically resilient is 2.610 times of his/her peers who
haven’t increased in familiarity with mathematics concepts.
• Across the six economies (except Estonia), higher level of
familiarity with mathematics concept is found associated with
better chance of academic resilience.
• Mathematics anxiety (X9) is able to make a contrast
between the East Asian economies and non-East
Asian economies regarding the prediction of the
resilience classification.
• For Canada, Estonia and Finland, the regression coefficients of
X9 are -0.175, -0.409 and -0.366 respectively, and the
corresponding odds ratios are 0.839, 0.664 and 0.694. The three
regression coefficients just mentioned are all statistically
significant (p<0.05), whereas those of the four East Asian
economies are not.
• Across the three non-East Asian economies, higher level of
mathematics anxiety is found statistically significantly
associated with lesser chance of academic resilience.
• Mathematics self-concept (X8) is able to make a
contrast between the East Asian economies (except
Macao) and non-East Asian economies regarding
the prediction of the resilience classification.
• For Canada, Estonia and Finland, the regression coefficients of
X8 are 0.429, 0.377 and 0.553 respectively, and the
corresponding odds ratios are 1.536, 1.458 and 1.739. The
three regression coefficients just mentioned are all statistically
significant (p<0.05), whereas those of the four East Asian
economies except Macao are not.
• Across the three non-East Asian economies, higher level of
mathematics self-concept is found statistically significantly
associated with higher chance of academic resilience, and the
only East Asian economy has this similar phenomenon is
Macao.
Implications of study
East Asian Economies
• Macao, Hong Kong, Korea and Japan have the largest
share of resilient students in their ESCS-disadvantaged
population, respectively amounting to 68%, 73%, 51%
and 46% of their 15-year-old cohort of secondary
students.
• The research evidence of this study show clearly that
ESCS-disadvantaged students who have attended
kindergarten more than one year, and who have not
repeat education in primary and/or secondary grades,
stand higher chances of beating the odds against them
and being identified as academically resilient than their
counterparts.
• This study establishes that the two learning
mathematics variables familiarity with mathematical
concepts and mathematics self-efficacy are not only
educational quality indicator variables, but also from
the resilience in learning perspective educational equity
indicator variables.
• Educationally and psychologically, students who are
more familiar with mathematical concepts and equipped
with higher degrees of mathematics self-efficacy attain
higher in mathematics literacy performance, and
importantly if they are ESCS-disadvantaged then they
stand higher chances to beat the odds against them to
become academic resilient students in their home
country/economy.
Non-East Asian Economies
• Canada, Estonia and Finland have a not small share of
resilient students in their ESCS-disadvantaged
population, amounting to 34%, 39%, and 33% of the 15-
year-old cohort of secondary students.
• The research evidence of this study show clearly that
ESCS-disadvantaged students who are male, native, and
who have not repeat education in primary and/or
secondary grades, stand higher chances of beating the
odds against them and being classified as academically
resilient than their counterparts.
• Apart from familiarity with mathematical concepts and
mathematics self-efficacy, this study establishes that there are two
other learning mathematics variables mathematics self-concept
and mathematics anxiety which are not only educational quality
indicator variables, but also from the resilience in learning
perspective educational equity indicator variables applicable
to the non-East Asian economies in this study.
• Educationally and psychologically, students who are less anxious
and equipped with higher degrees of mathematics self-concept
attain higher in mathematics literacy performance, and
importantly if they are ESCS-disadvantaged stand higher chances
to beat the odds against them to become academic resilient
students in their home country.
Conclusions
• Seven economies, four East Asian and three non-East Asian, are
considered as high-performing and high-equity in mathematical
literacy in PISA 2012.
• One hypothesis applicable to these seven economies is that
academic resilience of the local ESCS-disadvantaged students
helps raise academic performance and as a result of this
contributes to educational equity.
• Through logistic regression of the resilience classification this
study establishes a number of demographic and self-regulatory
learning mathematics characteristics that can predict whether an
ESCS-disadvantaged student is likely academically resilient or
not.
• Amongst the five demographic characteristics, grade repetition
and immigration status of student are important predictors for
both East Asian and non-East Asian countries, whereas
kindergarten attendance and gender of student predict certain
economies in unique local ways.
• Amongst the four learning mathematics characteristics, it is found
that when an ESCS-disadvantaged student is familiar with
mathematical concepts in the school curriculum and his/her
degree of mathematics self-efficacy in the tackling daily-life
mathematical tasks is high he/she is more likely an academic
resilient student.
• This conclusion is valid whether the student concerned comes
from the East Asian or non-East Asian economies or not.
• However, when an ESCS-disadvantaged student whose
degree of mathematics self-concept is high and at the same
time his/her mathematics anxiety is low, he/she is more likely
an academic resilient student.
• This conclusion is valid when the student concerned comes
from the three non-East Asian economies, i.e. Canada,
Estonia and Finland. It is also valid for mathematics self-
concept in the case of Macao, an economy with serious
problem of grade repetition.
Stakeholders can make reference of the findings
of this study to tailor pedagogy and psychology
of learning to improve quality and equity in
mathematics education in their own
countries/economies.
Thank you