mathematical modelling and statistical analysis of school ...€¦ · mathematical modelling and...

Mathematical Modelling and Statistical

Analysis of School-Based Student

Performance Data

Jessica Y. C. Tan

Thesis submitted for the degree of

Master of Philosophy

in

Applied Mathematics and Statistics

at

The University of Adelaide

School of Mathematical Sciences

January 2013

Contents

Page

List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix

List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii

Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii

Signed Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix

Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxi

1 Introduction 1

1.1 The Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 Background and Literature Review . . . . . . . . . . . . . . . . . . . 2

1.2.1 National Assessment Program - Literacy and Numeracy . . . . 2

1.2.2 My School . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.2.3 Value-added Measurement . . . . . . . . . . . . . . . . . . . . 6

Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.2.4 Rasch Scaling . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

1.2.5 The Estimation of School E�ects . . . . . . . . . . . . . . . . 12

1.2.6 Hierarchical and Longitudinal Modelling . . . . . . . . . . . . 13

1.3 Underlying Research Question . . . . . . . . . . . . . . . . . . . . . . 15

1.4 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2 Data Analysis 17

2.1 The Basic Skills Test Data . . . . . . . . . . . . . . . . . . . . . . . 17

2.2 The Variables: Background Information . . . . . . . . . . . . . . . . 18

2.3 Descriptive Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

2.3.1 Number of Participants in Schools . . . . . . . . . . . . . . . 21

2.3.2 Univariate Statistics . . . . . . . . . . . . . . . . . . . . . . . 21

Drop-o� in participants in 2004 . . . . . . . . . . . . . . . . . 22

iii

Test scores . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Cohort Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . 28

2.3.3 Number of Tests: Longitudinal View . . . . . . . . . . . . . . 43

2.3.4 Test Aspects . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

2.3.5 Missing Test Aspects . . . . . . . . . . . . . . . . . . . . . . 44

Missing Literacy Writing Results . . . . . . . . . . . . . . . . 46

Goodness of Fit - Binomial Model . . . . . . . . . . . . . . . . 47

2.4 Cleaning the Data: Forensic Statistics . . . . . . . . . . . . . . . . . 49

2.4.1 Consistency of School Data . . . . . . . . . . . . . . . . . . . 49

2.4.2 Consistency of Student Data . . . . . . . . . . . . . . . . . . 50

2.4.3 Score Check . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

2.4.4 Inference of Data . . . . . . . . . . . . . . . . . . . . . . . . . 61

2.5 Bivariate Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

2.5.1 Categorical Variables . . . . . . . . . . . . . . . . . . . . . . . 61

2.5.2 Quantitative Variables . . . . . . . . . . . . . . . . . . . . . . 71

2.5.3 Principal Component Analysis (PCA) . . . . . . . . . . . . . . 74

Background Theory . . . . . . . . . . . . . . . . . . . . . . . . 74

Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

2.6 Final Data Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

3 Initial Model Selection 81

3.1 Manual Reduction of Data and Predictor Variables . . . . . . . . . . 81

Categorical Variables . . . . . . . . . . . . . . . . . . . . . . . 82

Quantitative Variables . . . . . . . . . . . . . . . . . . . . . . 82

Missing Data . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

3.2 Statistical Reduction of Predictor Variables . . . . . . . . . . . . . . . 83

3.2.1 Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

Linear Regression . . . . . . . . . . . . . . . . . . . . . . . . . 84

Model Selection . . . . . . . . . . . . . . . . . . . . . . . . . . 84

3.2.2 Signi�cant Variables in Linear Regression . . . . . . . . . . . 85

School-Number Model . . . . . . . . . . . . . . . . . . . . . . 86

School-Covariates Model . . . . . . . . . . . . . . . . . . . . . 90

3.2.3 Simplest Main E�ects Model using stepAIC . . . . . . . . . . 91



3.2.4 Investigation of procyear . . . . . . . . . . . . . . . . . . . . 106

3.2.5 Comparison of School-Number and School-Covariates Models . 107

Statistical Theory . . . . . . . . . . . . . . . . . . . . . . . . . 111

Transformed Data . . . . . . . . . . . . . . . . . . . . . . . . 113

Signi�cant Schools . . . . . . . . . . . . . . . . . . . . . . . . 114

3.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

4 Hierarchical Modelling: Mixed E�ects Model 119

4.1 Theory of Linear Multilevel Mixed E�ects Models . . . . . . . . . . . 120

4.2 Hierarchical Model Formulation . . . . . . . . . . . . . . . . . . . . . 122

4.3 Hierarchical Model Selection . . . . . . . . . . . . . . . . . . . . . . . 124

4.3.1 Model Selection by Markov Chain Monte Carlo Sampling . . . 126

4.3.2 Model Selection by Likelihood Ratio Test . . . . . . . . . . . . 128

4.3.3 Model Selection by glmulti . . . . . . . . . . . . . . . . . . . 130

4.4 Analysis of Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

5 Bayesian Hierarchical Modelling 135

5.1 Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

5.1.1 Bayesian Statistics . . . . . . . . . . . . . . . . . . . . . . . . 135

5.1.2 Markov Chain Monte Carlo Simulation and Sampling . . . . . 136

Gibbs Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . 137

5.2 Hierarchical Modelling Using BUGS . . . . . . . . . . . . . . . . . . . 138

5.2.1 The Hierarchical Model . . . . . . . . . . . . . . . . . . . . . . 138

5.2.2 The Program . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

5.2.3 Directed Acyclic Graphs . . . . . . . . . . . . . . . . . . . . . 140

5.2.4 Analysis of BUGS output . . . . . . . . . . . . . . . . . . . . 141

Validity of the Model: Diagnosis of Convergence . . . . . . . . 144

Visualisation of Results . . . . . . . . . . . . . . . . . . . . . . 146

5.3 Hierarchical Modelling Using Stan . . . . . . . . . . . . . . . . . . . . 153

5.3.1 Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153

Hamiltonian Monte Carlo . . . . . . . . . . . . . . . . . . . . 153

No-U-Turn Sampler . . . . . . . . . . . . . . . . . . . . . . . . 154

5.3.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155

Validity of the Model: Diagnosis of Convergence . . . . . . . . 155

Interpretation of Regression Coe�cients . . . . . . . . . . . . 158

6 Model Validation 159

6.1 Student-level Prediction . . . . . . . . . . . . . . . . . . . . . . . . . 159

6.1.1 Prediction Intervals for lmer . . . . . . . . . . . . . . . . . . . 160

6.1.2 Prediction Intervals from Stan . . . . . . . . . . . . . . . . . . 161

6.1.3 Comparison of lmer and Stan . . . . . . . . . . . . . . . . . . 162

6.2 Analysis of School E�ect . . . . . . . . . . . . . . . . . . . . . . . . . 165

6.3 Heteroscedasticity and School Size . . . . . . . . . . . . . . . . . . . . 169

6.4 Conclusion and Impact . . . . . . . . . . . . . . . . . . . . . . . . . . 171

7 Initial Longitudinal Analysis 173

7.1 Summary Statistics of Data from Sequential Tests . . . . . . . . . . . 173

7.2 Grade 3 and Grade 5 Tests . . . . . . . . . . . . . . . . . . . . . . . . 174

7.2.1 Individual Scores . . . . . . . . . . . . . . . . . . . . . . . . . 175

7.2.2 Di�erence in Scores . . . . . . . . . . . . . . . . . . . . . . . . 175

Simple Linear Regression . . . . . . . . . . . . . . . . . . . . . 178

Hierarchical Modelling using Linear Multilevel Mixed E�ects

Models . . . . . . . . . . . . . . . . . . . . . . . . . 182

7.3 Grade 3, 5 and 7 Tests . . . . . . . . . . . . . . . . . . . . . . . . . . 186

7.3.1 Longitudinal Modelling . . . . . . . . . . . . . . . . . . . . . . 191

8 Conclusion 195

8.1 Summary and Conclusions . . . . . . . . . . . . . . . . . . . . . . . . 195

8.2 Practical Implications and Future Work . . . . . . . . . . . . . . . . . 196

A Coding of Variables 201

A.1 Test Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201

A.2 School Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201

A.3 Student Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203

B Plots 207

B.1 Boxplots of LL Rasch and NN Rasch for Categorical Variables . . . . 207

B.2 Grade 3 and Grade 5 Tests . . . . . . . . . . . . . . . . . . . . . . . 228

C Output 233

C.1 Chapter 3: Initial Model Selection . . . . . . . . . . . . . . . . . . . . 233

C.1.1 Full model with Main E�ects . . . . . . . . . . . . . . . . . . 233



C.1.2 Simplest Main E�ects Model by stepAIC . . . . . . . . . . . . 245



C.2 Chapter 7. Initial Longitudinal Analysis . . . . . . . . . . . . . . . . 258

C.2.1 Grade 3 and Grade 5 Tests . . . . . . . . . . . . . . . . . . . 258


References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267

List of Tables

Page

2.2.1 Test variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.2.2 School variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

2.2.3 Student variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

2.3.1 Number and percentage of participants in each calendar year . . . 22

2.3.2 Number and percentage of participants in each grade . . . . . . . . 22

2.3.3 Number of participants for each calendar year and grade . . . . . . 23

2.3.4 Number of participants and schools each year . . . . . . . . . . . . 24

2.3.5 Descriptive statistics for raw and Rasch scores for Literacy and Nu-

meracy aspects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

2.3.6 Coding for cohort numbers . . . . . . . . . . . . . . . . . . . . . . . 28

2.3.7 Number of participants in each cohort . . . . . . . . . . . . . . . . . 29

2.3.8 Mean scores for each cohort in each grade . . . . . . . . . . . . . . . 42

2.3.9 Number of participants by year, grade and number of tests to date . 43

2.3.10 Number of participants with test aspects in each year and grade . . 45

2.3.11 Number and proportion of participants with missing aspects . . . . 46

2.3.12 Number of participants with missing LW by year . . . . . . . . . . . 46

2.3.13 Number of participants with missing scores for aspects . . . . . . . 48

2.3.14 Observed and expected frequencies for the goodness-of-�t χ2 test of

the Binomial model . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

2.4.1 Anomalies in student data . . . . . . . . . . . . . . . . . . . . . . . 51

2.4.2 Number of schools and participants with anomalies in the sum of

scores . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

2.4.3 Number of participants in 2003 and 2004 with Literacy Flag 1 . . . 52

2.4.4 Number of participants in 2001 and 2002 with Literacy Flag 3 . . . 53

ix

2.4.5 Linear regression output from LL vs (LR+LS+LW) for Literacy

Flag 1 tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

2.4.6 Linear regression output from LL vs LR, LS and LW individually

for Literacy Flag 1 tests . . . . . . . . . . . . . . . . . . . . . . . . 58

2.5.1 ANOVA for LL Rasch against p_g_nesb . . . . . . . . . . . . . . . 62

2.5.2 ANOVA for NN Rasch against p_g_nesb . . . . . . . . . . . . . . . 62

2.5.3 ANOVA for NN Rasch against visa_sub_c . . . . . . . . . . . . . . 62

2.5.4 P -value and largest di�erence in group means for categorical vari-

ables against LL Rasch and NN Rasch . . . . . . . . . . . . . . . . 65

2.5.5 P -value and R2 value for quantitative variables against LL Rasch

and NN Rasch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

2.5.6 The standard deviation, proportion of variance explained and the

cumulative proportion for each of the principal components . . . . . 76

3.2.1 Regression output for gpokm and school size from the school-covariates

model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

3.2.2 Regression output for gpokm and school size from the simplest-

stepAIC school-covariates model . . . . . . . . . . . . . . . . . . . . 102

3.2.3 Comparison of models with and without procyear - simplest-stepAIC

school-number model . . . . . . . . . . . . . . . . . . . . . . . . . . 102

3.2.4 Comparison of models with and without procyear - simplest-stepAIC

school-covariates model . . . . . . . . . . . . . . . . . . . . . . . . . 104

3.2.5 Comparison of models with and without p_g_nesb - simplest-stepAIC

school-number model . . . . . . . . . . . . . . . . . . . . . . . . . . 104

3.2.6 Comparison of models with and without p_g_nesb - simplest-stepAIC

school-covariates model . . . . . . . . . . . . . . . . . . . . . . . . . 104

3.2.7 Type II Anova test for the full school-number model . . . . . . . . . 105

3.2.8 Type II Anova test for the full school-covariates model . . . . . . . 105

3.2.9 Summary linear regression output of procyear in the school-number

model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106

3.2.10 Summary linear regression output of procyear in the school-covariates

model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

3.2.11 Subset of data to illustrate raw and �tted scores . . . . . . . . . . . 108

3.2.12 Summary output of Raw ∼ Model linear regression . . . . . . . . . 108

3.2.13 Summary output of linear regression of transformed data . . . . . . 114

3.2.14 Comparison of the number of statistically signi�cant over- and under-

performing schools . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

3.2.15 χ2 test for association between the signi�cance groups of schools

and school covariates . . . . . . . . . . . . . . . . . . . . . . . . . . 117

4.3.1 Output of �xed and random e�ects from linear mixed e�ects model 125

4.3.2 Estimates and P -values of �xed and random e�ects estimated by

Markov Chain Monte Carlo sampling . . . . . . . . . . . . . . . . . 127


maximum likelihood . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

5.2.1 Summary output of hierarchical model without procyear using

OpenBUGS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143

5.3.1 Summary output of hierarchical model with procyear as �xed e�ect

using Stan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157

6.1.1 Counts and proportions of students in each performance category

from the lmer model . . . . . . . . . . . . . . . . . . . . . . . . . . 161

6.1.2 Counts and proportions of students in each performance category

from the Stan model . . . . . . . . . . . . . . . . . . . . . . . . . . 162

6.1.3 3×3 table of the counts of students for performance categories under

lmer and Stan models . . . . . . . . . . . . . . . . . . . . . . . . . 162

7.1.1 Counts of students who sat two appropriate sequential tests . . . . . 174

7.2.1 ANOVA between Grade 3 and 5 scores and schoolno . . . . . . . . 177

7.2.2 Linear regression output of school-covariates model . . . . . . . . . 181

7.2.3 Output of �xed and random e�ects from linear mixed e�ects model 183


Markov Chain Monte Carlo sampling . . . . . . . . . . . . . . . . . 184


maximum likelihood . . . . . . . . . . . . . . . . . . . . . . . . . . . 185

7.3.1 Counts of data for all the categorical predictors in the data of Grade

3, 5 and 7 scores . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187

7.3.2 Counts of students in the categories of atsi and school_edu . . . . 193

C.1.1 Linear regression output of school-number model . . . . . . . . . . . 233

C.1.2 Linear regression output of school-covariates model . . . . . . . . . 244

C.1.3 Linear regression output of simplest-stepAIC school-number model . 245

C.1.4 Linear regression output of simplest-stepAIC school-covariates model256

C.2.1 Linear regression output of school-number model . . . . . . . . . . . 258

List of Figures

Page

1.2.1 National Assessment Scale for NAPLAN . . . . . . . . . . . . . . . 3

1.2.2 Exemplar of My School . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.3.1 Number of participants in each school. . . . . . . . . . . . . . . . . 21

2.3.2 Ratio of participants in 2004 compared to 2003 for all schools in all

grades. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

2.3.3 Histograms of Literacy raw scores over all years and grades. . . . . . 26

2.3.4 Histograms of Literacy Rasch scores over all years and grades. . . . 27

2.3.5 Boxplots of Literacy scores for the cohorts in Grade 3 . . . . . . . . 30

2.3.6 Boxplots of Numeracy scores for the cohorts in Grade 3 . . . . . . . 31





2.3.11 Histograms of raw and Rasch LL and LW Grade 3 scores in Cohort 8 37

2.3.12 Mean aspect score in each multi-year cohort and across grades. . . . 39

2.3.13 Cohort mean scores along grades in each Literacy aspect . . . . . . 40

2.3.14 Cohort mean scores along grades in each Numeracy aspect . . . . . 41

2.3.15 Proportion of participants in each school missing LW. . . . . . . . . 47

2.4.1 Proportion of participants in a school which have Literacy Flag 1

in 2003 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54


in 2004 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55


in 2001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

xiii


in 2002 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

2.4.5 Plot of LR+LS+LW versus LL for Literacy Flag 1 tests . . . . . . . 59

2.4.6 Plot of LR+LS+LW versus LL for Literacy Flag 1 tests . . . . . . . 60

2.5.1 Heatmap of the P -values from the χ2 test for independence between

the categorical explanatory variables. . . . . . . . . . . . . . . . . . 63

2.5.2 Boxplot of LL Rasch against disability . . . . . . . . . . . . . . . 67

2.5.3 Boxplot of LL Rasch against procyear . . . . . . . . . . . . . . . . 68

2.5.4 Boxplot of LL Rasch against gradedyear . . . . . . . . . . . . . . . 68

2.5.5 Boxplots of LL Rasch and NN Rasch for gender . . . . . . . . . . . 69

2.5.6 Boxplot of LL Rasch against procyear split into grades . . . . . . . 70

2.5.7 Heatmap of the correlation between the quantitative explanatory

variables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

2.5.8 Barplot of the variances explained by the principal components. . . 75

2.5.9 3D scatter plot of the scores of the �rst three principal components. 75

2.6.1 Flowchart of data sets. . . . . . . . . . . . . . . . . . . . . . . . . . 79

3.2.1 Coe�cient plot for the school-number model . . . . . . . . . . . . . 88

3.2.2 Subset of coe�cient plot for the school-number model . . . . . . . . 89

3.2.3 Coe�cient plot for the school-covariates model . . . . . . . . . . . . 92

3.2.4 Subset of coe�cient plot for the school-covariates model . . . . . . . 93

3.2.5 Coe�cient plot for the simplest-stepAIC school-number model . . . 96

3.2.6 Subset of coe�cient plot for the simplest-stepAIC school-number

model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

3.2.7 Coe�cients for the simplest-stepAIC school-number model plotted

against the regression coe�cients for the school-number model . . . 98

3.2.8 Coe�cient plot for the simplest-stepAIC school-covariates model . . 100

3.2.9 Subset of coe�cient plot for the simplest-stepAIC school-covariates

model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

3.2.10 Coe�cients for the simplest-stepAIC school-covariates model plot-

ted against the regression coe�cients for the school-covariates model 103

3.2.11 Linear regression of original raw mean school e�ects against the

�tted original model mean school e�ects . . . . . . . . . . . . . . . 109

3.2.12 Diagnostic plots to assess validity of assumptions in the linear re-

gression of Raw ∼ Model. . . . . . . . . . . . . . . . . . . . . . . . 110

3.2.13 Plot of residuals versus school size. . . . . . . . . . . . . . . . . . . . 111

3.2.14 Plot of residuals versus school size for linear regression on trans-

formed data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114

3.2.15 Prediction intervals of the original and transformed data . . . . . . 116

4.0.1 Hierarchy of the school education system. . . . . . . . . . . . . . . . 120

4.3.1 AIC value for the best 100 models in the exhaustive search of the

main e�ects model . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

4.3.2 The relative weights of model terms . . . . . . . . . . . . . . . . . . 132

5.2.1 Directed acyclic graph for hierarchical model in equation (5.2.2). . . 141

5.2.2 Histogram of parameters' posterior distribution. . . . . . . . . . . . 146

5.2.3 Density plot of parameters' posterior distribution . . . . . . . . . . 147

5.2.4 Traceplots of parameters . . . . . . . . . . . . . . . . . . . . . . . . 148

5.2.5 Running means of parameters in each chain. . . . . . . . . . . . . . 148

5.2.6 Autocorrelation plots of parameters. . . . . . . . . . . . . . . . . . . 149

5.2.7 Crosscorrelations of parameters. . . . . . . . . . . . . . . . . . . . . 150

5.2.8 Highest posterior density plot for the school and student parameters.152

5.3.1 Comparison of NUTS with Metropolis and Gibbs sampling . . . . . 155

5.3.2 Autocorrelation plot of parameters from Stan. . . . . . . . . . . . . 156

6.1.1 Plot of the Stan �tted values against the lmer �tted values . . . . . 164

6.2.1 Ranked proportion of students who are over-performing in a school

plotted with the 95% con�dence interval . . . . . . . . . . . . . . . 165

6.2.2 Ranked proportion of students who are under-performing in a school

plotted with the 95% con�dence interval . . . . . . . . . . . . . . . 166

6.2.3 Normal Q-Q plot of the random e�ects estimates. . . . . . . . . . . 168

6.3.1 Plot of the variance of the residuals for each school against school size170

6.4.1 Student scores for school 115 . . . . . . . . . . . . . . . . . . . . . . 171

7.2.1 Individual Rasch scores for Grades 3 and 5 . . . . . . . . . . . . . . 176

7.2.2 Di�erence versus Average in Grade 3 and Grade 5 scores. . . . . . . 177

7.2.3 Mean di�erence in Grade 3 and Grade 5 values in each school . . . 178

7.3.1 Descriptives for the Grade 3, 5 and 7 data . . . . . . . . . . . . . . 188

7.3.2 Individual Rasch scores for Grades 3, 5 and 7 . . . . . . . . . . . . . 189

7.3.3 Possible trends between Grades 3, 5 and 7 . . . . . . . . . . . . . . 190

7.3.4 Longitudinal and hierarchical structure of the education system. . . 191

B.1.1 Boxplot of LL Rasch against procyear split into grades. . . . . . . 227

B.2.1 Descriptives for the Grade 3 and 5 data - page 1 . . . . . . . . . . . 229



Abstract

In order to improve the education system so that students are not only reaching the

minimum standards of literacy and numeracy but are also given the opportunity to

excel, accurate measures of school performance are vital. These measures need to

focus on student progress so that schools and teachers can focus on improving all

students - particularly those most in need.

A current trend in primary and secondary education in Australia is national testing,

with the introduction of the National Assessment Program - Literacy and Numeracy

(NAPLAN). This, along with similar tests like the Basic Skills Tests, measures

student performance and progress over sequential years. The issue now is how to

interpret these results for individuals, schools and educational authorities.

The education system can be modelled with a hierarchical model due to the nesting

of variables at the education system, school, classroom and student levels. Inter-

twined with the hierarchical nature of the model is also the longitudinal aspect of

the data as students sit biennial NAPLAN assessments. We mainly investigate hier-

archical modelling techniques, such as linear multilevel mixed e�ects and a Bayesian

approach, to �t a statistical model which incorporates student and school predictor

variables. In addition to hierarchical modelling methods, we also consider longitu-

dinal techniques.

The objective of this thesis is to investigate and determine what conclusions about

student progress and school performance can be reliably drawn from regular stan-

dardised system-wide assessment, such as NAPLAN. Various models are investigated

and the end result is a model which comprehensively and accurately models the re-

sults of students, from which we can predict students' scores and assess the e�ect of

schools in that way.

xvii

Declaration

I, Jessica Tan, certify that this work contains no material which has been accepted

for the award of any other degree or diploma in any university or other tertiary insti-

tution and to the best of my knowledge and belief, contains no material previously

published or written by another person, except where due reference has been made

in the text. In addition, I certify that no part of this work will, in the future, be

used in a submission for any other degree or diploma in any university or other ter-

tiary institution without the prior approval of the University of Adelaide and where

applicable, any partner institution responsible for the joint-award of this degree.

I give consent to this copy of my thesis, when deposited in the University Library,

being made available for loan and photocopying, subject to the provisions of the

Copyright Act 1968.

I also give permission for the digital version of my thesis to be made available

on the web, via the University's digital research repository, the Library catalogue

and also through web search engines, unless permission has been granted by the

University to restrict access for a period of time.

Signature: ....................... Date: .......................

xix

Acknowledgements

Firstly, I would like to thank my supervisors, Professor Nigel Bean and Dr Jono Tuke,

for their invaluable supervision and support during these two years of my Masters -

at the end of this journey, you are more like friends than supervisors. Both of you

have given generously of your time and mathematical expertise and your thoughts

are insightful and explanations clear, re�ecting the depth and breadth of your knowl-

edge. I accredit you and thank you for teaching me research skills and adding tools

to my virtual toolbox. I consider it a privilege that you both agreed to supervise me.

To my family, you get a big thank you. Thank you, Mum and Dad, for allow-

ing me to spend two years completing this Masters and for your continual support

and encouragement throughout my entire education. Thank you to my brother Dar-

ren for helping me take a break from work on occasion, and to my sisters, Sarah

and Hannah.

Thanks is also due to Dr Murray Thompson, a great friend and mentor from my

school days, who provided a wealth of information about the education theory as-

pect of my project. I would also like to acknowledge Professor John Keeves for his

help and Dr Darmawan for providing the data set.

My �home away from home� has been the School of Mathematical Sciences at the

University of Adelaide and I would like to express my appreciation for the wonderful

sense of community among the o�ce sta�, lecturers and students. A great part of

my enjoyment of university has been because of all my friends and fellow postgrads

- there are too many to name individually - but I would like to particularly thank

Pricey, Michael, Hayden, Sophie, Kale, Ben, Mingmei, Kate M, Kate R, Kyle, Lydia

xxi

A and Max for all the fun and laughs we have had.

�Mathematics is the queen of the sciences.�

(Carl Friedrich Gauss)

�The essence of mathematics is not to make simple things complicated,

but to make complicated things simple.�

(Stan Gudder)

�What lies behind us, and what lies before us are tiny matters,

compared to what lies within us.�

(Ralph Emerson)

Chapter 1

Introduction

1.1 The Problem

There has been substantial interest in policy activities related to outcomes-based

performance indicators [6, 8, 15, 22]. A performance indicator is a summary statis-

tical measurement on an institution or system which is related to the `quality' of

its functioning and may measure di�erent aspects of the system or re�ect di�erent

objectives. One application of performance indicators is in education and the grow-

ing demand for the accountability of teachers and schools. Coupled with this, is the

ranking of schools in school league tables [20, 23]. Many examples of the assess-

ment of education systems based on this structure exist internationally, for example

in England and Scotland [23, 39]. This topic especially applies to the Australian

education system in the last few years with the introduction of the NAPLAN tests

and the controversial My School 1TM

website. As a result of the My School website,

much debate has �ared up over the accuracy and interpretation of the ranking of

Australian schools.

Even with all the limitations of league tables and the associated statistical issues

involved in comparisons of school performance, it is possible that they are useful if

used carefully and in the proper context. However, care must be taken in the making

of conclusions and the question is �what can be sensibly and reliably concluded?�.

1 http://www.myschool.edu.au

1

2 1.2. Background and Literature Review

1.2 Background and Literature Review

To provide context to this problem, some of the research in measuring and esti-

mating school e�ects and the speci�cs of the Australian education system under

consideration will be discussed as background and a literature review.

1.2.1 National Assessment Program - Literacy and Numer-

acy

The National Assessment Program - Literacy and Numeracy (NAPLAN) was �rst

introduced in 2008 as part of the federal government initiative to support national

literacy and numeracy levels. NAPLAN is part of the National Assessment Program

(NAP), the measure through which governments, education authorities and schools

can determine whether or not young Australians are meeting important educational

outcomes [35]. NAP encompasses tests endorsed by the Ministerial Council for Edu-

cation, Early Childhood Development and Youth A�airs (MCEECDYA). Both NAP

and NAPLAN are under the jurisdiction of the Australian Curriculum, Assessment

and Reporting Authority (ACARA) which is �the independent authority responsible

for the development of a national curriculum, a national assessment program and a

national data collection and reporting program that supports 21st century learning

for all Australian students� [1].

NAPLAN replaced a variety of state-based exams, including the Basic Skills Test

in South Australia. All students in Years 3, 5, 7 and 9 of government and non-

government schools take part in the nation-wide standardised NAPLAN tests and

are examined in the domains of Reading, Writing, Language Conventions (Spelling,

Grammar and Punctuation) and Numeracy at their respective year levels. NAPLAN

is designed to test the requirements for literacy and numeracy common amongst the

curricula of each state and territory. NAPLAN tests are developed collaboratively by

the States and Territories, the Australian Government and non-government school

sectors with the aid of eminent assessment and educational measurement experts.

In addition, national protocols for test administration aim to ensure consistency

in administering the tests by all test administration authorities and schools across

Australia [1].

Chapter 1. Introduction 3

The NAPLAN test results are reported via individual student, school and national

reports. These provide information on how students have performed in relation

to other students in the same year group, against their state or territory, the na-

tional average and the National Minimum Standards. Student outcomes are re-

ported against achievement bands for each of the �ve NAPLAN assessment domains

of Reading, Writing, Spelling, Grammar and Punctuation and Numeracy (Figure

1.2.1). The range of the bands, from one to ten, re�ects the increasing complexity

of skills and understandings demonstrated by a student and assessed by NAPLAN

testing as the student progresses from Year 3 to Year 9. For any one year, the full

range of student performance is reported using six of the ten bands [1].

Figure 1.2.1: National Assessment Scale for NAPLAN. [35]

The use of common national assessment scales and reporting bands that span Years

3, 5, 7 and 9 enable the progress of an individual's or group's performance to be

monitored. To ensure that it is possible to compare scores from the current year's

tests with those of the previous years, test di�erences between the years are taken

into account, using a rigorous equating process [1].

Other results which are published in the NAPLAN National Report for each year

level, test domain and state, territory and Australia, are:

A NOTE:

This figure/table/image has been removed to comply with copyright regulations. It is included in the print copy of the thesis held by the University of Adelaide Library.


• NAPLAN results by gender, Indigenous status, language background other

than English status, parental occupation, parental education and location,

• the performance of each state and territory relative to other states, territories

and the whole of Australia,

• participation rates and participation categories,

• comparison between the results of di�erent calendar years in the form of pair-

wise di�erences (classi�ed into average achievement statistically signi�cantly

higher, no statistically signi�cant di�erence and average achievement statisti-

cally signi�cantly lower), and

• cohort gains or the di�erence between results across two years of testing.

NAPLAN results are reported in the form of mean scale scores with variance repre-

sented by standard deviations or con�dence intervals of the mean.

1.2.2 My School

The My School website graphically and numerically publishes the NAPLAN results

for each school in Australia. However, it only displays the average score of a school's

students in the NAPLAN assessments for each domain and year level, the margin of

error at a 90% level of con�dence and compares a particular school to the average for

all Australian schools (Figure 1.2.2). TheMy School website also reports school per-

formance by comparing schools' NAPLAN scores within statistically-similar-school

groups. These school groups are determined by various demographic and socio-

economic factors like remoteness, Indigenous population and proxies of the socio-

economic status of the student population, summarised by the Index of Community

Socio-Educational Advantage (ICSEA) for each school [28, 41].

For each school, the My School website includes

• a school pro�le - information about the student and sta� population,

• information about school �nances,

• a list of `like schools' throughout Australia,


Figure 1.2.2: Exemplar of the My School website. [35]

• NAPLAN results in graphs, numbers and bands,

• a breakdown of the percentage of students in each NAP band for each domain,

the data is then compared with the national and `like school' results,

• student progress over time,

• a list of twenty local schools, including detailed results,

• data on vocational education and training program results, and

• the school Index of Community Socio-Economic Advantage (ICSEA) value.

NAPLAN results are reported using averages and their stated purpose is to be able

to measure the school e�ect. However, that concept is de�ned as something known

as value-added measurement.

A NOTE:



1.2.3 Value-added Measurement

Theory

There has been much discussion in the literature advocating the use of value-added

scores over raw averages [21, 38]. The Organisation of Economic Co-operation and

Development (OECD) de�ned school value-added as:

�The contribution of a school to students' progress towards stated or pre-

scribed education objectives (for example, cognitive achievement). The

contribution is net of other factors that contribute to students' educa-

tional progress.� [28]

Given the above de�nition, value-added modelling was similarly de�ned by the

OECD as:

�A class of statistical models that estimate the contributions of schools to

student progress in stated or prescribed education objectives (eg. cognitive

achievement) measured at at least two points in time.� [28]

Based on this theory, the major recommendation of the Grattan Institute report [28]

is to replace measurement of average school performance with so-called value-added

indices. The idea is to measure student progress as the primary outcome and employ

an appropriate statistical regression model to extract the school-attributable compo-

nent of the improvement. The report concluded that measuring student outcomes at

one time point, which is the current measure of school performance published on the

My School website in the form of NAPLAN scores, and averaging over each school

does not provide a valid measure of school performance. Obviously, this depends

on what one means by a school's performance. If one means the ability of a school

to attract and retain smart students while deterring less smart students, then the

average student outcome is probably a very meaningful measure. But if by school

performance, it refers to the e�ect of the school on the student's learning outcome,

then school averages will be hopelessly biased. The reason is that students are not

randomly allocated to schools, but rather, gifted students tend to concentrate in

some schools while disadvantaged students concentrate in others.

A school's value-added score represents the contribution the school makes to the

progress of its students, and hence, it is claimed that it measures school perfor-


mance more accurately, because it is better able to isolate the performance of the

school from other factors that a�ect student performance. The result is a fairer

system that is not biased against schools serving lower socio-economic communities.

School value-added scores are calculated using a statistical model which compares

the progress made by each student between assessments to the progress of other

students with the same initial level of attainment, measuring the contribution the

school makes to that progress and controlling for students' background factors. We

are particularly interested in measuring a school's contribution to student progress

between NAPLAN assessments of literacy and numeracy at Years 3, 5, 7 and 9.

At the moment, cross-sectional measures of student performance are being used

to indicate school performance. However, a school might be drastically improving

the test scores of its students, but be still achieving comparatively low scores in

the national or statewide tests when compared against statistically similar schools.

By the current de�nition of �good versus bad� amid the ranking of schools, such a

school would be classi�ed as a �bad� school. However, this may not be the case - a

more appropriate measure of school performance might be a longitudinal measure

of student improvement. Under such a de�nition, schools which produce improved

students would be justly recognised.

The education system's interest in the ranking of schools, in order to allocate funding

and identify schools which are not producing their expected results, should be based

on this longitudinal measure of improvement. In contrast, parents want to know

which school is best suited to their child. Improvement is still important, but parents

are not interested in averages, only in their individual child and whether, given

the characteristics of their child, a particular school can be expected to produce

better subsequent achievements than any other chosen school or schools. It is well

understood that the education system is based on a hierarchical model of students

in schools and then schools within the education system. These two perspectives

result in completely di�erent models and ways of analysing the problem, one at the

individual student level and the other at the school level of the hierarchical model.

A further advancement is the �contextual value-added � system which, in addition to

adjusting for the prior achievements of an individual student, also attempts to adjust

for such factors as the average prior achievement of a student's peers. Goldstein [23]


states that a value-added system takes account of the di�ering intake achievements of

students entering the school. Explicit or implicit selection procedures, for example,

would a�ect the value-added score.

Gains or value-added data incorporates the �gain� or improvement of students. How-

ever, one problem is that an increase of one unit at the higher end of the assessment

scale is not equivalent to a one unit increase among average marks. It is harder to

measure the improvement or �gain� of students who are already getting close to full

marks.

Darmawan and Keeves [16] de�ne the essence of the value-added approach as the

statistical isolation of the contribution of teachers and schools to growth in student

achievement at a given grade level.

Meyer's paper [34] de�nes two types of value-added indicators, the total and intrinsic

school performance indicators which are appropriate for purposes of school choice

and school accountability, respectively. Meyer uses a two-level model of student

achievement. The �rst level of the model captures the in�uences of student and

family characteristics on growth in student achievement.

PostTestis = θPreTestis + αStudCharis + ηs + εis (1.2.1)

where i indexes individual students and s indexes schools; PostTestis and PreTestis

represent student achievement for a given individual in consecutive grades; StudCharis

represents a set of individual and family characteristics assumed to determine growth

in student achievement (a constant term); εis captures the unobserved student level

determinants of achievement growth; θ and α are model parameters that must be

estimated and ηs is a school-level e�ect that also must be estimated. The param-

eter ηs re�ects the contribution of school s to growth in student achievement after

controlling for all student-level factors such as pre-test and student characteristics.

The second level of the model captures the school-level factors that contribute to

student achievement growth.

ηs = δ1Externals + δ2Internals + us

where ηs is the school e�ect for school s from equation (1.2.1), Externals and

Internals represent all observed school-level characteristics assumed to determine

growth in student achievement, us is the unobserved determinant of total school


performance and δ1 and δ2 are estimated parameters. The intrinsic school perfor-

mance indicator, denoted φs, is de�ned to be

φs = δ2Internals + us = ηs − δ1Externals.This basic model is extended to more complicated value-added models by Meyer

[34].

1.2.4 Rasch Scaling

One way to compare scores between students over years is Rasch scaling [7, 33, 48].

Rasch scaling is a well-established method used in many human sciences, particularly

psychometrics, and increasingly in the health profession. In the context of education,

the objective of Rasch scaling is that measures of education variables like results or

scores should have a general meaning independent of the species of instrument (for

example, tests and teachers) used to obtain them so that they are comparable [3].

The Rasch model was �rst introduced by George Rasch in the 1960s. To motivate

the mathematical theory of the Rasch model, suppose there is a single true/false

question - what is the probability that it is answered correctly? The probability that

the question is answered correctly has a Bernoulli distribution but is also dependent

on the level of di�culty of the question and the person's ability. The mathematical

formulation for this probability is

ln

[Pr(xsi = 1)

(1− Pr(xsi = 1))

]= θs − βi (1.2.2)

where Pr(Xsi = 1) is the probability of person s answering item i correctly, θs is

the ability of person s and βi is the di�culty of item i. This formula says that

the di�erence between a person's underlying ability θs and the item's di�culty βi

determines the log-odds of a person answering the item correctly. The parameters

θs and βi are estimated from the true/false response data. Equation (1.2.2) can then

be algebraically rearranged to give an expression for the probability

Pr(xsi = 1|θs, βi) =exp(θs − βi)

1 + exp(θs − βi).


From this probability, the Binomial likelihood function is

L(xs1, xs2, . . . , xsi|θs, βi) =I∏i=1

P (θs, βi)xsi(1− P (θs, βi))

(1−xsi)

where P (θs, βi) denotes Pr(xsi = 1|θs, βi). To estimate the ability of students (θs)

and the item di�culty (βi), an iterative process of joint and conditional maximum

likelihood estimation is used. One such method is the Expectation-Maximisation

(EM) algorithm, an iterative method for �nding maximum likelihood estimates of

parameters in statistical models, where the model depends on unobserved latent

variables. One very important point about Rasch scaling is that the sum of the

estimated item di�culty parameters is zero [7] and this deals with the spare degree

of freedom when �tting the Rasch model. The ability of students is the latent

variable and the Rasch model is based on a log-linear simple item response model

where tests of �t and item parameter estimation can take place without assumptions

about the distribution of the latent variable [12].

In practice, the Rasch model is applied using computer software. A selection of these

Rasch measurement programs are Winsteps, RUMM, Facets, Quest and ConQuest

[42].

With these probabilities Pr(xsi = 1|θs, βi), the expected score (Es) for a test of I

items can be calculated from the predictions made by the Rasch model

Es =I∑i=1

Pr(xsi = 1|θs, βi).

Stemming from the example of answering a single true/false question, all of the above

mathematical formulation is for the dichotomous Rasch model. The dichotomous

model can be extended to the non-dichotomous model, also known as the polytomous

or partial credit model. Rather than having only two possible answers for a question

(for example, true/false, multiple choice questions), there is now a range of marks

which can be awarded to a single question. Expected scores are now calculated by

Esi =

mi∑k=0

Pr(xsi = k|θs, βi)

where k is between 0 and the maximum possible score for item i, mi.

The wide application of Rasch scaling in human sciences, health sciences and market

research is because of its usefulness in test design and comparability in the analysis


of tests or surveys. In the area of test design and assessing whether the questions are

appropriate for the target audience, Rasch scaling is used to identify test questions

which give insu�cient information to di�erentiate between students, for example

when a question is so easy that all students answer it correctly or at the other

extreme, no one correctly answers a very hard question. These questions do not

help in determining the ability of students or separating students based on their

ability. Rasch scaling also identi�es test questions which are �odd� in that they

are questions which the less-able students answer correctly and the students with

greater ability answer incorrectly. Such a situation could be when a question is

vaguely worded and guessing the answer has a higher probability of being correct.

Under the National Assessment Program, pilot studies with sample tests are given

to a sample of students, and from their Rasch scores, educational authorities can

assess the questions' level of di�culty and appropriateness.

Once the NAPLAN tests have been administered, scaling of the raw scores under

Rasch scaling enables the results to be benchmarked for comparability. Rasch scaling

compares tests through having common questions between years in the same grade

- this enables the tracking of a cohort's progress over time. Common questions are

also included across grades in the same year, and it is expected that Rasch scores

will increase with grade. For example, suppose the common questions are pitched

at a Year 5 level and are included in the Year 3, Year 5 and Year 7 tests. It is

expected that compared to the Year 5 students, Year 3 students will answer less

of these common questions correctly and Year 7 students will get more of these

common questions correct. Hence due to the presence of these common questions,

the natural progression of total Rasch scores is to increase with the ability of the

student, and therefore the grade of the test.

However, one key point about Rasch scaling which must be understood for correct

interpretation, is that Rasch scaling is completely relative and not absolute in any

way. Rasch scores depend on the choice of scale for the model and only have a

meaningful relative interpretation.

One feature of Rasch scaling is that it standardises and normalises student scores,

using standardised residuals and normalisation to sum to zero. This aids in the

comparison of scores across di�erent grades and years. The actual process of stan-


dardising and normalising is somewhat arbitrary, depending on the computer soft-

ware. As mentioned, there are many Rasch measurement programs [42] and these

operate mainly as �black boxes�. It is possible to investigate how each of the many

software standardises and normalises student scores through the Rasch model, but

that is beyond the scope of this literature review.

In summary, the Rasch model is used in situations where the variable of interest is

latent and is only measured indirectly, and the responses are either dichotomous or

fall into ordered categories. For this reason, Rasch scaling is used for school tests

to separate the ability of test takers and assess the quality of the test, and it is

standard practice for student results to be converted into Rasch scores.

1.2.5 The Estimation of School E�ects

The increasing public demand to hold schools accountable for their e�ect on student

outcomes lends urgency to the task of clarifying statistical issues pertaining to the

study of school e�ects. A school e�ect can be interpreted in two di�erent ways. The

term may refer to the e�ect on a student outcome of a particular policy or practice

or may be the extent to which a particular school modi�es a student's outcome - we

are mainly concerned with the latter de�nition.

Based on Willms and Raudenbush [47], Raudenbush and Willms [39] present a

statistical model that de�nes two di�erent types of school e�ect implicit in a school

accountability system: one appropriate for parents choosing schools for their children

(Type A), the other for agencies evaluating school practice (Type B). Firstly, a

statistical model for school e�ects is

Yij = µ+ Pij + Cij + Sij + eij

where Yij is the outcome for student i in school j; µ is the overall mean, Pij is

the e�ect of school practices (for example, school resources, organizational structure

and instructional leadership) on student i in school j; Cij is the contribution of

school context (for example, the mean socio-economic level of the school's students,

the unemployment rate of the community); Sij is the in�uence of measured student

background variables (for example, pre-entry aptitude or socio-economic status) and

eij is a random error term, including unmeasurable sources of a particular student's


outcome, assumed to be statistically independent of P,C and S. In this model, the

in�uence of school practice P and context C is allowed to vary across students within

a school. Technically, this means that the model can include both main e�ects of

school-level variables and interactions between school- and student-level variables.

The Type A e�ect is the discrepancy between a child i's potential outcome in school

j, say Yij(Sij, Cij, Pij, eij) and that child's potential outcomes in school j′, that

is, Yij′(Sij′ , Cij′ , Pij′ , eij′) [38]. Alternatively, Type A e�ects are used to ascertain

the expected output achievement of a particular student conditional on their own

characteristics. It is denoted as

Aij = Pij + Cij.

In contrast, the Type B e�ect is the di�erence between child i's potential outcome

in school j when school practice P ∗ij is in operation, yielding Y ∗ij(Sij, Cij, P∗ij, e

∗ij) and

that child's potential outcomes in school j when school practice Pij is in operation,

that is, yielding Yij(Sij, Cij, Pij, eij), denoted

Bij = Pij.

It can also be interpreted as a measure of those institutional characteristics which

explain di�erences between schools.

Darmawan and Keeves [16] then extend the above model to accommodate classroom

or teacher e�ects by splitting school context (C) into classroom (CC) and school

context (SC). Furthermore, school policies and practices (P ) can be divided into

identi�ed (IP ) and unidenti�ed (UP ). Identi�ed policies and practices (IP ) can be

further subdivided into malleable (MP ) and non-malleable (NP ) polices and prac-

tices. In addition to the speci�cation of Type A and Type B e�ects, Darmawan and

Keeves de�ned Type X e�ects [27] to refer to how well the students in a classroom

perform, when compared to similar students in classrooms and schools with simi-

lar contexts as well as similar non-malleable policies and practices. The remaining

e�ects after controlling for malleable policy and practices are labelled Type Z e�ects.

1.2.6 Hierarchical and Longitudinal Modelling

The hierarchy of students, classes, schools and educational authorities naturally

evokes multilevel or hierarchical modelling. In addition, the gathering of data in the


form of assessment scores over time on the same students creates the longitudinal

aspect of the data. Both hierarchical and longitudinal models are examples of linear

mixed e�ects models, and the vast extent of literature and research on these topics in

general are beyond this project. Some of the selected papers are written by Gelman

et al. [18], Harville [24, 25], Laird and Ware [30], Snijders [43] and Willms and

Raudenbush [47], to name a few.

Hierarchical and longitudinal modelling fall under the heading of multiple regression

y = Xβ + ε,

ε ∼ Nn(0, σ2In)

where y = (y1, y2, . . . , yn)′ is the response vector; X is the model matrix with typ-

ical row x′i = (x1i, x2i, . . . , xpi); β = (β1, β2, . . . , βp)′ is the vector of regression

coe�cients; ε = (ε1, ε2, . . . , εn)′ is the vector of errors; Nn represents the n-variable

multivariate-normal distribution; 0 is an n× 1 vector of zeros and In is the order-n

identity matrix. The regression coe�cients are then classi�ed into �xed or random

e�ects. A �xed e�ects model is one in which the coe�cients do not vary by group

and are constant across individuals. The random e�ects are modelled using prob-

ability distributions of a random variable. A mixed e�ects model is a model that

involves a combination of �xed and random e�ects.

Hierarchical models are either linear or generalised linear models in which the param-

eters are given a probability model. Another de�nition is that the hierarchical linear

model is a random coe�cient model with nested random coe�cients. This second-

level model has parameters of its own which are known as the hyper-parameters of

the model and are estimated from the data. Longitudinal modelling is necessary

when an array of variables have been recorded for each subject at several points in

time.

A longitudinal hierarchical linear model for estimating school e�ects, which has

been widely cited, is by Willms and Raudenbush [47]. The �rst level is a separate

regression of outcomes on student-level background variables within each school and

at each point in time:

Yijt = βjt0 + βjt1Xijt1 + . . .+ βjtK−1XijtK−1 +Rijt,

for student i (i = 1, . . . , nj) in school j (j = 1, . . . , J) at occasion t (t = 1, . . . , T )


such that Yijt is the outcome score, βjtk are within-school regression coe�cients, Rijt

are student-level residuals and there are k = 1, . . . , K−1 independent variables Xijtk

which describe the background characteristics of students. The βjt0 are estimates of

the performance for each school j at occasion t, after adjusting for the covariates in

the model.

The second level is a between-occasion model for school j to �nd the intake-adjusted

levels of performance βjt0 based on policy (P ) and context (C) variables and on ωjt,

the time of the tth observation for each school

βjt0 = θj0 + θj1ωt + θ2(Pjt − Pj) + θ3(Cjt − Cj) + Ujt.

Finally, the average e�ectiveness of each school and the variation between schools'

trend in achievement can be estimated respectively by

θj0 = Φ00 + Φ01Pj + Φ02Cj + Vj0,

θj1 = Φ10 + Vj1,

with the hyper-parameters Φ00,Φ01,Φ02 and Φ10.

1.3 Underlying Research Question

The purpose of NAPLAN is to report national and jurisdictional achievements in

literacy and numeracy as well as providing accurate and reliable measures of student

and school performance. However, educational experts question the current use and

interpretation of the published results.

�NAPLAN is perhaps one of the most signi�cant data sets on school-

ing in Australia. However, in its current form, the questions it could

potentially help answer cannot be addressed. The di�culty of comparing

schools statistically is well recognised by those charged with analysing and

reporting NAPLAN results but these professionals are restricted in what

they can (and should) report.

Further, the potential power of the NAPLAN data could be geometrically

advanced if it were included in a broader national research agenda, open

to a larger body of researchers who know what can be done with it.

16 1.4. Outline

But we cannot even begin the task of developing alternative within-school

practices intelligently until we harness our data and research capacity in

a more educationally productive manner.� [29]

This summarises the general thrust of my research objective - to apply mathemat-

ical and statistical modelling and analysis techniques to NAPLAN data, or data of

a similar nature, to investigate and determine what conclusions about school per-

formance can be drawn from such data. In other words, how can we accurately

measure a school's e�ect on student improvement?

1.4 Outline

In Chapter 2, we investigate a data set of the Basic Skills Tests and the univariate

and bivariate descriptive statistics. From a clean data set, we then apply model

selection techniques to identify the signi�cant variables and discuss the results of

simple linear regression models in Chapter 3.

Simple linear regression is extended in Chapters 4 and 5 to a hierarchical model

which is �tted using linear multilevel mixed e�ects models (Chapter 4) and a

Bayesian approach through the BUGS and Stan software (Chapter 5). Chapter

6 then discusses the validation of the model and how we have achieved a well-�tting

model which explains the relationship between student and school covariates, and

can be used for accurate prediction.

Having �t a hierarchical model, Chapter 7 looks at an initial longitudinal model

analysis before some conclusions, the practical implications of our �ndings to the

NAPLAN data in Australia and some ideas for further work that could be performed

in this area, are given in Chapter 8.

Chapter 2

Data Analysis

A subset of the South Australian Basic Skills Test data was procured which contains

54 recorded variables on the test results and background information of 49 341

student identities from 426 schools in the years 1997 to 2005. Other than the

data itself, no further information about the classi�cation of variables or the data

collection method was supplied, and all attempts to contact the data owners were

unsuccessful. To maintain the con�dentiality of the schools and their results, all

school identi�ers were anonymised and hence, we could not contact schools directly

to investigate further. Since it was not possible to go back to the data source, or

the schools themselves, when issues and questions arose regarding the data, it is in

this context that we cannot report any factual explanations.

2.1 The Basic Skills Test Data

The Basic Skills Tests are administered in Grades 3, 5 and 7 and have a Literacy

and a Numeracy component. Each component has sub-tests or aspects - Literacy

is divided into Reading (LR), Spelling (LS) and Writing (LW) while Unit (NU),

Spatial (NS) and Measurement (NM) tests make up the Numeracy component. The

total scores for Literacy and Numeracy are represented by LL and NN respectively.

Importantly, Literacy - Writing (LW) was only introduced in 2001. As a result of

the exploratory data analysis, various issues concerning missing and incomplete data

were observed.

17

18 2.2. The Variables: Background Information

A student is de�ned as the general term for a person who is formally engaged in

learning, in particular, one enrolled in a school. However for each actual test and

its associated year and grade, at least one result is recorded for only a subset of

the students. These students are de�ned to be the participants of a test or in other

words, a participant is a person for which data is recorded for a particular time in

the Basic Skills Test data set which we are considering. A test refers to a physical

Basic Skills test while the recorded numerical scores of the tests are referred to as

scores.

2.2 The Variables: Background Information

In total, there are 54 variables which can be divided into three categories - test vari-

ables, school variables and student variables (Tables 2.2.1 to 2.2.3). An explanation

of the coding of the variables is given in Appendix A.

An individual participant is identi�ed by their combined studentide and schoolno

values. Student ID numbers are speci�ed by the school and do not uniquely de�ne a

student. Hence, it is not possible to track participants by their ID numbers should

they move between schools.

Table 2.2.1: Test variablesVariable Name Descriptionprocyear calendar yearaspect literacy or numeracy aspectnocorrect raw test markstandardsc standardised score under Rasch scalinggradedyear grade of the test

Chapter 2. Data Analysis 19

Table 2.2.2: School variablesVariable Name Descriptionschoolno code number of the schoolgpokm distance from Adelaide General

Post O�ceisolation isolation indexspatial_ar MCEETYA* classi�cation for

Rurality and Remotenessstaff_metr classi�cation by DECS** sta�cap Country Areas Program***x006_enr, x005_enr, x004_enr enrolment numbers in 2006, 2005

& 2004x006_abs, x005_abs, x004_abs absentee rate in 2006, 2005 &

2004x006_beh, x005_beh, x004_beh number of behavioral incidents in

2006, 2005 & 2004x006_scrd, x005_scrd, x004_scrd number of School Cards**** in

2006, 2005 & 2004x006_mob, x005_mob, x004_mob mobility of students in 2006, 2005

& 2004x006_tch, x005_tch, x004_tch number of teachers in 2006, 2005

& 2004x006_tmob, x005_tmob, x004_tmob teacher mobility in 2006, 2005 &

2004

* Ministerial Council on Education, Employment, Training and Youth A�airs

** Department of Education and Children's Services

*** The Country Areas Program is an Australian government program which pro-

vides �nancial help to rural schools.

(https://deewr.gov.au/country-areas-program)

**** The School Card Scheme is a government initiative which provides �nancial

assistance towards educational expenses for eligible families.

(http://www.decd.sa.gov.au/goldbook/pages/school_card/schoolcard/?reFlag=1)

20 2.2. The Variables: Background Information

Table 2.2.3: Student variablesVariable Name Descriptionstudentide student ID numberatsi Aboriginal or Torres Strait Islanderlbote language background other than Englishstatus current status of student within schoolgender male or femaledate_of_bi date of birthaboriginal Aboriginal statusdisability disability statusschool_car individual School Cardoccupation parental occupation groupschool_edu parental school educationnon_school parental non-school educationp_g_gender gender of principal guardian or parentp_g_cultur cultural background of principal guardian or parentp_g_countr parental country of originp_g_nesb parental non-English speaking backgroundcountry_of country of originnesb_code non-English speaking backgroundhome_langu English as home languagecultural_b cultural backgroundvisa_sub_c Australian permanent visa number


2.3 Descriptive Statistics

2.3.1 Number of Participants in Schools

The number of recorded participants in each school ranges from a minimum of

one participant to a maximum of 671 participants. These numbers are depicted

graphically in Figure 2.3.1. The observed trend is that the number of participants

in a school tends to increase as the school ID number also increases.

●

●●●●●

●●●●●●●●●●●●

●●●●●●●●●

●●●●●●●●

●

●●●

●

●●●●●●●●●●●●●●●

●

●

●●●●●

●

●●●●●

●

●

●

●●●

●

●●●●

●

●●●●

●

●●

●●●●●●●●

●

●●●●

●

●

●●●●

●

●

●

●●

●●

●●●

●

●●●●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●●●●●

●●

●●

●●

●●

●

●

●

●

●●

●

●●●●●

●●

●●

●

●

●

●●

●●

●

●

●

●●

●

●

●●

●

●

●

●●

●●

●

●●●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●●

●

●●

●

●

●

●

●●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●

●●●

●

●

●

●

●

●●

●

●

●

●●

●●●

●

●

●

●●

●

●

●

●

●

●●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●●●●

●

●●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

0

200

400

600

0 200 400 600

School Number

Num

ber

of P

arti

cipa

nts

Figure 2.3.1: Number of participants in each school.

2.3.2 Univariate Statistics

To avoid confusion, LL, LR, LS, LW, NN, NM, NU and NS are called the aspects

of the test and a participant is de�ned to have a result for at least one Literacy or

Numeracy aspect for a given student, year and grade.

22 2.3. Descriptive Statistics

From Table 2.3.1, we observe that the majority of the participants in the data set

are from the calendar years 2000 to 2004 with the two standout years being 2002

and 2003. Each of these two years accounts for almost a third of the total number

of students in the entire data set. There is also over 60% of participants in Grade 3

while Grade 7 constitutes only 4.5% of the data (Table 2.3.2).

Table 2.3.1: Number and percentage of participants in each calendar year(procyear)

procyear Number Percentage1997 7 0.011998 8 0.011999 73 0.112000 9 282 13.392001 9 180 13.242002 19 654 28.342003 21 638 31.202004 9 489 13.682005 13 0.02

Table 2.3.2: Number and percentage of participants in each grade (gradedyear)

gradedyear Number Percentage3 42 375 61.115 23 826 34.367 3 143 4.53

A further break down of the participants into grades in each year is given in Table

2.3.3. We note again the concentration of the data in certain grades and years.

Drop-o� in participants in 2004

An observation is that there is a dramatic drop-o� in the number of participants in

2004, and one possible reason could be the selection of student participation in a

school. Another suggested reason for the drop in numbers could be because there

are less schools involved in the Basic Skills tests, and hence, less students tested.

However, the number of schools in 2004 is very similar to the number of schools in


Table 2.3.3: Number of participants for each calendar year and grade

procyear gradedyear Number1997 3 61997 5 11998 3 71998 5 11999 3 681999 5 52000 3 9 2722000 5 102001 3 9 1252001 5 522001 7 32002 3 9 7402002 5 9 9062002 7 82003 3 10 9662003 5 10 6442003 7 282004 3 3 1842004 5 3 2042004 7 3 1012005 3 72005 5 32005 7 3

the years 2000 to 2003 (Table 2.3.4). Are all schools giving less Basic Skills tests

in 2004 to cause the number of participants to drop evenly at the same rate or did

some schools dramatically drop while other schools stayed constant? The ratio of

participants in 2004 compared to 2003 across all grades for schools is given in Figure

2.3.2. The unimodal, rather than bimodal, feature of the plot seems to indicate that

the decrease in participants is similar over all schools. The few schools which have

a high ratio of participants all have very few participants in 2003 and is presumably

a result of quite di�erent numbers of students in the respective grades in di�erent

years.

Since the school factor does not seem to be the cause for the decrease in participant

numbers in 2004, we continue to investigate at the school level. Looking at the years


Table 2.3.4: The number of participants and schools each year

Year No. of Participants No. of Schools1997 7 71998 8 81999 73 622000 9 282 4052001 9 180 4122002 19 654 4152003 21 638 4162004 9 489 4192005 13 13

0

20

40

60

80

100

120

0 1 2 3 4 5 6

Ratio

Cou

nt

Figure 2.3.2: Ratio of participants in 2004 compared to 2003 for all schools in allgrades.

which each individual school participated in the Basic Skills Tests, we notice that

there are many schools who participate in the block of years from 2000 to 2004.

However, thirteen out of the 426 schools do not participate in consecutive years but

skip at least one year.


Test scores

The main variables of interest - the measured variables - are the participants' scores

for the test aspects. Table 2.3.5 gives the mean, standard deviation, median, in-

terquartile range and observed number for each of the Literacy and Numeracy as-

pects using both the raw and Rasch scores. Note that the mean raw scores vary

signi�cantly since the total available marks vary depending on the aspect and the

Basic Skills test itself. The mean Rasch scores are all similar and comparable since

they have been standardised and normalised under the Rasch model - see Section

1.2.4 for a discussion of Rasch scaling. When comparing the mean to the median

for each of the aspects, all are similar except for LW.

The histogram of LW (Figure 2.3.3) exhibits distinct right skewness and is possibly

bimodal, compared to other Literacy aspects. Looking at the Rasch scores, LW has

a smaller interquartile range of 9.32 compared to the other aspects which range from

10.76 to 13.21. This feature can be observed in Figure 2.3.4.

Table 2.3.5: Descriptive statistics for the raw scores and the Rasch scores for Literacyand Numeracy aspects

Mean Std Deviation Median IQR Number ObservedLL 46.76 16.08 47.00 23.00 59 869LR 23.90 8.75 25.00 12.00 59 949LS 15.63 6.74 15.00 9.00 59 764LW 21.74 14.20 16.00 23.00 48 886NM 8.38 4.02 8.00 5.00 59 947NN 27.36 9.69 27.00 13.00 60 078NS 7.57 2.85 7.00 3.00 59 912NU 11.44 4.42 11.00 6.00 59 944LL Rasch 51.78 7.76 51.98 10.76 59 869LR Rasch 51.89 8.45 52.21 11.45 59 949LS Rasch 52.19 9.45 51.80 12.02 59 764LW Rasch 52.80 8.75 52.95 9.32 48 886NM Rasch 53.67 10.69 53.30 12.73 59 947NN Rasch 53.23 9.31 53.16 12.24 60 078NS Rasch 52.99 10.63 52.89 12.80 59 912NU Rasch 53.54 10.47 53.44 13.21 59 944


0

1000

2000

3000

4000

5000

0 20 40 60 80 100

LL

Cou

nt

0

1000

2000

3000

4000

5000

0 10 20 30 40 50

LR

Cou

nt

0

1000

2000

3000

4000

5000

6000

0 10 20 30

LS

Cou

nt

0

1000

2000

3000

4000

5000

6000

0 20 40 60

LW

Cou

nt

Figure 2.3.3: Histograms of Literacy raw scores over all years and grades.


0

2000

4000

6000

8000

20 40 60 80

LL Rasch

Cou

nt

0

2000

4000

6000

20 40 60 80

LR Rasch

Cou

nt

0

2000

4000

6000

8000

20 40 60 80 100

LS Rasch

Cou

nt

0

2000

4000

6000

8000

0 20 40 60 80 100

LW Rasch

Cou

nt

Figure 2.3.4: Histograms of Literacy Rasch scores over all years and grades.


Cohort Analysis

Students who remain in the school system usually progress from Grade 3 to Grade

5 in two years time and similarly to Grade 7. This progression enables cohorts of

participants to be followed and their scores compared. From the data set, a total of

11 cohorts are de�ned by the years and grades in which participants sat Basic Skills

tests (Table 2.3.6), ordered according to student birth year.

Table 2.3.6: Coding for cohort numbers

Cohort No. Grade 3 Grade 5 Grade 71 - 1997 -2 - 1998 -3 1997 1999 20014 1998 2000 20025 1999 2001 20036 2000 2002 20047 2001 2003 20058 2002 2004 -9 2003 2005 -10 2004 - -11 2005 - -

Table 2.3.7 gives further information on the number of participants in each cohort.

To recap, a participant is a person for which at least one Literacy or Numeracy score

has been recorded for a particular test in the Basic Skills data set. In order to draw

useful conclusions from the statistical analysis of the cohorts' results, we require a

reasonable cohort size at each grade level. We assume that a sample size greater

than one hundred participants is adequate. Hence, cohort 6 is a �three-grade� cohort,

cohorts 7 and 8 are �two-grade� cohorts and cohorts 9 and 10 only have adequate

numbers at the Grade 3 level (these are in bold in Table 2.3.7). All other cohorts

are considered to have insu�cient data.

It is important to note that cohorts are de�ned to contain participants who took

tests in Grade 3, 5 and 7 in the appropriate years. Cohorts are not restrained to

only include participants which took tests in all three grades. Some participants

may only have been recorded for two of the three tests but are still included in

the cohort. If this restriction is imposed, the cohort size would drastically drop,


Table 2.3.7: Number of participants in each cohort (bold entries indicate cohortswith reasonable numbers of participants)

Cohort procyear gradedyear Number1 1997 5 12 1998 5 13 1997 3 63 1999 5 53 2001 7 34 1998 3 74 2000 5 104 2002 7 85 1999 3 685 2001 5 525 2003 7 286 2000 3 9 2726 2002 5 9 9066 2004 7 3 1017 2001 3 9 1257 2003 5 10 6447 2005 7 38 2002 3 9 7408 2004 5 3 2049 2003 3 10 9669 2005 5 310 2004 3 3 18411 2005 3 7

especially as we cannot track participants by their ID when they change schools.

Statistical conclusions about a cohort's performance would be stronger if it only

contains participants with all three tests, but with the lack of data, we decide to

resort to the looser de�nition of a cohort.

The performance of participants in each grade, cohort and aspect is one of the points

which we wish to consider and compare. The distributions of the Literacy and Nu-

meracy scores are depicted in Figures 2.3.5 to 2.3.10. It can be seen that the shape,

spread and location of the aspects' raw scores vary greatly (as previously identi�ed

from the summary statistics), but the Rasch scores have all been standardised and

normalised. These plots clearly show the e�ect of Rasch scaling on the raw scores.


●●●●●●●●●●● ●●●●●●●●●●●●●●●●● ●●●●●●●

8302 8207 8677 8912 2657

●

●

●●●●●●●●●●●

●

●●●

●

●

●

●●●

●

●●●

●

●●●●●●

●

●

●●●

●

●

●

●●●●●●

●●

●

●●

●●

●

●●●

●●●

●

●

●

●

●

●●

●●

●●

●●●

●

●●

●

●

●●

●

●

●

●●●

●●●

●

●

●●●●●

●●

●●●●●

●●

●●●●●

●

●

●

●●

●

●●●

●

●●

●

●

●●

●

●●

●●

●●●●●

●●●●

●●

●●●

●●●●●●●●●●●●●●●

●

●

●

●●●●●●●●●●●●●●●●●●●●●●

●

●●●

●

●●

●

●●●●●●●●●●●●●

●

●●●

●

●

●

●

●

●

●

●

●

●●●●●●

●

●

●●

●

●●●

●●

●●●

●

●

●

●●

●

●●●

●

●●●●

●●

●

●●

●●●●

●●●●●●●●●●●●●●

●

●●●●●●

●

● ●

●●●

●●●

●●●●●

●

●●●●●

●●

●●●●

●●

●

●●

●

●●

●

●●

●●●●

●

●

●

●●●●●

●

●●●●●●●

●

●

●

●●

●

●

●

●

●

●

8302 8207 8677 8912 2657

8300 8071 8674 9056 2621

●●●

●

●●●

●

●

●●

●●●●

●

●

●●●●●●

●●

●

●

●●●●●●

●●

●●

●

●●●

●

●

●●●●●

●

●

●●

●

●

●●

●

●

●

●

●

●

●●

●●●●

●●●●

●

●

●

●

●●

●●

●

●

●●●

●

●●

●

●●

●

●●●

●

●●●

●

●●●●●

●●●●

●●

●●

●

●●

●●●

●

●●

●●

●●

●●●●

●

●●

●●

●

●●●

●●

●●

●●

●

●

●

●●

●

●

●

●●

●●●●

●●

●

●

●●●●

●

●

●

●

●●●●●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●●●

●●●●

●

●

●

●

●

●

●

●

●

●●

●

●●●

●

●●

●●

●

●●

●

●

●●

●●●●

●

●●●●●●●●●●

●●

●●●

●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●

●●●

●●●●●●

●●

●●

●

●●●●

●●●●

●

●

●

●●●

●

●●

●

●

●●

●●●

●

●

●●

●●●

●●

●●

●●●●●●

●

●●●●●●●●●●●●●

●

●●●●●●●●●●●●

●

●

●

●

●

●

●

●

●

●●

●

●

●●●●●●●

●

●●●

●

●●●●●

●

●●●●●●

●

●●●

●

●●●●●

●

●

●●

●

●

●●●

●

●

●●●●●●

●

●

●●

●

●●

●●●●●●

●

●

●●

●

●

●●●

●

●

●

●

●

●

●

●

●

●●

●●●●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

8300 8071 8674 9056 2621

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

8330 8070 8687 8937 2639

●

●●●●●

●

●●●

●●

●

●

●●●●

●

●

●

●●●●●●

●●●

●●●

●

●

●

●●●

●●

●

●

●●●●

●

●●

●

●●●●

●

●

●

●●●●

●

●●

●

●●

●

●●●

●●

●

●

●●●●

●●●

●

●●●

●●●

●

●

●●●

●

●●

●

●●●●●

●●●●●

●

●●●●●

●

●●

●

●●●●●

●●●●●

●●●●

●●

●

●

●

●

●

●

●●●

●●

●●●●

●

●

●

●

●●

●●

●

●●

●

●●

●●

●●●

●●

●

●

●●●●

●

●●

●

●

●●●●●●

●●●●●●

●●●●

●●●●

●●●●●●●●●●

●

●●●

●●

●●●●●

●●

●●●●●●●

●

●●

●

●●●●●

●

●●

●●

●●

●

●●

●

●●●●●●●●

●

●●●●●

●

●●

●

●●●●

●●

●●●●●

●

●

●●

●

●●●●●●

●

●●●●●●●

●

●

●

●●●

●

●●●●●●●●●●●●●●●●●●●●

●

●●

●

●●●●●●●

●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●

●●●●

●

●●

●

●●

●

●●●●●●●●

●

●●●●●●●

●●●

●

●

●●

●

●●

●

●

●●

●●

●●

●●

●●

●

●

●●●●●

●

●●●

●

●●

●●●●●●●

●●●●●

●●●●●

●●

●●●

●

●

●●

●●

●

●●

●●●●●●●●

●

●

●●

●●

●●●●

●

●

●

●

●●

●

●

●●●●

●●●

●●

●●●●●

●

●

●

●

●

●●●●●●●●●

●

●●●●●●●●●●●●●●●

●

●●●●

●

●●●●●●●●●●●●●●●

●

●●●●●●●●●●●●●●●●●●●●●●●

●

●

●

●

●

●●

●

●●●●●●●

●●

●●

●●●

●

●●●●

●

●●

●●

●

●●●

●●●●

●●

●●●●●

●

●●●●●

●●●●●●●●

●●●●

●

●

●●

●

●●

●

●

●●●

●●

●●

●

●

●●●●

●●●●

●

●●●

●●●●●

●

●●

●●●●●●

●●●●

●●●●

●

●

●●

●●●

●

●●●●●●

●●

●●

●●●●

●

●●

●●

●●●●

●

●●

●●

●

●●

●●●●●●●●●●●●

●●●

●

●

●

●

●

●●

●●

●●●●●

●

●

●

●

●

●●

●

●●

●●●

●●●●

●●

●●●●

●

●●

●●●

●●●●●●●●●

●

●●

●

●●●

●

●●●●●●●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●●●●●●●●●

●

●

●

●●●●●●●●●●●

●

●●●

●

●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

8330 8070 8687 8937 2639

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●

●●●●●●●●●●●●●●●●●●●●●●●●

●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●

●●●●●●●●●●●●

●

●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●

●

●●●

●

●●●●●●●●●●●●●●●●●●

●

●●

●●

●

●

●●●●●●

●

●

●

●●●●●●

●

●●●●●●●●●●●●

●

●●●●●●

●

●●

●

●

●

●●●●

●●

●●

●●

●●

●●●●●●

●●●

7254 7693 9072 2642

●

●

●

●

●●●

●●

●

●●●●●●●●

●

●

●

●

●

●

●

●

●

●●

●●●

●●

●●●

●

●

●●●

●●

●●

●

●●●●●

●●●●●

●●

●●

●●●

●●●

●●

●

●●

●●●●●●●

●●●

●●●●

●

●●●●

●

●

●

●

●

●

●

●

●●●●●

●●●●

●●

●●

●

●

●

●

●

●

●●●●●●

●

●●

●●

●

●

●●●●

●●

●●●●●

●

●

●

●

●●●●●●●●

●

●

●

●

●

●

●

●

●●●●●●●●●●●

●●

●●

●

●●

●●●

●

●

●

●●●

●

●●

●

●

●●●●

●●●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●●●●●●●●●●

●●

●

●

●

●

●

●

●●●

●

●

●

●●

●●

●●●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●

●●

●

●●

●

●

●●●●

●

●

●●●●●●●

●●

●

●

●●

●●●

●

●

●

●●●●

●

●●

●

●

●●●

●

●●●●

●

●●●

●

●

●●●●●●●

●

●

●●●

●

●●

●

●

●

●●

●

●

●●

●●

●

●

●●●

●

●●

●

●

●

●●●●

●

●●

●

●

●●●

●

●

●●

●

●

●

●

●●

●●

●●

●

●

●●●●

●

●

●

●●

●

●●●

●

●

●●

●

●●

●●

●●●●●●●

●

●●●●●●●

●

●●●

●

●

●

●●

●●

●●●

●

●

●

●●

●

●●●●

●

●

●

●

●

●●●●

●

●●●●●●●●●●●●●

●●●●

●

●●●

●

●

●●

●

●

●●●●●

●●

●

●●●●●●●●

●

●●●●

●

●●

●●●●●

●●

●●●●

●

●●

●

●●●

●●

●

●

●

●●

●●

●

●●

●●●

●

●

●

●

●

●●

●

●

●

●

●●●

●

●●

●

●

●

●●

●

●●

●●●●●●●●●●●●

●

●

●

●

●

●●

●

●●

●

●●

●

●

●●●●●●

●

●

●●

●

●

●

●●●●●●

●

●●

●●

●●

●

●●●●●●

●

●

●

●

●●

●●●●

●

●●●

●●●●●

●

●

●

●●●●●●●●●●●●●●●●●●●●●●●

●

●●

●

●●●

●●

●

●

●●●●

●

●

●

●

●

●

●

●●●

●●●

●

●

●●●●●●●●●

●

●●

●●●●●●

●●●●●●

●●●

●

●

●

●●●

●

●●●

●

●

●●

●

●

●●

●

●●●●●

●

●●●●●●

●

●

●

●●

●

●●●●

●

●

●

●●

●

●●●

●

●

●

●●●

●

●●●●

●●

●

●

●

●

●

●●●●●●

●

●

●●●

●●●

●

●

●●●●

●●●

●●

●●●●●

●

●

●●

●

●

●

●

●●●●●●●

●●●●●

●●●●●●●

●●

●

●●

●

●●

●

●

●●

●●

●●●●●●●

●

●

●

●

●

●●

●

●

7254 7693 9072 2642

LL LL Rasch

LR LR Rasch

LS LS Rasch

LW LW Rasch

0

20

40

60

80

100

0

20

40

60

80

100

0

20

40

60

80

100

0

20

40

60

80

100

6 7 8 9 10 6 7 8 9 10Cohort

Scor

e

Figure 2.3.5: Boxplots of Literacy scores for the cohorts in Grade 3 (the sample sizeis given above each boxplot).


●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●

8365 8113 8691 9109 2618

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●

●●●

●

●●●●

●●●●●●●●●

●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●

●●●●●●●●●●●●●●●●●●●●●●

●●

●

●●●●●

●

●●●●

●

●

●●

●●●●

●

●

●●

●●●●●●

●

●●

●●

●●

●

●●●

●

●●●●●●●

●

●●●●●●●●●●

8365 8113 8691 9109 2618

●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●● ●●

8336 8209 8683 9086 2619

●●

●

●

●

●

●

●

●●●●

●

●

●●

●

●●

●

●●

●

●

●●

●

●

●●

●

●

●

●

●

●●●

●

●

●●●

●

●

●

●●●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●●●

●●●●

●●

●

●●

●

●

●●

●●

●

●●●●●●●●

●

●●●

●●

●●●●●

●

●●●

●

●●●●●●●

●●

●●●●●

●●

●●

●

●●

●●

●●

●

●●●

●●●●●

●

●●

●

●●

●

●●●

●●

●

●

●●

●

●●●●●

●●

●●●

●

●●●●●

●

●●●

●●●●

●

●●●

●

●●●●●

●●●

●

●

●●●

●●●

●●●●●●

●

●●●

●

●●●●●●●●●●

●

●

●

●●●●●●●●●●●●●●●●

●

●●

●

●●●●●●●●

●●

●●

●

●●●●

●●●

●●

●

●●●●●●

●

●

●●●

●

●●●●●●

●

●●●

●

●●●●●●

●

●●

●

●

●

●●●●●

●

●●

●●●●●●

●

●●

●●●

●●

●

●●●●●●●●●●●●

●●

●

●●

●●

● ●●●●

●

●

●●●

●

●●●●●●

●

●●●

●

●

●●

●

●

●●●●●●●

●

●●●

●●

●●●

●

●●●●●●●●●●●

●

●

●

●●●●●●●

●

●●●●●●

●

●

●

●

●●

●

●

●

●●●●

●

●

●

●

●●

●●

●

●

●

●●●●●●●●●●●●●

●

●

●

●●●●●●

●

●●●●●●●

●

●●●

●●

●●●●●●●

●

●●●

●

●

●●●●

●

●

●●

●●●

●●●

●

●

●

●●●

●

●●●●

●●●●●

●

●●

●●●●●

●

●●

●●

●●

●

●●●

●

●●

●

●

●●

●

●

●●

●●●

●●●●

●

●

●●●

●●

●●●●

●●●●●●

●●●

●

●●●●●

●

●●●

●●●

●●●

●●●

●

●●●●●●●

●

●●●●●

●

●●●●●●●

●

●●●

●

●

●●

●●●●

●●●●

●

●

●●●●●●●

●●

●●●●●

8336 8209 8683 9086 2619

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

8386 8101 8745 9078 2611

●●●●●●

●●●●

●●

●

●

●●●●●●

●●

●

●●●●

●●●●●

●

●

●

●●

●

●●●●

●

●●●●●

●

●●●

●●

●

●●

●●●

●

●

●

●●●●●

●

●●

●●

●●●

●

●●●●●

●

●●●●●

●

●

●

●●●

●

●●●●●●

●

●●●

●

●●●

●●

●

●

●

●

●●●●●●

●

●●

●

●●●●●

●

●●●

●

●●●●

●

●●●●●●

●

●

●

●●●●●●●●●●●●●●●

●

●●

●●

●●●●●

●●

●●●●●

●●

●

●

●●●●

●

●●●●●

●●●

●●

●

●●●

●

●●●●●●●●

●

●●●●

●

●●

●●

●●●●●●

●

●●

●

●

●

●●●●

●

●

●

●

●

●●●●●

●●

●●●●●●●●

●●

●●●●

●

●

●

●

●●

●

●●

●

●●●●●

●●

●●●●

●

●

●●●

●●●●

●●●

●

●●

●

●●

●●●●

●●●●

●●●●

●●

●●●

●

●

●

●●

●●

●

●

●

●

●

●●

●

●

●

●●●

●●

●●

●

●●●

●

●●

●●●

●●●●●●

●

●

●

●●

●●●

●

●

●●

●

●

●

●

●

●●●

●

●●●

●

●

●

●

●

●

●●●●

●●

●

●

●●●●●●●●●●●

●

●●●●●●●●

●

●

●●●

●●

●●

●●●●●●

●

●●●●

●●●●

●

●

●

●●●●●●

●

●●●●●

●

●●●●●

●●●●

●●●●●

●●

●

●

●

●

●●

●●

●

●

●

●●

●●

●●

●●●●●●●●

●

●●

●

●●●●

●

●●

●●

●●●

●●●●●

●

●

●●

●

●●●●●●

●●

●●●

●

●●●●●●

●●

●

●

●

●

●

●●

●●

●●

●

●●●●

●●●

●●

●

●●●

●●

●●●●●●

●

●●●●●

●

●●●●●●●●●●●

●

●

●●●

●●●●●

●●●

●●●●●●●●●

●

●

●●●●●●

●●

●

●

●●●●●●●●●●●●●●●●●●●●

●●●●

●

●●●●●●●●

●

●●●●●

●

●

●●

●

●

●

●

●●●

●

●●●

●●●●●●●

●

●

●

●

●

●●●●

●

●

●

●●

●

●●●●

●●●●

●

●

●●●

●

●●●●

●●●●

●●●●

●●●●●●

●

●

●●●

●

●●●

●

●

●

●●●

●

●●●●●●●●

●

●

●

●●●●●

●

●●

●

●

●●●●●●

●●●●●●

●

●●

●●●●●

●

●●

●●

●

●

●

●●

●

●●●

●●●

●

●

●

●●●●

●

●

●●

●

●●●●

●

●●●●●●

●●●

●

●

●●●●●●

●●

●

●●●

●●●●●●

●

●

●

●

●●

●●●●

●●●●●●●●●●●●●●●●●

●●

●

●●

●

●●●●●●●●●●●

●●●●●

●

●●●●●●●

●

●●

●

●●●●●●●●●●●●●●●●●

●

●●●●●●

●●

●●

●

●●

●

●

●●●

●●

●

●●●●●●●●●

●

●●

●●●●

●

●●●●●●

●●●●●●●●●●

●●●●

●●

●

●

●

●●●

●

●●

●●●●●

●

●●

●

●●

●●●

●●●●●

●

●●

●

●●●

●

●●●●●●●

●

●●●●●●●

●

●●●

●

●●●●

●

●●●●

●

●●●

●

●●●

●●●●

●●

●●

●●●

●●

●●●●●●

●

●●●

●●

●●●●●●●●●●●●●●

●

●●

●

●

●

●●●●●●●●●●●●

●●●

●●●●●●●

●

●●

●●●

●

●●●

●

●

●●

●●●

●

●

●

●

●

●

●●●●

●●

●

●●

●

●●

●

●

●●●

●●

●

●

●

●●●●

●●●

●●

●

●●

●●

●

●●

●

●●

●

●

●●●●●●●●●●

●

●●●●●

●

●●●●●●●●●

●●

●

●●

●●

●●●●●

●●●●●●●●●●●●

●

●●

●●

●●

●●

●●●●●●●●

●●

●●●●●●●●● ●●●

●

●●●

●●

●●

●●●●●

●●●●

●

●●

●●

●●

●●●

●

●

●●●●●

●●

●●

●

●●●●●●●●●

●●●●

●●●

●

●●●●●●●●

●

●

8386 8101 8745 9078 2611

●●●●●●●●●●●● ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●● ●●●●●●●●●●

8350 8069 8726 9107 2612

●●

●

●●●

●

●●

●

●●●●●●●

●●

●●

●●

●●

●●

●●

●

●

●

●●

●

●●●●●

●

●

●

●●

●●

●●●●●

●

●●

●

●●●●

●

●●

●●

●●●●●

●

●●●●●●●●●●●●●●●●

●

●●●●

●

●

●

●●●

●

●●●

●

●●●●

●

●●●●●●●●●●●●●●●●●●●●●●

●●

●●●●●●

●

●●●●●●●●●●

●

●●●●●●●●●●

●●

●

●

●

●

●

●●●●

●

●●●

●

●●

●

●

●●

●

●

●

●●●●

●

●●

●

●●

●

●

●●●●●●●●

●

●

●●

●

●●●

●

●

●●●

●

●●●●

●

●●●●

●●●●●●

●

●●

●

●●

●

●●

●

●●

●

●●●

●●

●●●

●

●

●

●●

●

●●●●●●●●●●●●●●

●

●●

●

●●●●●●●●●●

●

●●●●

●

●●●●●

●●●●●●

●

●

●●●●

●●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●●

●●●●●●●●●

●

●●●●

●

●

●●

●

●●●●●●

●

●

●

●●●●●●●●●●●

●

●●

●

●

●●

●

●●●●●●

●

●●●

●●

●

●

●●●●●●

●

●●

●

●

●●●

●

●●

●●

●●

●●

●●

●

●

●

●

●

●●

●

●●

●●●

●

●

●

●●

●

●●●

●●

●

●

●●●●●

●●

●●

●

●●●●●●●●

●●●●

●●

●

●

●●●

●●

●

●

●

●●

●

●●●

●●●●

●●

●●

●●●

●●

●

●●●●●●●

●●

●

●●●●

●

●

●●●●

●

●●●●●●●●●●●●

●●

●

●

●

●

●●

●●●●●

●

●

●●●

●

●●●●●●●

●

●●

●

●

●

●

●●●

●●●

●

●●●●●

●●●

●

●●●

●●●●●

●●

●

●

●●●●●●●●

●●●●●

●●●

●

●

●

●●●●●

●●●

●●

●

●●

●

●

●●●●●

●●●●●

●●●●

●

●●●●●

●

●●●

●

●

●

●●

●●●

●●●

●●

●●

●

●

●

●●●●●

●●

●

●

●

●

●

●

●

●

●●●

●●●●●

●

●

●●

●

●

●●

●

●●

●

●

●●

●●●●●●

●

●

●

●●●●●●

●

●●●●●●●●

●

●●●●

●

●●

●

●●

●●

●●

●●

●

●●

●

●

●

●

●●

●●●●●

●

●

●

●

●●

●●●●

●

●●●●●●●●●● ●●●●●

●

●●●●●

●●

●●●●●●●●●●●●●●●●●●●●●●●●●

●

●●●●●●●●●●

●

●

●

●●●●●

●

●●

●

●●●●●

●

●●●●●●●●●●●●●●●

●

●●●●●●●●●●●●●●●●●●●●●●●●●

8350 8069 8726 9107 2612

NM NM Rasch

NN NN Rasch

NS NS Rasch

NU NU Rasch

0

20

40

60

80

100

0

20

40

60

80

100

0

20

40

60

80

100

0

20

40

60

80

100

6 7 8 9 10 6 7 8 9 10Cohort

Scor

e

Figure 2.3.6: Boxplots of Numeracy scores for the cohorts in Grade 3 (the samplesize is given above each boxplot).


●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●

8863 8889 2662

●●●●●●●●●●●●●●●●●●●●●●●

●

●●●●●●●●●●●●●

●

●●●●●

●

●●

●

●●

●

●

●●●

●

●●●●

●●

●●●

●●●

●

●●

●

●●●●●●●●

●

●●●●

●●

●

●●

●

●●●

●●●●●●●

●●

●

●

●

●●●●

●

●●●●●●●

●●●

●●●●●●●●

●

●●●●

●

●

●

●

●

●●

●

●

●●●

●●●●●●

●

●●●

●

●●●

●

●

●●●●●●

●●

●●●●●

●●●

●●●●

●

●●

●●●●

●●●●

●●

●●

●●●

●

●

●●●

●

●

●

●●●

●●●●●●●●●●●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

8863 8889 2662

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●● ●●●●●●●

8907 8912 2669

●●●

●

●●

●

●●●●●

●

●

●●●

●●●●

●

●●

●●●

●●

●●

●●

●●●●●

●●

●●●●●●

●

●●

●●●●●●●

●●●●●

●●

●●

●

●●●●●

●

●

●●

●●●●●●●

●●

●●●●●

●

●●●●●●●●●●●●●●

●

●

●●●●

●●●

●●●●●●●●●●●●●●●

●

●●

●●●

●●

●

●●

●●●●

●●

●

●●●●●●

●

●●●●●●●●

●

●●●●●

●

●

●

●●

●

●●●●

●

●●●●●

●●

●●

●●●●

●●●●●●●●●●●

●

●

●

●

●●●●●●●

●

●●

●

●●

●

●●●

●●

●

●

●

●

●●●●●●●●●

●

●●

●

●

●●●●

●

●●

●

●●

●

●

●●

●

●●

●●●

●

●

●

●●

●●●●●

●●●●

●

●

●●

●●

●

●●●

8907 8912 2669

●●●●●●●●●●●●●●●

8900 8817 2658

●●●●●

●

●●●●●●

●

●

●

●●●●●

●

●●

●

●●

●

●

●

●●●●

●●

●●●

●

●●●●●●

●●

●●

●

●●●●

●●

●●

●

●

●●●

●●●

●●●●

●

●●●

●●●●

●●●

●

●●

●

●●

●●

●●●●●

●●

●

●

●●

●

●

●●●

●

●

●

●●

●

●●●●

●●

●●●●●●

●●●●

●●●●

●

●●●●●●●●

●●●●●

●

●●●

●●

●●●●●●●●●●●●●●●●●

●

●●●●●●●●

●

●●●●●●●●●●●●●●●●●●

●

●●●●●●●●●●●●●●●

●

●●●●●●

●

●●●●●●●●●●●●●●

●

●

●

●●●●●●●●●●●●●●●●

●

●●●●●●●●●●●●●●●●●●●●●

●

●●●

●●

●●●●●●●●●●●●●●●●●

●

●●●

●

●●●●●●●●●●●●●●●●●●●●●

●

●●●●●

●

●●●●●●●

●

●●●

●●●

●●●●●●●●●

●

●●●●●●●●●●●●●●●●●

●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●

●●

●

●●●●●

●●●●

●●●●●●●

●●

●

●

●●●●●●

●

●●●●

●●

●●●

●

●●●●●

●

●●●●●●●●

●

●

●

●●●●●●●

●

●●●

●

●●●●●●●●●●●●●●●●●

●●●

●●●●●●

●

●●●●

●

●●●●●●●●●●

●

●

●●

●●

●

●●●●●●●

●

●●●

●●●●

●●

●

8900 8817 2658

●●●●●

●

●

●

●●

●

●

●●●●

●

●●●●

●

●●●●●

●●

●●●●●●●●●

●●●●●

●

●●●●●

●●●

●

●●●●●●●●●●●●●●●●●●●●●●●●

●●

●

●●

●●●

●

●●●

●●

●

●

●

●●●

●

●●●●

●●

●●●●●●

●●●

●●

●

●●

●●

●●

●●●

●●

●●●●●●●

●●●

●●●●●●●

●●

●●●●●

●●●

●●

●

●●

●●●

●

●●●●●●

●●

●

●

●●●●

●●

●●

●●●

●●●●●

●

●●

●●●

●●●

●

●

●●●●●●●

●●●●

●

●●●●●

●●●

●●

●

●●

●●

●

●●●●

●●●●●

●●●●

●

●

●●

●

●

●

●●●

●

●●●●●●●●●

●●●●●●●●

●

●●

●

●

●●●●●●●●●

●●●●●●●

●●●

●●

●

●

●

●●

●●●●●

●

●●●●●●●

●

●●●●●

●●

●●●●●●●●●●

●●●●●●

●●●●●●●●

●●●●●●

●●●●●●●●●

●

●●●

●●●●●

●

●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●

●●●

●

●●●●●●●●●●●●●●●●●●●●●●

●

●●●●●

●●

●●

●

●●●●●●●●●●●●●●●●

●

●●●

●

●●

●

●●●●●●●●

●●

●●

●●

●●●●●●●●●●●●●

●

●●

●●●●●●

●

●●●●●●●●

●

●

●

●●

●

●

●●●

●

●

●●

●

●●●●●●●●●●●●●●●

●

●

●

●

●

●

●

●●●●

●

●●●

●●●

●●●●

●

●

●●

●

●

●●

●

●

●

●

●

●●●●●

●

●●●

●

●●

●●

●●

●●●

8047 8847 2692

●

●●●

●

●

●

●

●

●

●

●

●

●

●●

●●●●

●●

●

●

●●

●●

●

●

●

●

●●

●

●

●●●

●

●

●

●●●

●●

●●●●●●●

●

●●●

●

●

●●●

●●

●●●

●●●●

●●

●

●

●

●●●

●●

●●●●●

●

●

●●

●●●

●●●

●●●

●

●●●

●

●●●●●

●●

●

●

●

●

●

●●●

●

●●●

●

●

●

●●●●

●●

●

●

●●

●

●

●

●

●

●

●●●●●

●

●●●●

●

●

●

●●●

●

●

●

●●

●●●●●●

●●

●●●●●●●

●

●●

●

●

●

●

●●

●

●

●

●●●

●

●●●●

●

●

●

●●●

●●

●●●

●●

●●●●●●●●●●●

●

●

●●●●●●●

●●

●

●●●●●

●●●●

●

●●●●●●

●

●●

●

●●

●

●

●●

●●

●

●●●

●

●●

●

●●

●●

●

●●

●

●

●●●

●

●

●●●

●●

●●

●

●●●●

●

●●●●●

●●

●●●

●●●●

●

●●●

●

●

●

●

●

●●

●

●

●●●●●●●

●●●●●

●●●●●●●

●

●

●

●

●

●

●●

●●

●

●●●●

●

●●●

●●●●●●●●●●●

●

●●

●

●●●

●

●●●

●●●●●

●●●●●●●●●●

●

●

●

●●●●

●

●●

●●●

●●

●●●

●●

●

●●●

●

●

●●●●●

●●●●●●●●

●●●

●●●●●●●●●●

●

●

●●●●

●●●●●●●●●●●

●

●

●

●

●●

●

●●●

●●●●●●

●●●

●

●●●●

●●●●●●●●●●●●●

●●

●

●

●●●●

●●●

●

●●●

●

●●

●

●●●●●●●●●●●●

●●●

●

●

●

●

●

●●●●

●●

●

●●●●●●

●

●

●

●

●●●●

●●●

●

●

●

●

●●●●●●●

●

●●●●●

●●

●●

●●

●

●

●●

●●

●

●

●●●

●●●

●

●●

●

●●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●●

●

●●

●

●

●●●

●

●

●

●

●●

●●●

●●●

●●●●

●●●

●●

●

●

●

●

●

●

●

●

●●●

●●

●

●●●

●●

●

●

●

●●●

●●●●●

●●

●●

●

●●

●

●

●●●

●

●●●●●

●

●

●●

●

●●●●●●●

●●

●●●

●

●

●

●

●

●●

●

●

●●

●●

●

●

●

●

●

●

●

●

●●●●●

●●

●●●

8047 8847 2692

LL LL Rasch

LR LR Rasch

LS LS Rasch

LW LW Rasch

0

20

40

60

80

100

0

20

40

60

80

100

0

20

40

60

80

100

0

20

40

60

80

100

6 7 8 6 7 8Cohort

Scor

e



●●●●●●●●●

8879 8812 2639

●

●

●●

●●●

●

●●

●●●

●

●

●

●●

●

●●

●

●

●

●●●

●●

●

●

●

●

●

●

●

●

●

●●●●

●

●

●

●●

●

●

●

●

●

●●●

●●

●●

●●●

●●●

●●

●●●

●

●

●

●●

●●

●●

●●

●●

●

●

●●

●

●

●●●

●

●●●

●

●●●●

●

●

●

●

●●

●●●

●●

●●●●

●

●

●●●

●

●●

●●●●●●●

●

●●

●●

●●

●

●

●●●●●

●

●●

●●●●●

●●

●

●●

●●●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●●●●

●

●●●●

●

●●

●

●●●●●

●

●

●●

●●

●●

●●●●●●●●●●●●●

●●

●

●

●

●

●

●

●

●

●●●

●

●●●

●

●

●●

●

●

●

●

●●

●

●●

●●

●●●

●

●●●●

●●●

●●●

●

●

●

●●●

●●

●●

●

●

●●●●●●●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●●●●

●

●

●

●

●

●

●

●

●●●●

●●●●

●

●

●●

●●●●

●

●●●●●●●

●●

●●●●●

●

●●●●●●●

●●

●

●●●●●●●●●

●●

●●

●●●

●

●

●

●●

●●●●●●●

●

●●

●

●●

●●

●

●●

●●

●

●●

●

●●

●●

●

●●●●●●

●

●

●●●●●

●

●

●

●

●●

●●●

●●

●

●

●●●

●

●●

●●

●●●●●●●●●●●●●●●●●

●

●

●●

●●

●●●

●●

●

●

●●●

●

●●●

●●●●

●

●

●●●●

●

●

●

●●●

●●

●●

●

●●

●●

●

●

●

●●

●

●

●●

●

●●●●●●

●

●

●

●●●●●

●●●●●●●

●

●●

●●●●●●●●●

●

●●●●●●●

●●

●

●

●●

●●●

●

●

●●

●

●

●●●

●

●

●

●●

●

●

●●●●●●●●

●

●

●

●●●●●●●

●

●

●

●●●

●

●

●

●

●

●

●

●

●●

●●

●

●●●

●

●●●●●●

●

●●

●

●

●

●●

●●

●●●●

●●

●●

●

●

●

●●

●

●

●●

●

●●

●●

●●

●●

●●●

●

●●●●●●●●

●

●●●

●

●●

●

●

●

●

●●

●

●●

●

●

8879 8812 2639

●●●●●●●●●●● ●●●●●●●●●

●

8920 8832 2671

●●●●

●

●●●●

●●

●●●●

●

●●

●

●

●

●

●

●●

●

●

●●●●

●

●●

●

●

●

●

●●●●

●●

●

●●

●●●

●

●●

●●●●

●●●●

●

●●

●●●●●●●

●

●

●●

●●●

●●●●

●

●●●

●●●

●●●

●

●

●●●●●

●

●

●

●

●●●

●

●

●

●

●●

●

●

●

●●●●●

●

●●●

●

●

●

●

●

●●

●●●●

●

●●●

●

●●

●●●

●

●

●

●●●

●●

●

●

●●

●●

●

●●●

●●

●●●●●

●

●

●

●

●

●●

●

●●●●●●●●●

●

●●●●●●●●

●●

●●●●

●

●●●●●●●

●

●●●●●●●●●●●●●●●

●

●●●●

●

●●●

●●●●

●

●●

●

●

●●●●●●●

8920 8832 2671

●●●●●●

●●

●●●●

●

●●●●●●

●●

●●

●

●●●●●●●●●●●●

●

●●●●●●●●●●

●

●

●●●●

●●●●●●●

●

●●

●

●●●●●●●●●●

●●

●●●●

●●

●

●

●●●●●●●●●

●

●●●

●●●●

●●●

●●

●●●●

●

●●●

●●

●

●

●●●●●●●

●●●

●●

●●●

●

●

●●

●

●●

●

●●●

●

●●●●●●

●

●

●●●●●

●

●●●●●●

●●●●●●

●●

●●●●●

●●●

●

●●

●●

●●

●●●

●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

8829 8836 2609

●

●

●

●●

●●

●

●

●

●

●●●

●

●●

●●

●

●

●

●

●●

●

●

●●

●●

●

●●●

●

●

●

●●

●

●●●●●

●

●●

●●

●●●

●●

●

●●●●

●

●

●

●

●

●

●

●●●

●

●

●

●

●●

●●●

●

●

●●●

●

●

●

●

●

●●●●

●

●

●●●●●

●

●●●●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●●

●●

●●●●●●

●

●●

●

●●●

●●●●●

●●

●

●

●

●

●●

●

●

●●

●

●

●●●

●

●

●●

●●●

●●

●

●

●

●

●●●●●●

●

●

●

●●

●

●●●

●

●●

●●●●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●●

●

●

●

●●●

●

●

●●●●

●

●●

●

●●

●

●

●

●●

●

●

●

●●●●●

●

●

●

●

●

●●●●●●●

●●

●●

●

●●●●●●

●

●●●●

●●

●●

●●

●●

●●

●

●●●

●●●●

●●

●●

●

●

●

●●●

●●

●

●

●

●●

●●

●

●●

●

●

●

●●

●

●●●●

●

●

●

●

●●

●●●

●

●

●

●●●●●●

●

●●

●

●

●

●

●●

●●

●

●●●●

●●

●

●

●●●

●

●●●●●●●●●●●●

●

●●

●

●●

●

●

●●

●

●●

●

●●●●

●

●●●●●●

●

●●●●

●

●●●●●●●

●●

●

●

●

●

●●●

●●

●

●●

●

●

●

●●

●●●●●●●●●●●

●

●●●●●

●●

●●●

●

●

●

●●●●●●●●●

●●

●●●●●●●●●●●●●

●

●●●●●●●●●●●●●●●●●●●●●

●

●●●●

●●●

●●●●●●●●

●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●

●

●

●●●●● ●●

●

●●●●●●●●

●

●

●

●●●●●●

●

●

●

●●●●

●

●●●●

●

●●●●●●●●●●●●●●●●●●●

8829 8836 2609

●●●●●●●●●●●

8896 8859 2635

●

●

●

●●●●●

●●●

●

●●

●

●

●

●●●

●●●●

●

●●●

●

●

●●

●●●●●●

●●

●●

●

●

●

●

●●●

●●

●●●●●●

●●

●

●●●

●

●

●

●

●

●

●●

●

●

●

●●●

●

●

●●

●

●

●

●

●●●●●

●

●●●●●

●●●

●

●●●

●●

●●●

●●

●

●●●●●●●

●●●●●

●

●●●●

●●

●●●

●

●

●●

●

●

●

●●

●●●

●●●●●●●●●●●

●

●●●

●

●●●●●●●●

●

●

●

●●●

●

●

●●●●●

●●●●●●

●

●●●●●●●●

●

●

●

●●

●●●

●●●●●●

●

●●●●●●●●

●

●

●

●●●●●●●●●●●●●●●

●●

●●●

●

●●●●●●●●●●

●●●●●

●●●●●●●●●●●●●●

●●

●●●●●●●●●●●

●

●●●

●

●●

●

●●●●

●●

●

●●●●●

●

●●●

●●

●●

●●

●●

●

●●

●●

●

●

●●●●

●●

●

●

●

●

●●●

●●

●

●

●

●●

●

●

●●

●

●●

●

●

●●●●●●

●●●●

●●

●●●●●●

●

●●●

●

●

●●●●

●●●

●●

●

●●●●

●●

●

●●●

●

●

●

●

●●●

●●

●●

●

●

●●

●

●●

●●

●

●●●●

●●●●

●●

●●●●

●●

●●●●●

●●●

●

●●●●●●●●●

●●

●●●●●●●●●●

●

●

●

●●

●

●

●●●●

●

●●

●

●●●●●●●●●●●

●

●

●

●●

8896 8859 2635

NM NM Rasch

NN NN Rasch

NS NS Rasch

NU NU Rasch

0

20

40

60

80

100

0

20

40

60

80

100

0

20

40

60

80

100

0

20

40

60

80

100

6 7 8 6 7 8Cohort

Scor

e



●●●●

●

●●●●

●●●●

2526

●●●●●●●

●●●●●

●

●

●

●

●

●

●●

●

●●●●●

●

●

●●

●

●

2526

●●●●●●●

2568

●●

●

●●●

●

●

●

●

●●

●●●

●

●

2568

2551

●●●●●●●●●●●●●●●

●

●●●●●●●●●●

●

●

●●

●●●●●●●

●●

●●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●●

●●

●

●●

●

●

●

●●

●●●●

●●

●

●●●●

●●●

●

●●

2551

●●●●●●●●●●●●●

●●

●

●

2555

●

●

●

●●●●●●

●●

●●●

●

●

●

●●

●

●

●

●●

●

●

●●●●

●

●●●

●

●●●●

2555

LL LL Rasch

LR LR Rasch

LS LS Rasch

LW LW Rasch

0

20

40

60

80

100

0

20

40

60

80

100

0

20

40

60

80

100

0

20

40

60

80

100

6 6Cohort

Scor

e



2556●●●●

●

●●●●●●●●●●●●

●

●●●●●●●●●●

●

●

●

●●●●●

●

●●●●

●

●

●

●●●●●●●●●●●

2556

2554

●

●

●●●

●

●●●●●●●●●

●

●●●●●●●●●●●●

●2554

2544●

●●●●

●

●●

●

●

●

●

●●

●●

●

●●

●

●

●●

●●●●

●

●●

●

●●

●●●

●

●

●●

●

●●

●

●

●

●●●●●

●

●

●

●●

●

●

●●●

●●●●

●

2544

2516●

●

●●

●

●●

●●

●●●●●●●●●●●

●●

●●●●●●●

●

●●●

●

●

●

●●

2516

NM NM Rasch

NN NN Rasch

NS NS Rasch

NU NU Rasch

0

20

40

60

80

100

0

20

40

60

80

100

0

20

40

60

80

100

0

20

40

60

80

100

6 6Cohort

Scor

e



It is important to note that the Rasch model can result in a symmetric, bell-shaped

distribution of the scores, even when the raw scores are not symmetrically or nor-

mally distributed. An example to illustrate this is given in Figure 2.3.11 where we

look at the literacy scores for Grade 3 participants in Cohort 8. The distribution

of the LL raw scores is skewed to the left, but the Rasch scores are normally dis-

tributed. Another interesting feature is found in the plots of the LW scores (Figure

2.3.11) as an �outlier� peak is highlighted in the Rasch distribution which is not as

apparent in the plot of the raw LW scores. This could be a result of the standard-

isation and normalisation of the scores, as participants with a mark of zero cannot

be scaled by the Rasch model.


LL LL Rasch

0

200

400

600

800

1000

0 20 40 60 80 0 20 40 60 80Score

Cou

nt

LW LW Rasch

0

500

1000

1500

0 20 40 60 0 20 40 60Score

Cou

nt

Figure 2.3.11: Histograms of the distribution of raw and Rasch LL and LW Grade3 scores in Cohort 8.


The mean Literacy and Numeracy scores are plotted for each cohort and grade in

Figure 2.3.12. One observation is the increasing trend of mean Rasch marks between

sequential grades. This is caused by the test design and the use of common questions

in the Grade 3, 5 and 7 tests of a particular year. It is expected that students

who have progressed further in their education and are in higher grades, are more

likely to correctly answer the common questions, compared to students in lower

grades, resulting in a higher mean score. Through the use of Rasch scaling, these

common questions allow comparability of results across grades, as the `di�culty' of

all questions is now on a common scale. Even in some instances, the mean raw score

decreases, but the mean Rasch score increases. One example of this is Cohort 5's LS

scores - the mean raw score decreases from Grade 3 to Grade 5, but the mean Rasch

score increases. This illustrates the purpose of Rasch scaling in placing all scores

on a standard scale. At each grade level, the mean scores for the Rasch aspects are

plotted almost on top of each other, which is to be expected from Rasch scaling.

Numeracy tests have lower mean scores than Literacy tests which is due to the lower

total available marks for Numeracy aspects. The total Literacy (LL) and Numeracy

(NN) aspects clearly have larger mean scores as they are the sum of all the other

Literacy and Numeracy aspects respectively.

In a similar manner, Figures 2.3.13 and 2.3.14 compare the progression of mean

scores of each cohort from Grades 3, 5 and 7 in each aspect. This plot compares

cohorts at each grade level and in each aspect. Each cohort would have sat a

di�erent test at each grade, hence the di�erence in the raw scores. Table 2.3.8 gives

the mean Rasch aspect scores for each cohort and grade. Analysis of variance to

test for equality of means concludes that there is a di�erence in the mean scores

in both Literacy and Numeracy (all P -values are < 2e-16). Further investigation

could be done using pair-wise comparisons to identify which cohorts are statistically

signi�cantly di�erent, but the di�erences are small - less than one Rasch mark - so

not very important when compared to overall Rasch scores of 50 to 60 marks.

Note that for the purposes of reproducible research, all R [45] output is recorded

as displayed in the default R workspace and to the same number of decimal places.

For this reason and throughout this thesis, P -values are stated as given in R.


6 7 8

10

20

30

40

50

60

10

20

30

40

50

60

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

LN

3 4 5 6 7 3 4 5 6 7 3 4 5 6 7

Grade

Mea

ns

Score Type ● Raw Rasch

Aspect ● ● ● ● ● ● ● ●LL LR LS LW NM NN NS NU

Figure 2.3.12: Mean aspect score in each multi-year cohort and across grades.


●

●

●

●

●

●

●

●●

●

●

●●

●●

●●●

●

●

●●

●

●●

●●

●

●

●

●

●

●●●

LL LL Rasch

LR LR Rasch

LS LS Rasch

LW LW Rasch

102030405060

102030405060

102030405060

102030405060

3 4 5 6 7 3 4 5 6 7

Grade

Mea

ns


Cohort ● ● ● ● ●6 7 8 9 10

Figure 2.3.13: Comparing the progression of cohorts' mean scores along grades ineach Literacy aspect.


● ●●

● ●●

●●●

●

● ●

●

●

●

●●●

●●

●●●

●●

●●

● ● ●●●

●●●●

NM NM Rasch

NN NN Rasch

NS NS Rasch

NU NU Rasch

102030405060

102030405060

102030405060

102030405060

3 4 5 6 7 3 4 5 6 7

Grade

Mea

ns


Cohort ● ● ● ● ●6 7 8 9 10

Figure 2.3.14: Comparing the progression of cohorts' mean scores along grades ineach Numeracy aspect.


Table 2.3.8: Mean scores for each cohort in each grade

Cohort gradedyear LL Rasch LR Rasch LS Rasch LW Rasch6 3 48.27 48.30 48.26 -7 3 49.15 49.04 49.63 48.458 3 49.10 49.33 49.10 50.969 3 48.99 48.76 49.94 49.5210 3 49.87 50.01 50.86 50.186 5 55.87 56.23 55.64 56.227 5 55.31 55.44 56.18 55.268 5 56.68 57.09 57.53 56.986 7 61.22 61.33 61.60 61.26

Cohort gradedyear NM Rasch NN Rasch NS Rasch NU Rasch6 3 49.24 48.87 48.35 49.457 3 50.00 49.30 48.01 50.188 3 52.30 51.14 50.86 51.149 3 49.70 49.47 49.74 49.6110 3 49.27 49.05 48.86 49.326 5 57.42 57.05 56.76 57.407 5 59.26 59.19 59.38 59.498 5 59.01 58.63 59.18 58.416 7 65.46 65.48 65.83 65.44


Table 2.3.9: Number of participants by year, grade and number of tests to date.Grades are along the rows, with calendar year and the number of tests to date inthe columns.

1997 19981 2 3 4 5 1 2 3 4 5

3 6 0 0 0 0 7 0 0 0 05 1 0 0 0 0 1 0 0 0 07 0 0 0 0 0 0 0 0 0 0

1999 20001 2 3 4 5 1 2 3 4 5

3 68 0 0 0 0 9259 12 1 0 05 1 4 0 0 0 4 6 0 0 07 0 0 0 0 0 0 0 0 0 0

2001 20021 2 3 4 5 1 2 3 4 5

3 9095 30 0 0 0 9696 43 1 0 05 16 36 0 0 0 2506 7381 18 1 07 1 1 1 0 0 1 2 3 1 1

2003 20041 2 3 4 5 1 2 3 4 5

3 10935 31 0 0 0 3167 17 0 0 05 2881 7725 38 0 0 935 2254 15 0 07 16 5 7 0 0 719 631 1744 7 0

20051 2 3 4 5

3 7 0 0 0 05 3 0 0 0 07 3 0 0 0 0

2.3.3 Number of Tests: Longitudinal View

The numbers of participants at each grade level and year and how many tests have

already been taken by a participant at that point in time, including the test at

that particular time point, is given in Table 2.3.9. In cohort 6, there are only 1744

participants who sat all three grades.

It can also be observed that there are a number of participants who do not seem

to follow the normal system and sit multiple tests at the same grade level. These


participants are represented in the numbers o� the diagonal for each year in Table

2.3.9. This might be explained by the re-use of student identi�cation numbers by

schools or the event that a student re-sits a grade.

2.3.4 Test Aspects

To investigate the data further at the student level, the number of participants with

test aspects in each year and grade is given in Table 2.3.10.

The number of test aspects in each grade and year are almost equal, but there is

a large di�erence between them and the participant numbers in the corresponding

grade and year - hence, there must be missing data for some participants. One

possible reason for missing data could be a student is ill on the day of the test, and

so, no score is recorded. Data entry could be another possible cause for a missing

test score.

As the Literacy and Numeracy aspects are very di�erent from each other, we consider

them separately for a single participant. Hence in total, there are 138 688 �tests�.

Out of these, 52 898 tests have at least one missing aspect.

2.3.5 Missing Test Aspects

As noted, some participants have missing aspect scores. So the question is �Is one

aspect more likely than others to be missing?� or �Is each aspect equally likely to be

missing?�. So conditioning on the participants who have at least one missing aspect,

for each aspect, the proportion of participants missing that aspect is given in Table

2.3.11.


Table 2.3.10: Number of participants with test aspects in each year and grade

1997LL LR LS LW NM NN NS NU

3 6 5 6 0 5 5 6 65 1 1 1 0 1 1 1 17 0 0 0 0 0 0 0 0


3 7 6 6 0 5 6 7 75 1 1 1 0 1 1 1 17 0 0 0 0 0 0 0 0


3 63 58 61 0 57 55 63 625 5 5 5 0 4 4 5 57 0 0 0 0 0 0 0 0


3 8 302 8 300 8 330 0 8 365 8 336 8 386 8 3505 6 8 10 0 10 10 7 77 0 0 0 0 0 0 0 0


3 8 207 8 071 8 070 7 254 8 113 8 209 8 101 8 0695 49 47 47 44 43 49 47 467 3 3 3 3 3 3 2 3


3 8 677 8 674 8 687 7 693 8 691 8 683 8 745 8 7265 8 863 8 907 8 900 8 047 8 879 8 920 8 829 8 8967 6 7 7 7 7 7 6 6


3 8 912 9 056 8 937 9 072 9 109 9 086 9 078 9 1075 8 889 8 912 8 817 8 847 8 812 8 832 8 836 8 8597 18 20 17 19 18 17 16 18


3 2 657 2 621 2 639 2 642 2 618 2 619 2 611 2 6135 2 662 2 669 2 658 2 692 2 639 2 671 2 609 2 6357 2 526 2 568 2 551 2 555 2 556 2 554 2 544 2 516


3 6 7 7 7 7 6 7 75 1 1 1 1 1 2 2 27 2 2 3 3 3 2 3 3


Table 2.3.11: Number and proportion of participants with missing aspects

Aspect Number ProportionLL 9 475 0.226LR 9 395 0.224LS 9 580 0.228LW 11 088 0.299NN 9 266 0.221NM 9 397 0.224NS 9 432 0.225NU 9 400 0.224

Missing Literacy Writing Results

From Table 2.3.11, a consistent percentage of 22% of participants are missing scores

for LL, LR, LS, NN, NM, NS or NU, but 30% of participants do not have a recorded

LW score. In order to try and explain why the proportion of LW missing is higher

than the other aspects, Table 2.3.12 gives the number of missing LW aspects com-

pared to all participants with a missing aspect, on a yearly basis. The LW aspect

was only introduced in 2001, hence we only have data for missing LW aspects from

2001 to 2005. The average proportion is 0.297 and the proportions in Table 2.3.12

are all larger than the proportions of other missing Literacy aspects in 2001-2005.

Table 2.3.12: The number of participants with missing LW by year

Year Number Missing Total Count Proportion2001 1 879 5 761 0.332002 3 907 12 215 0.322003 3 700 13 278 0.282004 1 600 5 816 0.282005 2 7 0.29

The proportion of the number of missing LW scores out of the total number of

participants in the school is plotted for all schools, combining all data from 2001 to

2005 for each school (Figure 2.3.15).

These results suggest the increased proportion of LW missing is not due to some

schools missing nearly all of the LW scores and the remaining results being around


0.0

0.2

0.4

0.6

0.8

1.0

●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●●

●

●

●●

●

●

●●

●●●

●

●

●

●

●●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●●●

●

●●

●

●

●●

●

●

●

●●●

●

●

●

●

●

●

●●

●●

●●

●●●●

●●●

●●

●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●●●

●

●●

●

●●

●

●●●

●●●●

●

●

●

●

●

●

●●●●

●

●●

●

●

●

●●

●

●

●

●●

●

●●●●

●●

●

●●

●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●

●●

●●

●

●

●●

●

●●●

●

●

●

●●●●●

●

●

●

●

●

●●

●

●●●●●●●●●●●

●

●

●

●

●●

●●

●

●

●●●

●

●●

●

●●●●●

●●

●

●●

●●

●

●●●

●

●●●

●●●●

●●●●

●

●

●

●

●

●

●●

100 200 300 400 500 600School Number

Pro

port

ion

Size of School●

●

●

●

●

●

(0,10]

(10,20]

(20,50]

(50,100]

(100,300]

(300,560]

Figure 2.3.15: Proportion of participants in each school missing LW.

the usual proportion of 0.22. Instead, there is clustering of schools around the state

average which is higher for LW compared to the other aspects.

Goodness of Fit - Binomial Model

In addition to which aspects are missing, we investigate the distribution of the

number of missing aspect scores. The number of participants with no, one, two,

three and four missing scores for Literacy and Numeracy are given in Table 2.3.13.

The Literacy scores are divided into those before 2001 (Literacy - pre 2001) and


those after 2001 (Literacy - post 2001) due to the introduction of LW in 2001, which

increases the number of recorded Literacy aspects from three to four aspects.

Table 2.3.13: The number of participants with missing scores for aspects

Number of Literacy - pre 2001 Literacy - post 2001 Numeracymissing scores0 6 796 35 343 43 6511 2 265 18 058 19 5002 277 3 497 3 2043 32 733 3694 - 2 343 2 620

Looking at the counts in Table 2.3.13, we notice that for Literacy-post 2001 and

Numeracy, the counts for having four, that is all, missing scores does not �t a

binomial distribution, but the observed outcome frequencies for having no, one, two

or three missing scores appear to follow an approximate binomial distribution. In

order to test how well the binomial distribution �ts the observed counts, we apply

a Pearson's χ2 test to measure the goodness of �t.

Table 2.3.14 compares the observed and expected counts, and we can see that for

all but the last row, the binomial distribution closely approximates the outcome

frequencies. However, the statistical conclusion of all three χ2 tests is that the

binomial distribution does not model the observed frequencies (P -values of 2.7e-10,

3.1e-271 and 5.9e-62 respectively).

Table 2.3.14: The observed and expected frequencies for the goodness-of-�t χ2 testof the Binomial model

Literacy - pre 2001 Literacy - post 2001 NumeracyNumber Observed Expected Observed Expected Observed ExpectedMissing0 6 796 6 746.8 35 343 34 449.6 43 651 43 190.91 2 265 2 341.8 18 058 19 337.5 19 500 20 215.22 277 270.9 3 497 3 618.2 3 204 3 153.93 32 10.4 733 225.7 369 164.0


2.4 Cleaning the Data: Forensic Statistics

The correct evaluation, presentation and interpretation of the data is vital, and we

describe the process of asking questions when �strange things� are observed and delv-

ing further into the layers of the data to try and statistically explain the observation

as forensically digging into the data - hence, forensic statistics.

Before any statistical analysis or even descriptive statistics can be done, the data

must be such that no obvious errors can be detected. The cleaning process involves

understanding the data, the data structure and identifying anomalies. We wish

to investigate any discrepancies, with the purpose of possibly �nding a systematic

observation which can be explained in context. We will then be able to either infer

results, given su�cient information, or exclude data. We are limited in what we can

explain and infer with con�dence because of the lack of information on reasons for

missing or incorrect data. This information would normally be supplied by those

who provided the data, but in our case, was unavailable. The individual schools

could not be approached either because all school information was de-identi�ed for

con�dentiality purposes and political reasons.

The initial analysis starts with looking at the blanks or missing data for the variables

schoolno, procyear, studentide, aspect, nocorrect, standardsc and gradedyear.

This then identi�es a group of students who do not have any recorded test scores

for both nocorrect and standardsc. However, eliminating these students from the

data set may cause misinterpretation later when we try to track students and their

tests longitudinally. The lack of scores for students could be explained if students

were exempted or not present for the tests, but they are still recorded in the school

records as being part of the school cohort but with no scores. It is concluded that

the students with no test results must be included as we can still �follow� them

through the years if they have sat other tests. More information about schools will

also be available as a result.

2.4.1 Consistency of School Data

From the table of school variables (Table 2.2.2), the values of these school variables

are checked to be the same for all students in each school and are found to be so.

50 2.4. Cleaning the Data: Forensic Statistics

2.4.2 Consistency of Student Data

As with the school data, we check for consistency across the records of all student

variables (Table 2.2.3) for each individual student. There are 136 437 students

for which their student data is consistent, but three main groups of anomalies are

identi�ed:

• ATSI changes between categories 1 and 2 within and between tests,

• LBOTE changes between categories 1 and 2 within and between tests, and

• other variables have discrepancies.

Out of the identi�ed anomalies, it is decided that changes in ATSI and LBOTE will

be de�ned to be �Inconsistent� and left in the data set for now. Similarly, among the

other variables of school_car, occupation, school_edu, non_school, p_g_gender,

p_g_cultur, p_g_countr, p_g_nesb, country_of, nesb_code, home_langu, cultural_b

and visa_sub_c, any �errors� involving parent information are also deemed not rel-

evant at this point in time. In the end, 13 participants and their data are removed

from the data set - reasons are given in Table 2.4.1.

It could also be possible that schools might recycle student ID numbers and have

more than one student with the same ID at a future point in time. This is a

possibility because in the Basic Skills data set, student identi�cation is not global,

but rather school speci�c and under the jurisdiction of the school - a student is

uniquely identi�ed by the combination of their school ID and student ID numbers.

2.4.3 Score Check

Part of the cleaning of the data involves a check of the results to make sure the sum

of the sub-aspect scores equals the recorded total score for Literacy and Numeracy.

Considering Literacy and Numeracy aspects separately, there are 50 884 participants

whose scores add up correctly, but 34 867 Literacy participants and 39 Numeracy

participants for which the added score does not equal the recorded total. We can

only consider participants for which all scores are recorded - having missing aspect

scores means that it cannot be checked whether the sum total equals the recorded


Table 2.4.1: Anomalies in student data

schoolno studentide Anomaly109 1243669 status changes from A to L

school_car changes from Y to N within a single test173 920059 disability changes from N to Y within a single test297 1257837 status changes from L to A and school_car changes

from N to Y within a single test297 858523 date_of_bi changes randomly and not in connection to

changes to aboriginal status297 969411 status changes from L to A and school_car changes

with status

361 1127557 changing date_of_bi

399 1051563 status changes between A and L426 862091 status changes from L to A

consistent with above changes are changes inschool_car (A with Y and L with N)

477 1006947 status changes from A to L alternately497 1143261 status changes from A to L within the same test546 1189293 status changes from A to L595 1116261 status changes from L to A alternately

consistent with changes in school_car (N and Y)623 1031323 disability changes within tests

total scores. So, we only consider the data for the participants with all recorded

scores.

We de�ne and categorise these errors to be

• Literacy Flag 1 ⇒ LR + LS + LW < LL,

• Literacy Flag 2 ⇒ LR + LS + LW > LL and LR + LS 6= LL,

• Literacy Flag 3 ⇒ LR + LS + LW > LL and LR + LS = LL,

• Numeracy Flag 1 ⇒ NM + NU + NS < NN, and

• Numeracy Flag 1 ⇒ NM + NU + NS > NN.

The number of participants and schools with these Literacy and Numeracy �ags are

given in Table 2.4.2.


Table 2.4.2: The number of schools and participants with anomalies in the sum ofthe scores

Test Category No. of No. ofSchools Participants

Literacy 1 (LR + LS + LW < LL) 417 18 3072 (LR+ LS + LW > LL & LR + LS 6= LL) 47 523 (LR + LS + LW > LL & LR + LS = LL) 413 16 508

Numeracy 1 (NM + NU + NS < NN) 4 42 (NM + NU + NS > NN) 33 35

The total number of schools is 426 schools, so the fact that such a large majority of

schools are committing Literacy Flags 1 and 3 is a worrying sign. With the relatively

small numbers for Literacy Flag 2 and Numeracy Flags 1 and 2, we attribute these

errors to random noise, possibly incorrect random data entry errors, and can remove

them from the data without much impact. However, the larger cohorts for Literacy

Flags 1 and 3 require further investigation.

Literacy Flag 1 only occurs in 2003 and 2004 (Table 2.4.3), while Literacy Flag 3

occurs in 2001 and 2002 (Table 2.4.4). It seems that when LW was �rst introduced

in 2001, the majority of schools did not include it in their total score. The source

of this error could have been the use of a former spreadsheet which did not take

into account the introduction of LW, since Literacy Flag 3 requires LR+LS=LL and

LR+LS+LW > LL. Then in 2003 and 2004, a structural problem or error led to the

total recorded score being larger than the sum of all the sub-aspects. This could

have been caused by an error in a replacement spreadsheet - no literature has been

found of yet to support this claim - but we were unable to identify the reason for

the error.

Table 2.4.3: The number of participants in 2003 and 2004 with Literacy Flag 1

Year Flagged Total Proportion2003 12 713 12 953 0.9812004 5 590 5 708 0.979

We now consider the distribution of the proportion of errors for each school on a

year-by-year basis to see whether schools are 100% wrong or only have a few errors


Table 2.4.4: The number of participants in 2001 and 2002 with Literacy Flag 3

Year Flagged Total Proportion2001 5 185 5 295 0.9792002 11 323 11 379 0.995

in the recorded scores for Literacy Flag 1 and Flag 3 (Figures 2.4.1 to 2.4.4).

In Figures 2.4.1 to 2.4.4, the top plot is the proportion of participants in each school

which have Literacy Flag 1 or 3, the middle plot is the proportion of the participants

in each school ordered from lowest to highest (this gives the ordered school index)

and depicted as a step function and the bottom plot is just a truncation of the

middle plot to highlight the proportions which are less than one. These plots exhibit

a certain clustering of points at discrete proportions like 0.5 or 0.75 for example.

These could be explained by a single class or teacher having identi�ed the error in

the recording of the scores and taken measures to correct and �x it. The bottom

plots of Figures 2.4.1, 2.4.2, 2.4.3 and 2.4.4 illustrate this discretisation more clearly.

We can observe from these plots that the majority of schools have 100% of their

participants with Literacy Flag 1 or 3, depending on the year.


0.5

0.6

0.7

0.8

0.9

1.0 ●●●

●

●●●●●●●●●●●●

●

●●●●●●●●●●

●

●●●●●●●●●

●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●

●

●●●●●●

●

●

●

●

●●●●●●●●●●

●

●●●●●●●●●●

●

●

●

●●

●

●●●●

●

●●

●

●●●●

●

●●●●●●

●

●●●

●

●

●

●●●●●

●●

●

●

●●●●●●●●●●

●

●●●

●

●

●

●

●●●●

●

●

●●●

●

●●

●

●

●

●●●●●●●●

●

●●●

●●●

●

●

●

●●

●

●

●

●

●

●

●●●●●●

●

●●●

●

●●●

●

●

●●

●

●

●

●

●●●

●

●

●●

●●

●

●

●

●

●●●

●●●

●●

●

●

●●

●

●●●●●

●●

●●

●●●

●●

●●

●

●●●●●●

●

●●

●

●

●●

●●

●

●●●●

●

●

●

●

●

●●●

●●

●●●●●●

●

●

●

●

●●●

●

●

●

●●

●

●●

●●●

●●

●

●●●

●

●

●●

●●●●

●

●

●

●

●

●●

●

●●●●●●

●

●●●●

●●●

●

●●●●●

●

●

●●

●

●

●●●

●

●

●●●●●●●●●

●

●

●●

●

●

●●●

●

●

●

●●

●● ●●

100 200 300 400 500 600

Literacy Flag 1 − 2003

School Number

Pro

port

ion

0.5

0.6

0.7

0.8

0.9

1.0

●

●

●●●●

●

●●●●

●●●●●●

●●●●●●●●●

●●●●●●●●●

●●●●●●●●●●

●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

100 200 300 400


Ordered School Index

Pro

port

ion

0.5

0.6

0.7

0.8

0.9

1.0

●

●

●●●●

●

●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●

50 100 150



Pro

port

ion

Figure 2.4.1: Proportion of participants in a school which have Literacy Flag 1 in2003. Middle plot is of the ordered proportions from lowest to highest, de�ning theordered school index, and the bottom plot shows the subset of the �rst 150 schools.(See text for further details.)


0.4

0.5

0.6

0.7

0.8

0.9

1.0 ●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●

●●●●●●●●●●●●●●●●●●●

●

●●

●

●●●●●●●●

●

●

●

●●●●●●

●

●●●●●●●

●

●

●●●●

●

●●●●●●●●●●●

●

●●●●●●●●●●●●●●●●

●

●●

●

●●●●

●

●●●●●●

●

●●●●●

●

●●●

●

●

●●●●●●●●

●

●●●●●●

●

●●●●●●●●●●

●

●●

●

●

●

●●

●

●●●●●●●●●●●●●●●

●

●

●

●

●●●

●

●●

●

●

●●●●●●

●

●

●

●●●

●

●●●●

●

●

●

●●●●●●●●●●

●

●●●●●●

●●

●●●●

●

●●●

●

●●●●●●●

●

●●●

●

●●●●

●

●

●●

●●●●●

●

●

●

●

●

●●

●

●●●●●●

●

●

●

●

●●

●●

●

●●●●●●●

●

●

●

●

●●●●●

●

●●●

●

●●●●●●

●

●

●

●

●

●

●

●

●●

●●●●

●

●

●

●●●

●

●●

●

●●●●●●●●●●●●●

●

●●

●

●●

●●

●

●

●●

●

●

●●●●●

●

●●

●

●●

●

●● ●●●●

●

● ●● ●

●

● ●●●●

100 200 300 400 500 600


School Number

Pro

port

ion

0.4

0.5

0.6

0.7

0.8

0.9

1.0

●

●●●●

●

●●●

●●●●

●●●●

●●●●●●●

●●●●●●●●●●●

●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

100 200 300 400



Pro

port

ion

0.4

0.5

0.6

0.7

0.8

0.9

1.0

●

●●●●

●

●●●

●●●●

●●●●

●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

20 40 60 80 100



Pro

port

ion



0.4

0.5

0.6

0.7

0.8

0.9

1.0 ●●●●●●●●●●●●●●●●●●●●●●

●

●●●

●

●●●●●●●●●●●●●

●

●●

●

●●●●●●●●●●●●●●●●●●●●●●●●●●●

●

●●●●●●

●

●●●●●●

●

●●●●●●●●●●●●●●

●

●●●●●●●●

●

●

●

●●●●●●●●●

●

●

●

●●●

●

●●●●

●

●●●

●

●●●●●●●●●●●●●●●●●

●

●

●●

●

●

●●●●●●●

●

●●

●

●●●●●●●●●●●

●

●

●

●

●

●●

●

●●●●

●

●●●●●●●

●

●●●

●

●●

●

●●●●●●●●

●●

●●●●●●

●

●●●●

●

●●●●●●●●●●●●●●●

●

●●

●

●●●●●

●

●

●●●●●●

●

●

●

●

●●●●●

●●

●●

●

●

●●●●●●●●●●●

●

●●

●

●●●●●●

●

●●

●●

●●●●

●

●

●●

●●

●●

●

●

●●●●

●

●

●

●

●●●●

●

●●●●●●●

●

●

●●

●●●●

●

●●●●●●●

●

●●

●●●●●●

●

●●

●

●●●

●

●

●

●●

●

●

●

●●●●●●●● ●●●

●●

● ●● ●●

●●●

100 200 300 400 500 600


School Number

Pro

port

ion

0.4

0.5

0.6

0.7

0.8

0.9

1.0

●

●●

●

●

●●

●●●●●●●

●●●●●●

●●●●●●●●●●●●●

●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

100 200 300 400



Pro

port

ion

0.4

0.5

0.6

0.7

0.8

0.9

1.0

●

●●

●

●

●●

●●●

●●●●

●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

20 40 60 80



Pro

port

ion



0.4

0.6

0.8

1.0 ●●●●●●●●●●●●●●

●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●

●●●●●●

●

●●●●●●●●●

●

●●●●●●●●●●●●●●●●

●

●●●●●

●

●●●●●●●●●●●●●●

●

●●●●

●

●●●●●●

●

●

●●●●●●●●

●

●●

●

●●●●●●●●●●●●●●●●●●●●●●●

●

●

●

●●●●●●●●●●●

●●

●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●

●

●●

●

●●●●

●

●●●●●●●●●●●

●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●

●

●

●●●●●

●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●

●

●●●●●●

●

●●●●●●●●●●●●●●●●●●●●●●●●●●

●

●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●

●●●●●●●

●●●●

●●●●●●

●● ●● ● ●●

100 200 300 400 500 600


School Number

Pro

port

ion

0.4

0.6

0.8

1.0

●

●

●

●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

100 200 300 400



Pro

port

ion

0.4

0.6

0.8

1.0

●

●

●

● ●● ●

●●

● ●●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ●

10 20 30 40 50 60



Pro

port

ion



For the case of Literacy Flag 1, where LR+LS+LW 6= LL, we wish to investigate

whether it is a common shift or scale in the marks or just random noise. The total

sum LR+LS+LW is plotted against the recorded total LL (Figures 2.4.5 and 2.4.6),

and the di�erence between the two quantities ranges from 2 to 12 approximately.

Note that all the data points lie below the line LL = LR+LS+LW as Literacy

Flag 1 is de�ned to be LR+LS+LW < LL. There appears to be a systematic trend

in Literacy Flag 1 due to grade and calendar year - however, this is caused by

the overplotting of the latest category on top of the previous points, so the R

functions alpha (aesthetic in ggplot2 package) and jitter (base package) are

used to assess the density and spread of the colours and evoke transparency. The

resultant conclusion is that there seems to be no identi�able trend due to grade or

calendar year.

The output of the �tted linear regression of LL versus LR+LS+LW is given in

Table 2.4.5. We see that both the intercept and slope parameters are statistically

signi�cant at the 5% signi�cance level. We then �t LL against each of the sub-

aspects separately (Table 2.4.6), and both the intercept and slope parameters are

signi�cant. From the similar magnitudes of the slope coe�cients of LR, LS and LW,

we cannot say that the discrepancy in Literacy Flag 1 is caused by the scores in any

one of the sub-aspects, LR, LS or LW.

Table 2.4.5: Linear regression output from LL vs (LR+LS+LW) for Literacy Flag1 tests

Estimate Std. Error t-value P -valueIntercept 0.3347 0.0467 7.17 < 2.2e-16LR + LS + LW 1.1236 0.0010 1 170.60 < 2.2e-16

Table 2.4.6: Linear regression output from LL vs LR, LS and LW individually forLiteracy Flag 1 tests

Estimate Std. Error t-value P -valueIntercept 0.3501 0.0471 7.43 < 2.2e-16LR 1.1265 0.0023 499.39 < 2.2e-16LS 1.1288 0.0043 260.02 < 2.2e-16LW 1.1102 0.0043 259.62 < 2.2e-16


20

40

60

80

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

● ●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●●

●

●

●

●

●

●

●●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

● ●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

● ●

●●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

● ●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●● ●

●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

● ● ●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

● ●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●●

● ●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●●

●●

●

●

●

●

●

●

● ●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●

●

●

●● ●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●●

●

●

●●

●

●

●

● ●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●●

●

●

●

●

●

●●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

● ●

●

●

●

●

●

●●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

● ●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●●

●

● ●

●

●

●

●

●

● ●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

● ●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

● ●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

● ●

●

●

●

●

●

● ●●

●

●

●

●

●

●

●

● ●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

● ●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

● ●

●

●●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

● ●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

● ●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

● ●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

● ●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

● ●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

● ●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

20 40 60 80 100

Literacy Flag 1

LL

LR

+ L

S +

LW Grade

●

●

●

3

5

7

Figure 2.4.5: Plot of LR+LS+LW versus LL for Literacy Flag 1 tests (coloursrepresent Grades 3, 5 and 7).


20

40

60

80

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

● ●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●●

●

●

●

●

●

●

●●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

● ●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

● ●

●●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

● ●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●● ●

●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

● ● ●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

● ●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●●

● ●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●●

●●

●

●

●

●

●

●

● ●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●

●

●

●● ●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●●

●

●

●●

●

●

●

● ●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●●

●

●

●

●

●

●●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

● ●

●

●

●

●

●

●●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

● ●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●●

●

● ●

●

●

●

●

●

● ●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

● ●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

● ●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

● ●

●

●

●

●

●

● ●●

●

●

●

●

●

●

●

● ●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

● ●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

● ●

●

●●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

● ●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

● ●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

● ●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

● ●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

● ●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

● ●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

20 40 60 80 100

Literacy Flag 1

LL

LR

+ L

S +

LW

Year●

●

2003

2004

Figure 2.4.6: Plot of LR+LS+LW versus LL for Literacy Flag 1 tests (coloursrepresent calendar years 2003 and 2004).


2.4.4 Inference of Data

Based on the �ndings in Section 2.4.3, we can conclude that participants with Liter-

acy Flag 3 should have included their LW score and change the LL scores to re�ect

the addition of the LW score.

For participants with multiple missing aspect scores, no imputation can be made.

However, a single missing score for a participant can be calculated and deduced from

the other corresponding raw scores, with the exception of participants who have a

missing LW score in 2001 or 2002. It is not possible to infer LW from LL in 2001

or 2002, due to the possibility of an inaccurate LL score under Literacy Flag 3. All

changes are made to the raw scores only.

2.5 Bivariate Statistics

We now consider the student and school covariates and how they a�ect the response

variables, which are the raw and Rasch scores. To assess the relationship of the

54 explanatory variables to the scores, we consider the categorical and quantitative

variables separately and apply di�erent methods.

2.5.1 Categorical Variables

In order to determine whether the mean Rasch scores are signi�cantly di�erent

within each categorical variable, a simple linear regression is �t to the response

variable (LL Rasch or NN Rasch) against the categorical variable (isolation,

spatial_ar, staff_metr, atsi, lbote, status, gender, aboriginal, disability,

school_car, occupation, school_edu, non_school, p_g_gender, p_g_nesb, nesb_code,

home_langu or visa_sub_c). All empty and �Inconsistent� entries are removed, and

the baseline group is taken to be the �rst alphabetical or numerical group for each

categorical variable.

To test whether the linear relationship between the categorical variables and LL

Rasch or NN Rasch is statistically signi�cant, analysis of variance (ANOVA) is ap-

plied to each categorical variable. The result is that p_g_nesb, the variable repre-

senting information on parental/guardian non-English speaking background, is not

62 2.5. Bivariate Statistics

signi�cant for both LL and NN Rasch, and the variable visa_sub_c is not found to

have a signi�cant linear relationship with NN Rasch (Tables 2.5.1, 2.5.2 & 2.5.3). All

other variables have an ANOVA P -value less than 0.05 and are signi�cant predictors

of LL Rasch and NN Rasch at the 5% signi�cance level.

Table 2.5.1: ANOVA for LL Rasch against p_g_nesb

Df Sum Sq Mean Sq F -value P -valuep_g_nesb 1 193.31 193.31 3.29 0.0698Residuals 39 633 2 330 172.15 58.79

Table 2.5.2: ANOVA for NN Rasch against p_g_nesb

Df Sum Sq Mean Sq F -value P -valuep_g_nesb 1 178.38 178.38 2.19 0.1387Residuals 39 698 3 230 177.28 81.37

Table 2.5.3: ANOVA for NN Rasch against visa_sub_c

Df Sum Sq Mean Sq F -value P -valuevisa_sub_c 6 238.33 39.72 1.28 0.4544Residuals 3 93.26 31.09

A χ2 test for the independence of the categorical variables is applied to each pair of

variables, and the corresponding P -values give the statistical conclusion to whether

the variables are independent or not. From a heatmap of the P -values (Figure

2.5.1), we see that at a 5% level of signi�cance, many of the variables are correlated

with each other. The shade of the colour indicates the magnitude of the P -value,

with dark purple corresponding to the P -values which are very small and close to

zero and the lightest shade of blue corresponding to P -values which are closer to

the maximum probability of one. The expanse of dark purple indicates that there

are very few pairs of uncorrelated variables - gender is independent of school_edu,

occupation and non_school; procyear is independent of isolation, staff_metr

and spatial_ar; while gradedyear is independent of staff_metr and spatial_ar.

A complete list of all the categorical variables along with their associated P -value

from a linear regression with LL Rasch or NN Rasch and the maximum di�erence


gend

er

scho

ol_

edu

occu

pati

on

non_

scho

ol

proc

year

hom

e_la

ngu

nesb

_co

de

p_g_

nesb

p_g_

gend

er

scho

ol_

car

disa

bilit

y

abor

igin

al

stat

us

atsi

lbot

e

isol

atio

n

grad

edye

ar

staf

f_m

etr

spat

ial_

ar

spatial_ar

staff_metr

gradedyear

isolation

lbote

atsi

status

aboriginal

disability

school_car

p_g_gender

p_g_nesb

nesb_code

home_langu

procyear

non_school

occupation

school_edu

gender

0 0 0 0 0.61 0 0 0 0 0 0 0 0 0 0 0 0.12 0 0

0 0 0 0 0.32 0 0 0 0 0 0 0 0 0 0 0 0.32 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.02 0 0.32 0.12

0 0 0 0 0.15 0 0 0 0 0 0 0 0 0 0 0 0.02 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0.15 0 0.32 0.61

0.77 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0.41 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0.27 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0.27 0.41 0.77 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0.4Value

Color Key

Figure 2.5.1: Heatmap of the P -values from the χ2 test for independence betweenthe categorical explanatory variables.


in the group means is given in Table 2.5.4. From this table, as before, we see that

only p_g_nesb is not signi�cant at the 5% level of signi�cance (P -value of 0.07 for

LL Rasch and 0.139 for NN Rasch). All other variables seem to be signi�cant or

in�uential in explaining the variance in the Rasch scores according to a linear model.

Looking at the Range of Means column in Table 2.5.4, the large di�erence in the

group means for procyear (11.49 for LL Rasch and 15.78 for NN Rasch) stand out as

surprising, since we would expect Rasch scaling to have resulted in very similar mean

scores across the calendar years 1997 to 2005. Boxplots of the distribution of the LL

Rasch and NN Rasch scores for all categories and variables are used to investigate

and are given in Appendix B, but we illustrate some points and observations with

selected plots. (Note that the na category represents missing data for the categorical

variable.)


Table 2.5.4: P -value and largest di�erence in group means for each of the categoricalvariables against LL Rasch and NN Rasch (ordered according to the maximumdi�erence in the group means). The extra value in brackets for procyear signify thetrimmed range of means due to small sample sizes in the years 1997-1999 and 2005.

Variable Response P -value Range of meansisolation LL Rasch < 10−16 14.24gradedyear LL Rasch < 10−16 12.24procyear LL Rasch < 10−16 11.49

(7.57)nesb_code LL Rasch < 10−16 9.12disability LL Rasch < 10−16 8.86status LL Rasch < 10−16 8.46atsi LL Rasch < 10−16 6.23aboriginal LL Rasch < 10−16 6.20school_edu LL Rasch < 10−16 5.32spatial_ar LL Rasch < 10−16 5.05non_school LL Rasch < 10−16 4.61occupation LL Rasch < 10−16 4.42school_car LL Rasch < 10−16 4.01gender LL Rasch < 10−16 2.05lbote LL Rasch < 10−16 1.70staff_metr LL Rasch < 10−16 1.39home_langu LL Rasch < 10−16 1.28p_g_gender LL Rasch 4.78e-04 0.36p_g_nesb LL Rasch 0.0698 0.25isolation NN Rasch < 10−16 16.79gradedyear NN Rasch < 10−16 15.80procyear NN Rasch < 10−16 15.78

(8.78)status NN Rasch < 10−16 12.96nesb_code NN Rasch < 10−16 10.97disability NN Rasch < 10−16 9.52atsi NN Rasch < 10−16 7.52aboriginal NN Rasch < 10−16 7.24spatial_ar NN Rasch < 10−16 5.78school_edu NN Rasch < 10−16 5.17school_car NN Rasch < 10−16 4.94occupation NN Rasch < 10−16 4.90non_school NN Rasch < 10−16 4.84lbote NN Rasch < 10−16 2.22home_langu NN Rasch < 10−16 1.64staff_metr NN Rasch < 10−16 1.04gender NN Rasch < 10−16 0.95p_g_gender NN Rasch 1.65e-04 0.46p_g_nesb NN Rasch 0.139 0.24


Figure 2.5.3 shows that the large range of means is dependent on the sample size of

results. The years with a reasonable sample size are from 2000 to 2004, and so we

report the trimmed range of means to be 7.57, the di�erence between the mean LL

Rasch score in 2004 and 2000. From this plot, we also observe that in the years 2000

to 2004, there is a trend of increasing mean LL Rasch scores. It could be supposed,

just from this observation and plot, that the increasing mean LL Rasch scores is

due to an improving education system over time. However, these samples include

participants from all grades (3, 5 and 7) in a given calendar year. We know that due

to test design, it is expected that participants in higher grades will have a higher

mean Rasch score. So including participants in all grades will increase the mean

Rasch score in the later years, as there are more recorded Grade 5 and 7 students

in 2002, 2003 and 2004 (Section 2.3.2). When Figure 2.5.3 is divided into grades

(Figure 2.5.6), we observe that the range of trimmed means is 1.6 Rasch marks in

Grade 3 and 0.81 Rasch marks in Grade 5. We conclude then that the increase in

students in Grades 5 and 7 and the comparison of students at a lower grade with

students at a higher grade is the cause for the observed increase in mean LL Rasch

scores. A similar observation and explanation applies to NN Rasch scores as well.

Figure 2.5.4 con�rms that there is a di�erence of approximately six Rasch marks

on average between students separated by two years of education. We take this as

a benchmark to compare all di�erences in mean Rasch scores - six Rasch marks is

equivalent to, or worth, two years of education. Using this, we can then say that

having a disability can be equivalent to almost three years of education (Figure

2.5.2).

Looking at the boxplots of gender (Figure 2.5.5), we observe that males have a

higher NN Rasch score on average compared to females (53.7 compared to 52.75),

but females perform better at Literacy on average (52.75 compared to 50.7). The

observed di�erence in Numeracy between genders is less than the observed di�erence

in Literacy, and it would appear that the di�erence in Numeracy is worth approxi-

mately 4 months of education while the di�erence in Literacy is worth approximately

8 months of education.

It is also important to note that the extent of the missing data in each of these

variables and �gures (Figures 2.5.2 to 2.5.6), can be substantial - for example, ap-


proximately a third of the students have `unknown' gender. This could bias the

estimated di�erence between the categories of each variable.

●

●

●

●

●

●●

●

●

●

●●●●●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●

●

●

●

●●●●●

●●

●

●

●●●

●●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●

●

●●

●

●

●●●

●

●

●

●

●●●●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●●●●●

●●

●●●

●

●

●●

●

●

●●●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●

●

●●

●

●

●

●

●

●●

●●

●

●

●

●

●●●

●

●

●

●●

●

●

●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●●●●●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●●●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●●

●

●

●

●●●

●

●

●

●●●●

●

●●●

●

●

●●●

●●

●

●

●

●●

●●●

●

●●

●

●

●●

●

●●

●

●

●

●●●●

●

●

●●

●

●●

●

●●●

●

●

●

●

●●

37236 2401 20232

52.24 43.38 51.94

disability

20

40

60

80

100

N Y NACategories

LL

Ras

ch

Figure 2.5.2: Boxplot of LL Rasch against disability (sample size and mean aregiven above each boxplot).


●●

●

●●

●●

●

●●●●

●●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●●

●●●

●

●

●●●

●

●

●

●●●

●●

●

●●

●

●●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●●

●●

●

●●

●

●●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●●●

●

●

●●

●

●●

●●●●●●●

●

●

●

●●●

●

●

●

●●

●●

●●

●●●●●●●●●●

●

●●●●●

●

●

●

●

●

●●

●

●●●●

●●

●●●●●●

●

●

●

●

●●●

●

●

●●●●

●

●●●

●

●●●●

●●●●●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●●

●

●

●

●●●

●●

●●●●

●

●●

●

●●

●

●

●●●

●

●

●

●

●

●●

●

●●●●

●

●

●●●●●

●●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●●●●●●●

●●

●●

●

●●

●

●

●

●

●●●●●●●

●●●

●

●●

●

●●●●●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●●

●

●●

●

●

●●●

●●

●●●●●

●

●

●

●

●●

●●●

●

●

●●●

●

●●●

●

●

●●●●●●●

●●●

●

●

●●

●

●

●

●

●●

●●

●

●

●

●●

●●

●

●

●

●

●●

●

●

●

●

●●

●

●

●●●

●

●

●

●

●

●

●

●

●

●●

●

●

●●●●●

●

●●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

7 8 68 8308 8259 17546 17819 7845 9

45.7 46.8 45.4 48.27 49.16 52.52 52.15 55.84 56.89

procyear

20

40

60

80

100

1997 1998 1999 2000 2001 2002 2003 2004 2005Categories

LL

Ras

ch

Figure 2.5.3: Boxplot of LL Rasch against procyear (sample size and mean aregiven above each boxplot).

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●●

●●

●

●●

●●

●

●●●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●●●●●

●

●

●●●●

●

●

●

●

●

●

●

●

●

●●●●

●●

●

●

●

●●●●●

●

●

●●●

●

●●

●●

●●

●

●

●

●●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●●●

●

●

●

●●

●●●●

●

●

●

●

●

●

●

●●●●

●●

●

●

●●

●

●●

●

●

●

●

●

●●

●

●●●●●

●

●●

●

●●●

●●●

●

●●

●●

●●

●

●

●

●

●●

●

●

●

●

●●

●●

●

●

●●

●●●●

●

●

●

●

●

●●

●

●●●

●

●●

●

●

●

●

●

●●●●●

●

●●

●●

●

●●

●

●●●●

●

●●●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●●

●

●●

●

●

●

●●●

●

●

●

●

●

●●

●

●●

●●●●●

●

●●

●

●

●●●

●

●

●●

●

●

●

●

●

●●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●●

●

●

●●

●●

●

●●

●

●●●●

●●

●

●●●●

●

●●

●

●

●

●●●●●

●

●

●●●

●

●

●●●

●

●

●

●●●

●

●

●

●

●

●●

●

●●

●●

●

●

●●

●

●

●●

●●

●

●●●●●

●

●●

●

●

●●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●●

●

●

●●

●●

●

●

●

●●●●●●

●

●●●

●

●

●

●●

●

●

●●

●

●

●

●

●

●●●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●●

●

●●●●●

●

●

●

●

●

●

36837 20477 2555

48.94 55.72 61.18

gradedyear

20

40

60

80

100

3 5 7Categories

LL

Ras

ch

Figure 2.5.4: Boxplot of LL Rasch against gradedyear (sample size and mean aregiven above each boxplot).


●

●

●

●

●

●

●

●

●

●

●

●

●●●●●●●

●●

●●●

●

●

●

●

●

●

●●

●

●

●

●●●

●●

●●●

●●

●●

●

●

●

●●●

●

●●

●

●

●●

●●

●

●●

●

●●

●●

●

●

●●

●●

●

●

●

●

●

●●●

●

●●

●

●

●

●●

●

●

●

●

●●

●

●

●●

●

●

●●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●●

●●

●●

●

●

●

●●

●

●

●

●

●●

●

●●●●●

●●

●

●

●

●

●●●●

●

●

●

●

●

●●

●

●●

●

●

●

●●

●●

●

●

●

●

●●●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●●

●

●

●

●●●

●

●

●●

●

●

●

●●●

●

●

●

●

●

●●● ●

●●

●

●●●●●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●●●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●●

●

●

●

●●●

●

●

●

●●●●

●

●●●

●

●

●●●

●●

●

●

●

●●

●●●

●

●●

●

●

●●

●

●●

●

●

●

●●●●

●

●

●●

●

●●

●

●●●

●

●

●

●

●●

19400 20237 20232

52.75 50.7 51.94

gender

20

40

60

80

100

F M NACategories

LL

Ras

ch

●

●

●●

●●

●

●

●●●

●

●

●●

●

●

●

●●

●

●

●

●●●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●●●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●●●

●●

●

●

●

●

●

●

●

●

●●

●●

●

●

●●●

●

●

●

●●●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●●

●

●

●

●●

●

●●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●●

●●●

●

●

●●

●●

●

●

●

●

●

●●●

●●●

●

●

●●

●

●

●

●

●●●

●

●

●

●

●●●

●

●●

●

●

●

●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●

●

●

●●●

●

●

●

●

●

●

●●

●

●●

●

●

●

●●

●●

●●

●

●

●●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●●●●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●

●●

●

●

●●

●●

●●●●●●

●

●●

●

●

●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●●●

●●

●●

●●

●

●●

●

●

●●●●

●

●●

●

●

●

●

●

●

●

●

●●●●

●●●

●

●

●

●

●●●

●

●

●

●●

●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●

●●●●

●

●

●●

●

●●●●●●●

●

●●

●

●●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●

●

●

●

●

●●

●●

●

●

●

●●

●●●

●

●

●●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●●

●

●●●●●●

●

●●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

19315 20388 20375

52.75 53.7 53.22

gender

20

40

60

80

100

F M NACategories

NN

Ras

ch

Figure 2.5.5: Boxplots of LL Rasch and NN Rasch for gender (sample size and meanare given above each boxplot).


●●● ●● ●●● ●●● ●●●● ● ●●● ●● ●●●● ●● ●● ● ● ●●●● ●● ●●●● ● ●●● ●● ● ●●●● ●● ● ●● ●● ●●●● ●● ●● ●● ● ●● ●●● ●●● ●●● ●●● ● ●● ●●● ●● ●● ●●●●● ● ● ●● ●● ●● ● ●●

●●●● ●● ●●● ●●●● ●● ●● ●● ● ●●● ●●●● ●●● ● ●●●●● ●●●●● ●●●●●●●● ●● ●●● ●● ●●● ●●●● ●●●●●●●●●●● ●●●●● ●●● ● ●●● ●●●●●●● ●●●●●●● ●

● ●●● ●● ●● ●● ●● ● ●● ●●● ●● ● ●● ●●●●● ●●●● ●● ●●● ●●●● ●● ●●●● ●●● ●● ●●●●● ●● ●●● ●●●●● ● ●●●● ●●● ●●● ●●●●●● ●●●● ●● ● ●●● ●● ● ●●● ●● ● ●● ● ●● ● ●●

● ● ●●● ●● ●●● ●● ●● ● ●●●●●●● ●● ●●●● ●● ●●

67

6383

0282

0786

7789

1226

576

44.2

646

.31

45.0

248

.27

49.1

549

.148

.99

49.8

752

.33

●

●●●●● ●● ●● ● ●● ●●● ● ●● ●● ●● ●● ●●● ●●●● ● ● ●● ●●● ● ●● ●●● ●●● ●●● ● ●●●● ●●● ●●● ●●

●● ●● ●●●● ●●●●● ●● ●● ●● ●●●● ●●● ●●● ● ●● ●● ● ●●●● ●●● ● ●●●● ● ● ●● ●●●●● ● ●● ●●● ●● ●● ● ● ●●● ●● ●● ● ●● ●●●● ●●● ● ● ●● ●● ●● ●●●● ●●● ●●●● ● ●●●●● ● ●● ●●●● ● ●●●● ● ●● ●●● ●● ● ● ●●● ●

● ●●● ● ●● ●●●● ●●●● ● ●●●● ●● ●● ●●● ● ●● ●●● ● ●● ●●

11

56

4988

6388

8926

621

54.3

950

.25

50.2

552

.86

52.0

855

.87

55.3

156

.68

62.0

4

●●

●●

● ●● ●● ●● ●●●●●● ● ●● ●● ●● ●●●● ● ● ●● ● ● ●●

36

1825

262

53.6

160

.58

57.2

361

.22

68

35

7

20406080100

1997

1998

1999

2000

2001

2002

2003

2004

2005

1997

1998

1999

2000

2001

2002

2003

2004

2005

1997

1998

1999

2000

2001

2002

2003

2004

2005

Cat

egor

ies

LL Rasch

Figure

2.5.6:

BoxplotofLLRasch

against

procyearsplitinto

grades

(sam

plesize

andmeanaregivenaboveeach

boxplot).


2.5.2 Quantitative Variables

To test the association between the quantitative variables, a Pearson's correlation

test is used between each variable and LL Rasch and NN Rasch separately. All

variables but one have a P -value less than 0.05 for the correlation tests - the stand-

out variable is NN Rasch ∼ x005_tmob (P -value of 0.509).

To visually assess the strength of the correlation between the quantitative explana-

tory variables, Figure 2.5.7 is a heatmap of the correlation values. Most variables

have a positive correlation of varying strength, and the 2004, 2005 and 2006 values

for certain variables form distinguishable 3x3 blocks in the heatmap.

A complete list of all the quantitative variables along with their associated P -value

and R2 value from the linear regression with LL Rasch or NN Rasch, ordered by R2

values, is given in Table 2.5.5. The P -values are very small and indicate that there

is an association between the explanatory variables. However, the R2 values are

also very small and seem to indicate that only a small proportion - less than 5% - of

the variance in the response variables can be explained by a linear relationship with

the quantitative variables. There is a very large sample size of at least 47 500 data

points for each of the explanatory variables, possibly causing the small P -values,

but we also note that the R2 value of the insigni�cant variable x005_tmob is less

than all the other R2 values by at least two orders of magnitude.

The overall conclusion from the bivariate statistics is that at this minimal level of

statistical analysis and testing, most of the explanatory variables are deemed to be

associated with the response variables of LL Rasch and NN Rasch scores.


●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●●

●●●●

●

●

●●●●

●

●

●

●●●

●

●

●

●

●●

●●●

●

●

●

●●●●

●

●

●

●●●

●

●

●

●

●●

●●●

●

●

●

●●●

●

●

●

●

●

●

●

●●●●●

●●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●●●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●●●

●

●●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●

●

●●●●●●●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●●●●●●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●●●●●●

●

●●

●

●●

●

●

●

●●●●

●

●

●●●●●●●

●

●

●●●●

●

●

●

●●●●

●

●

●●●●●●●

●

●

●●●●

●

●

●

●●●●

●

●

●●●●●●●

●

●

●●●

●

●

●

●

●

●

●

●●●●●

●●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●●●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●●●

●

●●

●

●

●●●

●

●

●

●

●

●

●

●●●

●

●

●

●

●●

●●●

●

●

●

●●●●

●

●

●

●●●

●

●

●

●

●●

●●●●

●

●

●●●

●

●

●

●

●●●

●

●

●

●

●●

●●●●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● −1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

gpok

mx0

06_

enr

x005

_en

rx0

04_

enr

x006

_ab

sx0

05_

abs

x004

_ab

sx0

06_

beh

x005

_be

hx0

04_

beh

x006

_sc

rdx0

05_

scrd

x004

_sc

rdx0

06_

mob

x005

_m

obx0

04_

mob

x006

_tc

hx0

05_

tch

x004

_tc

hx0

06_

tmob

x005

_tm

obx0

04_

tmob

gpokmx006_enrx005_enrx004_enrx006_absx005_absx004_absx006_behx005_behx004_behx006_scrdx005_scrdx004_scrdx006_mobx005_mobx004_mobx006_tchx005_tchx004_tch

x006_tmobx005_tmobx004_tmob

Figure 2.5.7: Heatmap of the correlation between the quantitative explanatory vari-ables.


Table 2.5.5: P -value and R2 value for each of the quantitative variables against LLRasch and NN Rasch (ordered according to R2 values)

Variable Response P -value R2 (%)x004_abs LL Rasch < 10−16 4.63x004_mob LL Rasch < 10−16 4.63x006_abs LL Rasch < 10−16 4.38x006_mob LL Rasch < 10−16 4.38x005_abs LL Rasch < 10−16 4.19x005_mob LL Rasch < 10−16 4.19x006_scrd LL Rasch < 10−16 1.23x004_scrd LL Rasch < 10−16 1.13x006_enr LL Rasch < 10−16 1.01x005_scrd LL Rasch < 10−16 0.96x005_enr LL Rasch < 10−16 0.93x006_beh LL Rasch < 10−16 0.89x004_enr LL Rasch < 10−16 0.85gpokm LL Rasch < 10−16 0.81x004_beh LL Rasch < 10−16 0.60x005_beh LL Rasch < 10−16 0.23x004_tmob LL Rasch < 10−16 0.23x006_tmob LL Rasch < 10−16 0.22x006_tch LL Rasch < 10−16 0.21x005_tch LL Rasch < 10−16 0.19x004_tch LL Rasch < 10−16 0.18x005_tmob LL Rasch 0.00842 0.0133x004_abs NN Rasch < 10−16 4.6x004_mob NN Rasch < 10−16 4.61x005_abs NN Rasch < 10−16 4.25x005_mob NN Rasch < 10−16 4.25x006_abs NN Rasch < 10−16 4.13x006_mob NN Rasch < 10−16 4.13x006_scrd NN Rasch < 10−16 1.62x004_scrd NN Rasch < 10−16 1.51x005_scrd NN Rasch < 10−16 1.31x006_beh NN Rasch < 10−16 0.84x004_beh NN Rasch < 10−16 0.71x006_enr NN Rasch < 10−16 0.58gpokm NN Rasch < 10−16 0.57x005_enr NN Rasch < 10−16 0.52x004_enr NN Rasch < 10−16 0.46x005_beh NN Rasch < 10−16 0.28x006_tmob NN Rasch < 10−16 0.25x004_tmob NN Rasch < 10−16 0.21x006_tch NN Rasch 3.93e-13 < 0.1x005_tch NN Rasch 4.21e-11 < 0.1x004_tch NN Rasch 2.55e-10 < 0.1x005_tmob NN Rasch 0.509 8.32e-04


2.5.3 Principal Component Analysis (PCA)

Background Theory

Principal component analysis (PCA) is a mathematical procedure that uses an or-

thogonal transformation to convert a set of observations of possibly correlated vari-

ables into a set of values of linearly uncorrelated variables called principal compo-

nents. The number of principal components is less than or equal to the number

of original variables. This transformation is de�ned in such a way that the �rst

principal component accounts for as much of the variability in the data as possible,

and each succeeding component in turn has the highest variance possible under the

constraint that it be orthogonal to, and uncorrelated with, the preceding compo-

nents. PCA is a form of multivariate analysis, and its application can often reveal

the internal structure of the data in a way which best explains the variance in the

data. PCA can simplify high-dimensional data to one with less variables by using

the �rst few principal components, so that the dimensionality of the transformed

data is reduced.

Results

The results from the principal component analysis on the continuous explanatory

variables are given in Table 2.5.6. We see that the �rst three components explain

the majority of the variance in the covariates, as the �rst component accounts for

approximately 72.5% and by the third component, a total of 98.5% of the variation.

This result can also be seen graphically (Figure 2.5.8).

The �rst three principal components are primarily constructed from variables per-

taining to enrolment, the number of School Cards issued and the school's distance

from the Adelaide General Post O�ce. Since almost all of the variance is explained

by the �rst three principal components, we look to see if any clusters of points can

be identi�ed in a 3-dimensional scatter plot of the scores of the �rst three prin-

cipal components (Figure 2.5.9). No obvious clusters of data points are observed,

and we conclude that from the principal component analysis, we cannot reduce the

dimensionality of the data to linear combinations of fewer meaningful variables.


Figure 2.5.8: Barplot of the variances explained by the principal components.

Figure 2.5.9: 3D scatter plot of the scores of the �rst three principal components.


Table 2.5.6: The standard deviation, proportion of variance explained and the cu-mulative proportion for each of the principal components

Principal Component Standard Dev Proportion Cumulative Proportion1 385.6 0.725 0.7252 176.5 0.152 0.8773 148.7 0.108 0.9854 37.6 0.007 0.9925 26.7 0.003 0.9956 19.2 0.002 0.9977 16.1 0.001 0.9988 10.8 5.7e-4 0.99939 6.8 2.2e-4 0.999510 6.2 2e-4 0.999711 4.9 1e-4 0.999812 3.6 6.3e-5 0.999913 1.9 1.7e-5 114 1.5 1.1e-5 115 1.2 7.5e-6 116 1.1 5.6e-5 117 0.1 5.1e-8 118 0.09 4e-8 119 0.07 2.6e-8 120 5.4e-8 1.4e-20 121 0 0 122 0 0 1


2.6 Final Data Sets

Having started with the original Basic Skills data subset and through the use of

forensic statistics, the data has been cleaned. Our �ndings and actions can be

summarised in the speci�ed order:

1. remove participants with more than one missing aspect score for either a Lit-

eracy or Numeracy participant, as inadequate information is available to infer

these missing scores,

2. remove the Numeracy scores of participants with Numeracy Flags 1 and 2,

attributing these to random error,

3. remove the Literacy scores of participants with Literacy Flag 2, attributing

these to random error,

4. remove the Literacy scores of participants with Literacy Flag 1 as we found

no explanation, and there is no method to rectify this discrepancy,

5. add the LW score to the LL scores of participants with Literacy Flag 3 as the

nature of the �ag was explained (Section 2.4.3), and

6. for participants who have one missing score which is not LW in 2001 or 2002,

we infer or impute the missing scores (note that we cannot infer LW from LL

in 2001 or 2002 due to the possibility of an incorrect LL score under Literacy

Flag 3).

Figure 2.6.1 diagramatically illustrates the above process and the data sets.

The steps above discuss the corrections and imputations of the raw scores. However,

the Rasch scores are the main response variables and picked to be so because Rasch

scaling enables results to be compared across grades and years. Consider the Literacy

scores. Some Rasch scores can be corrected by �rstly imputing the single missing

raw scores and then attributing the Rasch score based on the Rasch score of students

with the same raw score. Unfortunately, an observation of the recorded scores is

that although we can correct for Literacy Flag 3 and change the corresponding LL

scores, the Rasch LL scores have been calculated on the �incorrect� LL scores, rather

78 2.6. Final Data Sets

than on the correct individual aspect scores for LR, LS and LW. This is veri�ed by

looking at the LL Rasch scores for all participants in the same grade and cohort

who received the same �incorrect� LL score - they all have the same LL Rasch

score independent of the di�ering raw scores for LR, LS and LW. So the imputed

LL Rasch scores would be as inaccurate as the recorded Rasch scores. In fact, we

cannot impute any LL Rasch scores because their calculation used original data,

and all LL Rasch scores are �tted to �incorrect� or inaccurate (apparently) data.

As a result, the Numeracy scores, after participants with Numeracy Flags 1 and

2 have been removed, is the most reliable data, and so we choose to restrict all

modelling to the Numeracy data.

After this investigation of the data, we would have liked to pursue various data

questions further with the data source, but since that was not possible, the data was

cleaned and corrected as much as possible in preparation for statistical modelling.


Redu

ced

Data

Data

Set

1Da

ta S

et 2

Orig

inal

Rem

ove

Mul

tiple

M

issin

g As

pect

s

Rem

ove

Num

erac

y Fl

ags

1 &

2

Rem

ove

Lite

racy

Flag

s 1

& 2

Corre

ct L

itera

cy F

lag

3Im

pute

Miss

ing

Mar

ks

138

688

50 8

8467

392

100

144

Figure

2.6.1:

Flowchartof

datasets.

Chapter 3

Initial Model Selection

From the principal component analysis in Chapter 2, we know that the explanatory

variables are highly correlated with each other. Unfortunately, as highlighted in

Chapter 2, we were not able to contact those responsible for collecting the data to

discuss which of the variables could be dropped based on logical reasons. With this

knowledge in the background, we will perform a simple multiple linear regression as

an initial primary assessment of the data. This will provide meaningful insight into

the physical variables themselves and will allow the data to speak for itself.

There is data recorded for 54 explanatory variables. However, we want to be able to

select the variables which are only statistically signi�cant and important in terms

of their relationship with the response variable, NN Rasch scores. We do this by

looking for missing data and whether the data values for each variable make logical

sense, before using standard statistical model selection techniques. This will then

result in the simplest and most relevant model for the Rasch scores.

3.1 Manual Reduction of Data and Predictor Vari-

ables

Looking at the data itself, certain variables and data provide either insu�cient

information, irrelevant information or exhibit discrepancies. Hence, we can remove

them from the list of possible regression variables. Each of the following paragraphs

81

82 3.1. Manual Reduction of Data and Predictor Variables

give reasons for the exclusion of variables and data.

Participants with Numeracy Flags 1 and 2 (Section 2.4.3) are removed from the

data set and similarly, participants with missing NN scores are also removed, as the

response variable is either incorrect or not recorded.

Categorical Variables

The categorical cap (Country Areas Program) variable has only one de�ned group,

`Y' (yes), and the non-entries could represent either `N' (no) or be missing data

- this information is not known. The variable status is also removed due to the

ill-de�ned categories for the status of students in the education system (Appendix

A.3). Students' date of birth (date_of_bi) is also deemed to give no further useful

information. The visa_sub_c variable is removed because the majority of values

are missing.

Certain variables on parental and cultural background are removed because of the

presence of missing data, for example, 54 217 out of 60 039 entries for nesb_code are

missing. For this reason, cultural_b, p_g_cultur, p_g_countr and country_of

are also excluded. For the recorded values which we have, these variables identify

the country of origin of a student, however, similar information on nationality can

be gleaned from the p_g_nesb variable which indicates the non-English speaking

background of the principal parent or guardian in the form of a binary variable.

Quantitative Variables

All the year-speci�c 2004-2006 variables are removed because we observed that the

x00y{4,5,6}_abs and x00y{4,5,6}_mob columns are exactly the same for all par-

ticipants. The number of School Cards for a school can also exceed the enrolment

number in a particular year, which does not seem plausible. In addition, a large

proportion of the values are missing for these variables, and data from 2004 to 2006

is not very relevant as we only have substantial numbers of recorded test data in the

years 2000 to 2004.

With the presence of missing data for school variables, schools either have all or no

data for a particular variable. With more complete data, teacher mobility would

Chapter 3. Initial Model Selection 83

be an important variable to include as a predictor in the model as it is politically

important and informative about school demography. There are some schools which

struggle to retain teachers - sometimes due to location or the student population -

and this is re�ected in higher teacher mobility. However, it is important to note that

teacher mobility is not a consistent variable over time. There is a lack of correlation

between teacher mobility in 2004, 2005 and 2006 (Figure 2.5.7), and so we cannot

use data for one year out of 2004, 2005 or 2006 to predict teacher mobility in another

of those years, let alone predict and extrapolate for teacher mobility in previous or

subsequent years.

Missing Data

Missing data is observed in school and student variables. Schools 19, 20, 21, 22, 23,

122 and 174 have no data for at least half of the variables and are not considered

in the model because of lack of data. In addition, 20 350 participants have missing

data for a majority of the variables - primarily in the student covariates - and are

removed, further reducing the �nal data set.

The remaining data set has 39 683 rows and 20 columns containing NN Rasch and

19 covariates - schoolno, procyear, gradedyear, gpokm, isolation, spatial_ar,

staff_metr, atsi, lbote, gender, aboriginal, disability, school_car, occupation,

school_edu, non_school, p_g_gender, p_g_nesb and home_langu.

3.2 Statistical Reduction of Predictor Variables

Having manually pruned the data set, we wish to �nd the statistically simplest model

which will best explain the relationship between the student and school variables

and the NN Rasch scores. We pursue two alternative methods - �rstly looking at

signi�cant predictor variables in a simple linear regression model and then using a

step-wise selection process based on Akaike's Information Criterion.

More complicated hierarchical modelling and model selection will be discussed in

Chapter 4 and Chapter 5.

84 3.2. Statistical Reduction of Predictor Variables

3.2.1 Theory

Linear Regression

Suppose we have data

(x1, y1), (x2, y2), . . . , (xn, yn),

where x represents the vector of k predictor variables. The classical linear regression

model can be written as

yi = Xiβ + εi

= β1Xi1 + . . . βkXik + εi for i = 1, . . . , n,

where the error εi is independently distributed εi ∼ N(0, σ2) and β is the vector of

regression coe�cients. The linear regression model in matrix form is

yi ∼ N(Xiβ, σ2) for i = 1, . . . , n,

or

y ∼ Nn(Xβ, σ2I)

where y is a vector of length n, X is a n × k matrix of predictors, β is a column

vector of length k and I is the n× n identity matrix.

It is well-known statistical theory that the least squares estimate of β is

β = (XTX)−1XTy.

Model Selection

As before, suppose we have data

(x1, y1), (x2, y2), . . . , (xn, yn)

and we wish to choose a suitable model which is parsimonious and well-�tting.

Ideally, we want to choose the smallest well-�tting model and to do this, we need to

balance the number of parameters to avoid unnecessarily increasing the complexity

of the model but at the same time, ensuring the model �ts the data. Exclusion of

important terms clearly leads to an incorrect model, which can then cause misleading

conclusions. On the other hand, including unnecessary terms diminishes the value

of the model as a simpli�cation of the data and can lead to over-�tting.

In linear regression and generalised linear regression models, the model selection


problem is often stated as a variable selection problem. We have a response variable

y and a set of predictors x = {x1, . . . , xn}. We wish to divide x into two groups,

x = (xA,xI), the active and inactive predictors, such that the distribution of y|xAis the same as the distribution of y|(xA,xI). Hence, all the information about y is

explained by the active predictors.

General model selection is concerned with not just �nding the active predictors

xA but also building the model itself. This includes de�ning the model predictors

and selecting distributions and other modelling assumptions. Initially, we address

limited variable selection by assuming the linear model and �nd the active variables

xA or, equivalently, delete the inactive variables xI . Subsets of the full model are

taken and for each, some chosen information criterion of model quality is calculated

and optimized.

Another important criterion for model selection is the satisfaction of the principle

of marginality. The principle of marginality states that when an interaction term is

included in a model, all implied lower order interactions and main e�ects must also be

included to be a sensible model. In general, it is wrong to test, estimate or interpret

the main e�ects of explanatory variables separate to a signi�cant interaction term or

to model interaction e�ects when main e�ects that are marginal to the interactions

are deleted. While such models are interpretable, they ignore the e�ects of the

marginal main e�ects and lack applicability. This principle of marginality must be

satis�ed in any �nal model as a result of model selection.

3.2.2 Signi�cant Variables in Linear Regression

We start by �tting a linear regression of NN Rasch against the main e�ects and

identify multicollinearity between the covariates of gpokm, isolation, spatial_ar,

school_eduInconsistent and home_languY with the schoolno variable. Multi-

collinearity occurs when two or more predictor variables in a multiple regression

model are highly correlated. This correlation then impacts on the ability to perform

the matrix inversion required for computing regression coe�cients, and parameter es-

timates cannot be reliably computed for highly correlated variables. Multicollinear-

ity is also a problem because it increases the standard errors of the coe�cients and

increased standard errors in turn could result in the false conclusion of the statistical


insigni�cance of a variable, when it is actually signi�cant.

In our data, collinearity is explained by the schools partitioning each category

of the school covariates, and as a result, schoolno gives the same information

as all the school covariates combined together. The R function alias expresses

the school covariates as a linear combination of schoolno categories, home_languY

and school_eduInconsistent, and this con�rms multicollinearity. To remove the

collinearity with home_languY, the variable home_langu is removed from the data

set. Data where school_edu is inconsistent is also removed as any conclusions for

this category are not reliable due to the inconsistency of the data.

On this basis, we consider two di�erent models - the school-number model which in-

cludes schoolno but excludes all school covariates, gpokm, isolation, spatial_ar

and staff_metr, and the school-covariates model where the school covariates re-

place schoolno in the model.

School-Number Model

Consider the full main e�ects model with schoolno - this model is henceforth known

as the school-number model.

School-Number Model

NN Rasch = schoolno + procyear + gradedyear + atsi + lbote + gender

+ aboriginal + disability + school_car + occupation + school_edu +

non_school + p_g_gender + p_g_nesb.

The linear regression of schoolno and the other main e�ects gives 10 signi�cant

variables (schoolno, gradedyear, atsi, lbote, gender, disability, school_car,

occupation, school_edu, non_school) and 50 categories which are signi�cant (Ap-

pendix C, Table C.1.1). The non-signi�cant variables are procyear, aboriginal,

p_g_gender and p_g_nesb and would be excluded from the model based on signi�-

cance. After the removal of the non-signi�cant variables, the model which shall be

referred to as the signi�cant-predictors school-number model is


Signi�cant-Predictors School-Number Model

NN Rasch = schoolno + gradedyear + atsi + lbote + gender + disability +

school_car + occupation + school_edu + non_school.

Out of the 400 schools, only 33 schools are signi�cantly di�erent from the inter-

cept school 27 at the 5% signi�cance level. Figure 3.2.1 plots all coe�cients of the

linear regression other than schoolno and the 95% con�dence intervals of the �tted

estimates. All of the regression coe�cients are compared to the baseline category or

intercept of a female student in school 27 and grade 3 in 1998 who does not come

from an Aboriginal, Torres Strait Islander or non-English speaking background, is

not identi�ed as having a disability or School Card and their primary guardian or

parent is female from an English speaking background whose status of occupation,

school education and non-school education is not stated. The signi�cant categories

are coloured blue and the non-signi�cant categories are coloured red in Figure 3.2.1.

This �gure plots all the regression coe�cients on a common scale from which we

can observe the relative e�ect of each of the levels and predictor variables on the

baseline category. Compared to all other parameters, the procyear categories have

wide error bars, and the widest error bars for 1999 and 2005 are due to the lack of

data in those years as discussed in Chapter 2. All other regression coe�cients have

a relatively small standard error compared to those three years. To better see the

majority of points and error bars around the line of no deviation from the intercept,

Figure 3.2.2 is Figure 3.2.1 restricted to the range of -5 to 5.


●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

Y

1

Inconsistent

Y

M

5

7

1

Inconsistent

5678

12348

M

Y

1999200020012002200320042005

Y

1234

aboriginalatsi

disabilitygender

gradedyearlbote

non_school

occupationp_

g_gender

p_g_

nesbprocyear

school_car

school_edu

−20 −15 −10 −5 0 5 10 15Value

Cat

egor

ies Significant

●

●

No

Yes

Figure 3.2.1: Coe�cient plot for the school-number model. (See text for de�nitionof baseline category. The Inconsistent category of the atsi and lbote variablesrepresents students with identi�ed anomalies in the data.)


●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

Y

1

Inconsistent

Y

M

5

7

1

Inconsistent

5678

12348

M

Y

1999200020012002200320042005

Y

1234

aboriginalatsi

disabilitygender

gradedyearlbote

non_school

occupationp_

g_gender

p_g_

nesbprocyear

school_car

school_edu

−5 −4 −3 −2 −1 0 1 2 3 4 5Value

Cat

egor

ies Significant

●

●

No

Yes

Figure 3.2.2: Subset of the coe�cient plot for the school-number model. The x-axisranges from -5 to 5. (See text for de�nition of baseline category. The Inconsis-tent category of the atsi and lbote variables represents students with identi�edanomalies in the data.)


School-Covariates Model

We �t the alternative school-covariates model, replacing schoolno with the school

covariates of gpokm, isolation, spatial_ar, staff_metr and school size. School

size is de�ned to be the number of recorded participants for a school in the data set

and is denoted by a di�erent font as it is a deduced variable and not an observed

variable.


NN Rasch= procyear+ gradedyear+ gpokm+ isolation+ spatial_ar staff_metr

+ atsi+ lbote+ gender+ aboriginal+ disability+ school_car+ occupation

+ school_edu + non_school + p_g_gender + p_g_nesb + school size.

The signi�cant variables are gradedyear, gpokm, isolation, spatial_ar, atsi,

lbote, gender, aboriginal, disability, school_car, occupation, school_edu,

non_school and school size (Appendix C, Table C.1.2). The model which contains

only the signi�cant predictor variables is known as the signi�cant-predictors school-

covariates model.

Signi�cant-Predictors School-Covariates Model

NN Rasch = gradedyear + gpokm + isolation + spatial_ar + atsi + lbote +

gender + aboriginal + disability + school_car + occupation + school_edu

+ non_school + school size.

Figure 3.2.3 plots all coe�cients of categorical variables in the linear regression and

the 95% con�dence intervals of the �tted estimates. All of the regression coe�cients

are compared to the baseline category or intercept of a female student in grade 3 in

1998 who does not come from an Aboriginal, Torres Strait Islander or non-English

speaking background, is not identi�ed as having a disability or School Card and their

primary guardian or parent is female from an English speaking background whose

status of occupation, school education and non-school education is not stated. The

characteristics of the reference school is one with an isolation factor of 1, spatial

value of 1.1 (metropolitan) and metro sta� classi�cation. The signi�cant categories

are coloured blue and the non-signi�cant categories are coloured red in Figure 3.2.3.


This �gure plots all the regression coe�cients on a common scale from which we

can observe the relative e�ect of each of the levels and predictor variables on the

baseline category. To better see the majority of points and error bars around the

line of no deviation from the intercept, Figure 3.2.4 is Figure 3.2.3 restricted to the

range of -5 to 5.

In addition to the categorical variables, the regression output for the two continu-

ous predictors, gpokm and school size, are given in Table 3.2.1. Both variables are

signi�cant at the 5% signi�cance level.

Table 3.2.1: Regression output for gpokm and school size from the school-covariatesmodel

Estimate Std. Error t-value P -valuegpokm -0.0095 0.0020 -4.7041 2.6e-06school size 0.0027 0.0006 4.4995 6.9e-06

3.2.3 Simplest Main E�ects Model using stepAIC

We have now established the full main e�ects linear model and observed that certain

variables are not signi�cant. Removing these variables from the model gives us the

simplest model based on the signi�cance of variables as stated in Section 3.2.2 for

both the school-number and school-covariates models.

Step-wise model selection is another method to eliminate the variables which are

not statistically signi�cant and to achieve the simplest model. In general, step-wise

model selection is a combination of forward and backward selection.

Suppose we wish to decrease or minimise a model selection criterion. Under the

forward selection algorithm, the initial model is the null model with no predictors

or the model with all pre-determined necessary predictors included. The algorithm

continues to add predictors until adding another variable increases the criterion of

interest. If a predictor decreases the criterion, it is included in the model, and this

process continues until no further variables are added. Similarly, the backward selec-

tion algorithm starts with the most complicated model which includes all predictors

and continues to remove variables until the criterion of interest is increased by the


●

●

●

●

●

●

●

●●

●●

●●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

Y

1

Inconsistent

Y

M

5

7

1.522.533.544.555.566.57

1

Inconsistent

5

6

7

8

12348

M

Y

1999200020012002200320042005

Y

1

2

3

4

2.2.1

2.2.2

3.1

3.2

C

aboriginalatsi

disabilitygender

gradedyearisolation

lbotenon_

schooloccupation

p_g_

genderp_

g_nesb

procyearschool_

carschool_

eduspatial_

arstaff_

metr

−20 −15 −10 −5 0 5 10 15 20Value

Cat

egor

ies Significant

●

●

No

Yes

Figure 3.2.3: Coe�cient plot for the school-covariates model. (See text for de�nitionof baseline category. The Inconsistent category of the atsi and lbote variablesrepresents students with identi�ed anomalies in the data.)


●

●

●

●

●●

●●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

Y

1

Inconsistent

Y

M

5

7

1.522.533.544.555.566.57

1

Inconsistent

5

6

7

8

12348

M

Y

1999200020012002200320042005

Y

1

2

3

4

2.2.1

2.2.2

3.1

3.2

C

aboriginalatsi

disabilitygender

gradedyearisolation

lbotenon_

schooloccupation

p_g_

genderp_

g_nesb

procyearschool_

carschool_

eduspatial_

arstaff_

metr

−5 −4 −3 −2 −1 0 1 2 3 4 5Value

Cat

egor

ies Significant

●

●

No

Yes

Figure 3.2.4: Subset of coe�cient plot for the school-covariates model. The x axisranges from -5 to 5. (See text for de�nition of baseline category. The Inconsis-tent category of the atsi and lbote variables represents students with identi�edanomalies in the data.)


removal of a variable. At this point, the model contains only signi�cant variables.

Step-wise model selection is a combination of both forward and backward selection

in which at each step, either the e�ect of the addition of a predictor or the removal of

a predictor on the criterion is considered. This process also applies to the situation

where the model selection criterion is maximised to indicate a �good� model.

We choose to use stepAIC which is a step-wise model selection process based on

minimising Akaike's Information Criterion (AIC). Akaike's Information Criterion

[2] is a measure of the relative goodness of �t of a statistical model. In the general

case, Akaike's Information Criterion is used to compare models of the same type,

for example, regression models. AIC is de�ned to be

AIC = 2k − 2 ln(L) (3.2.1)

where k is the number of parameters in the statistical model and L is the maximised

value of the likelihood function for the estimated model. As with all model selection,

there needs to be a balance between the model �t and a penalty for model complexity

and the number of parameters. This is accounted for in equation (3.2.1) where

D = −2 ln(L) is the deviance and is a measure of model �t and 2k compensates for

the number of estimated parameters. The model with the smallest AIC is chosen as

the best model.

A few important points to consider when using stepAIC and interpreting the results

is that AIC can be applied to compare nested models, and the process of �tting

models is by maximum likelihood. It is also important to note that often there is

no one single best model, and it is wise to consider all models which are within two

units of the minimal AIC [40].

With this statistical theory, the stepAIC function in R is contained in the MASS

package and performs step-wise model selection by AIC. It takes a model object

and uses it as the initial model in the step-wise search. The default direction for

stepAIC is a backward step-wise search.

School-Number Model

From the school-number model, the variables p_g_nesb and p_g_gender are re-

moved by stepAIC to give the simplest-stepAIC school-number model.


Simplest-stepAIC School-Number Model

NN Rasch = schoolno + procyear + gradedyear + atsi + lbote + gender

+ aboriginal + disability + school_car + occupation + school_edu +

non_school.

Table C.1.3 in Appendix C gives the linear regression output, and we can see that the

variables procyear and aboriginal are not signi�cant but are still included. The

model with only signi�cant predictors is the signi�cant-predictors simplest-stepAIC

school-number model.

Signi�cant-Predictors Simplest-stepAIC School-Number Model

NN Rasch = schoolno + gradedyear + atsi + lbote + gender + disability +

school_car + occupation + school_edu + non_school. (3.2.2)

Figure 3.2.5 plots all coe�cients of the simple linear regression model other than

schoolno and the 95% con�dence intervals of the �tted estimates. All of the re-

gression coe�cients are compared to the baseline category or intercept of a female

student in school 27 and grade 3 in 1998 who does not come from an Aboriginal,

Torres Strait Islander or non-English speaking background, is not identi�ed as hav-

ing a disability or School Card and their primary guardian or parent is female from

an English speaking background whose status of occupation, school education and

non-school education is not stated. The signi�cant categories are coloured blue and

the non-signi�cant categories are coloured red in Figure 3.2.5. This �gure plots all

the regression coe�cients on a common scale from which we can observe the relative

e�ect of each of the levels and the predictor variables on the baseline category. To

better see the majority of points and error bars around the line of no deviation from

the intercept, Figure 3.2.6 is Figure 3.2.5 restricted to the range of -5 to 5.

The model in equation (3.2.2) is the same as the signi�cant-predictors school-number

model in Section 3.2.2 after the manual removal of non-signi�cant variables. We can

see from Figure 3.2.7 that there is no observable di�erence between the coe�cients

of the categories and variables in both the full school-number model and the full

simplest-stepAIC school-number model.


●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

Y

1

Inconsistent

Y

M

5

7

1

Inconsistent

5

6

7

8

12348

1999200020012002200320042005

Y

1

2

3

4

aboriginalatsi

disabilitygender

gradedyearlbote

non_school

occupationprocyear

school_car

school_edu

−20 −15 −10 −5 0 5 10 15Value

Cat

egor

ies Significant

●

●

No

Yes

Figure 3.2.5: Coe�cient plot for the simplest-stepAIC school-number model. (Seetext for de�nition of baseline category. The Inconsistent category of the atsi andlbote variables represents students with identi�ed anomalies in the data.)


●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

Y

1

Inconsistent

Y

M

5

7

1

Inconsistent

5

6

7

8

12348

1999200020012002200320042005

Y

1

2

3

4

aboriginalatsi

disabilitygender

gradedyearlbote

non_school

occupationprocyear

school_car

school_edu

−5 −4 −3 −2 −1 0 1 2 3 4 5Value

Cat

egor

ies Significant

●

●

No

Yes

Figure 3.2.6: Subset of coe�cient plot for the simplest-stepAIC school-numbermodel. (See text for de�nition of baseline category. The Inconsistent categoryof the atsi and lbote variables represents students with identi�ed anomalies in thedata.)


●

●

●●

●●

●

●

●

●

●●

●●

●

●

●

●

●●●●●

●●●

●●

●●

●

−10

0

10

20

30

40

50

−10 0 10 20 30 40 50Coefficients − School−Number model

Coe

ffic

ient

s −

Sim

ples

t−st

epA

IC S

choo

l−N

umbe

r m

odel

Significant●

●

No

Yes

Figure 3.2.7: Coe�cients for the simplest-stepAIC school-number model plottedagainst the regression coe�cients for the school-number model. The dotted line isthe line y=x.



From the school-covariates model, the variables staff_metr, p_g_gender and p_g_nesb

are removed by stepAIC to give the simplest-stepAIC school-covariates model.

Simplest-stepAIC School-Covariates Model

NN Rasch = procyear + gradedyear + gpokm + isolation + spatial_ar + atsi

+ lbote + gender + aboriginal + disability + school_car + occupation +

school_edu + non_school + school size.

Table C.1.4 in Appendix C gives the linear regression output and the procyear vari-

able is not signi�cant. The model with only signi�cant predictors is the signi�cant-

predictors simplest-stepAIC school-covariates model.

Signi�cant-Predictors Simplest-stepAIC School-Covariates Model

NN Rasch = gradedyear + gpokm + isolation + spatial_ar + atsi + lbote +

gender + aboriginal + disability + school_car + occupation + school_edu

+ non_school + school size. (3.2.3)

Figure 3.2.8 plots all coe�cients of categorical variables in the linear regression

and the 95% con�dence intervals of the �tted estimates. All of the regression co-

e�cients are compared to the baseline category or intercept of a female student in

grade 3 in 1998 who does not come from an Aboriginal Torres Strait Islander or non-

English speaking background, is not identi�ed as having a disability or School Card

and their primary guardian's or parent's status of occupation, school education and

non-school education is not stated. The characteristics of the reference school is one

with an isolation factor of 1 and spatial value of 1.1 (metropolitan). The signi�cant

categories are coloured blue and the non-signi�cant categories are coloured red in

Figure 3.2.8. This �gure plots all the regression coe�cients on a common scale from

which we can observe the relative e�ect of each of the levels and predictor variables

on the baseline category. To better see the majority of points and error bars around

the line of no deviation from the intercept, Figure 3.2.9 is Figure 3.2.8 restricted to

the range of -5 to 5.


●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

Y

1

Inconsistent

Y

M

5

7

1.52

2.53

3.54

4.55

5.56

6.57

1

Inconsistent

5

6

7

8

1

2

3

4

8

1999200020012002200320042005

Y

1

2

3

4

2.2.1

2.2.2

3.1

3.2

aboriginalatsi

disabilitygender

gradedyearisolation

lbotenon_school

occupationprocyear

school_carschool_edu

spatial_ar

−20 −15 −10 −5 0 5 10 15 20Value

Cate

gorie

s Significant

●

●

No

Yes

Figure 3.2.8: Coe�cient plot for the simplest-stepAIC school-covariates model. (Seetext for de�nition of baseline category. The Inconsistent category of the atsi andlbote variables represents students with identi�ed anomalies in the data.)


●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

Y

1

Inconsistent

Y

M

5

7

1.52

2.53

3.54

4.55

5.56

6.57

1

Inconsistent

5

6

7

8

1

2

3

4

8

1999200020012002200320042005

Y

1

2

3

4

2.2.1

2.2.2

3.1

3.2

aboriginalatsi

disabilitygender

gradedyearisolation

lbotenon_school

occupationprocyear

school_carschool_edu

spatial_ar

−5 −4 −3 −2 −1 0 1 2 3 4 5Value

Cate

gorie

s Significant

●

●

No

Yes

Figure 3.2.9: Subset of coe�cient plot for the simplest-stepAIC school-covariatesmodel. The x axis ranges from -5 to 5. (See text for de�nition of baseline category.The Inconsistent category of the atsi and lbote variables represents students withidenti�ed anomalies in the data.)


In addition to the categorical variables, the regression output for the two continu-

ous predictors, gpokm and school size, are given in Table 3.2.2. Both variables are

signi�cant at the 5% signi�cance level.

Table 3.2.2: Regression output for gpokm and school size from the simplest-stepAICschool-covariates model

Estimate Std. Error t-value P -valuegpokm -0.0095 0.0020 -4.7313 2.2e-06school size 0.0027 0.0006 4.5364 5.8e-06

The model in equation (3.2.3) is the same as the signi�cant-predictors school-

covariates model in Section 3.2.2 after the manual removal of non-signi�cant vari-

ables. We can see from Figure 3.2.10 that there is no important di�erence between

the coe�cients of the categories and variables in both the full school-covariates

model and the full simplest-stepAIC school-covariates model.

We observe that although procyear is included in the simplest stepAIC model for

both the school-number and school-covariates models, it is not a signi�cant variable

- none of the calendar years have a signi�cant e�ect. Comparison of the models - one

not including and the other including procyear - using ANOVA (Tables 3.2.3 and

3.2.4), indicate that there is a signi�cant di�erence between the two models, and

procyear should be included in the simplest model. For comparison, p_g_nesb is a

variable which is not included in the simplest model by stepAIC, and the ANOVA

tables (Tables 3.2.5 and 3.2.6) say that it should not be included. This di�erence in

the ANOVA results support the inclusion of procyear by stepAIC.

Table 3.2.3: Comparison of models with and without procyear - simplest-stepAICschool-number model. Model 1 is without procyear and model 2 is with procyear

Model Res.Df RSS Df Sum of Sq F -value P -value1 19 733 894 796.912 19 726 890 826.32 7 3 970.59 12.56 < 10−16


●

●

●●

●●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●

●●

●

●

●●

●●

●

●

●

●

●●●●

●

●●

●●●

●●

●0

20

40

0 20 40Coefficients − School−Covariates model

Coe

ffic

ient

s −

Sim

ples

t−st

epA

IC S

choo

l−C

ovar

iate

s m

odel

Significant●

●

No

Yes

Figure 3.2.10: Coe�cients for the simplest-stepAIC school-covariates model plottedagainst the regression coe�cients for the school-covariates model. The dotted lineis the line y=x.


Table 3.2.4: Comparison of models with and without procyear - simplest-stepAICschool-covariates model. Model 1 is without procyear and model 2 is with procyear

Model Res.Df RSS Df Sum of Sq F -value P -value1 20 092 1 000 816.992 20 085 996 914.10 7 3 902.89 11.23 < 10−16

Table 3.2.5: Comparison of models with and without p_g_nesb - simplest-stepAICschool-number model. Model 1 is without p_g_nesb and model 2 is with p_g_nesb.

Model Res.Df RSS Df Sum of Sq F -value P -value1 19 726 890 826.322 19 725 890 778.47 1 47.85 1.06 0.3033

Table 3.2.6: Comparison of models with and without p_g_nesb - simplest-stepAICschool-covariates model. Model 1 is without p_g_nesb and model 2 is withp_g_nesb.

Model Res.Df RSS Df Sum of Sq F -value P -value1 20 085 996 914.102 20 084 996 911.18 1 2.93 0.06 0.8082

Sequential analysis of variance using Anova from the car package in R, reports Type

II hypothesis tests. Pairs of models are compared, and it tests the addition of one

of the predictors to a model that includes all the other predictors. For the school-

number model, aboriginal, p_g_gender and p_g_nesb are not signi�cant variables

(P -values of 0.1123, 0.8148 and 0.2995 respectively - Table 3.2.7), and similarly,

staff_metr, p_g_gender and p_g_nesb are not signi�cant predictors in the school-

covariates model (P -values of 0.5363, 0.3368 and 0.7863 respectively - Table 3.2.8),

both of which agree with the conclusion from the stepAIC model selection.


Table 3.2.7: Type II Anova test for the full school-number model

Variable Sum Sq Df F -value P -valueschoolno 118 172.64 401 6.53 < 2.2e-16procyear 3 984.70 7 12.60 3.1e-16gradedyear 273 038.52 2 3 022.88 < 2.2e-16atsi 417.13 2 4.62 0.0099lbote 393.94 2 4.36 0.0128gender 5 604.44 1 124.10 < 2.2e-16aboriginal 113.87 1 2.52 0.1123disability 64 839.04 1 1 435.70 < 2.2e-16school_car 794.79 1 17.60 2.7e-05occupation 826.27 5 3.66 0.0026school_edu 5 586.92 4 30.93 < 2.2e-16non_school 3 138.21 4 17.37 3.1e-14p_g_gender 2.48 1 0.05 0.8148p_g_nesb 48.62 1 1.08 0.2995Residuals 890 775.99 19 724

Table 3.2.8: Type II Anova test for the full school-covariates model

Variable Sum Sq Df F -value P -valueprocyear 3 916.88 7 11.27 2.5e-14gradedyear 282 113.13 2 2 841.66 < 2.2e-16gpokm 1 098.43 1 22.13 2.6e-06isolation 6 395.00 12 10.74 < 2.2e-16spatial_ar 971.62 4 4.89 0.0006staff_metr 18.99 1 0.38 0.5363atsi 781.25 2 7.87 0.0004lbote 545.45 2 5.49 0.0041gender 6 577.81 1 132.51 < 2.2e-16aboriginal 190.61 1 3.84 0.0501disability 74 593.08 1 1 502.72 < 2.2e-16school_car 1 537.15 1 30.97 2.7e-08occupation 4 369.33 5 17.60 < 2.2e-16school_edu 11 022.63 4 55.51 < 2.2e-16non_school 6 626.15 4 33.37 < 2.2e-16p_g_gender 45.80 1 0.92 0.3368p_g_nesb 3.65 1 0.07 0.7863school size 1 004.94 1 20.25 6.9e-06Residuals 996 845.64 20 082


3.2.4 Investigation of procyear

Based on the theory of Rasch modelling, there should not be a signi�cant di�erence

in NN Rasch scores across di�erent years - the Rasch model should take that into

account and normalise all results. This is observed as none of the calendar years

have a signi�cant e�ect.

When we look at the estimated parameters for procyear in Tables C.1.3 and C.1.4

in Appendix C, we observe that the years which are most di�erent from the baseline

year of 1998 are 1999 and 2005. We observe that these are the years with small

cohorts (Table 2.3.1, Section 2.3.2), and it is permissible for them to be removed

from the data. When the data from 1998, 1999 and 2005 are removed from the

overall data set, the variable procyear is now signi�cant in 2002 and 2003 in the

linear regression of the school-number and the school-covariates models (Tables 3.2.9

and 3.2.10).

Table 3.2.9: Summary linear regression output of procyear in the school-numbermodel

Estimate Std. Error t-value P -valueIntercept 46.5944 3.023 15.413 < 2e-16

procyear2001 0.1181 0.2730 0.432 0.6655procyear2002 1.1456 0.2615 4.380 1.19e-05procyear2003 0.5230 0.2639 1.982 0.0475procyear2004 -0.1221 0.2859 -0.427 0.6695

......

......

...

This result then indicates that having taken into account the theory of Rasch mod-

elling, which states that there should be no statistically signi�cant �uctuation in

NN Rasch scores over years, and the observed increasing trend in the Rasch scores

of the data which is due to the in�ux of Grade 5 and Grade 7 students in the later

years (Section 2.5), there is still some yearly variation which is identi�ed by the

model. We can model this variation of procyear as a random e�ect with mean zero

to satisfy the theory of Rasch modelling. This will be explored further in mixed

e�ects models (Chapter 4) and hierarchical Bayesian models (Chapter 5).


Table 3.2.10: Summary linear regression output of procyear in the school-covariatesmodel

Estimate Std. Error t-value P -valueIntercept 49.8429 0.2894 172.206 < 2e-16

procyear2001 0.1967 0.2795 0.704 0.4816procyear2002 1.2656 0.268 4.723 2.34e-06procyear2003 0.7541 0.2670 2.824 0.0047procyear2004 0.1588 0.2893 0.549 0.5831

......

......

...

3.2.5 Comparison of School-Number and School-Covariates

Models

We have now two models - the school-number model and the school-covariates model.

The school-number model identi�es schools whose mean scores are signi�cantly dif-

ferent from the average NN Rasch mark, but from this model, we do not have any

information, for example, about the location or the level of government funding

of these schools. This information is contained in the school covariates of gpokm,

isolation, spatial_ar and staff_metr and may be important in determining the

reason why certain schools are under- or over-performing. The school-covariates

model contains more information and hence, has increased usefulness for making

conclusions and decisions at the school level. For this reason, we disregard the

school-number model from this point in favour of the school-covariates model.

However, the school factor or school e�ect is not included in the school-covariates

model, and we wish to still be able to identify individual schools. The school-

covariates model we are referring to here and for the rest of this section is the

signi�cant-predictors simplest-stepAIC school-covariates model in equation (3.2.3)

�t on data excluding procyears 1998, 1999 and 2005 (as established in Sections

3.2.3 and 3.2.4). To identify individual schools, we calculate the mean �tted score

of each school from the school-covariates model and consider that to be the school

e�ect from the school-covariates model. The data for a small worked example is

given in Table 3.2.11, and each row represents the data for an individual student in

school 27.


Table 3.2.11: Subset of data to illustrate raw and �tted scores

School Number Raw Scores Fitted Model Scores27 42.03732 45.8631527 46.55742 50.9644327 48.64362 50.6049127 47.39190 52.6054427 57.89244 50.75956

Mean 48.50454 50.1595

From Table 3.2.11, the mean school e�ect from the raw data is 48.5 and the mean

school e�ect from the school-covariates model is 50.16. These means shall be known

as the original raw mean school e�ect and the original model mean school e�ect

respectively for school 27 and are calculated for all schools in the data from which

the school-covariates model was �tted. This calculated data of original raw mean

school e�ects and original model mean school e�ects is called original to distinguish

it from the transformed data considered later.

To test whether the original model mean school e�ects are signi�cant predictors

of the original raw mean school e�ects, we do a linear regression of original raw

mean school e�ects against original model mean school e�ects (Table 3.2.12 and

Figure 3.2.11). From the summary output, the original model mean school e�ect is

a signi�cant predictor at a 5% signi�cance level.

Table 3.2.12: Summary output of Raw ∼ Model linear regression

Estimate Std. Error t-value P -valueIntercept -5.6243 2.8476 -1.98 0.0489Model 1.1084 0.0538 20.59 < 2e-16

The regression line has a slope which is signi�cantly di�erent from one at the 5%

signi�cance level (test statistic of 2.01 and P -value of 0.045). Hence, there is a

signi�cant di�erence between the least squares line and the line Raw = Model.

To assess the validity of the assumptions of the linear regression, the diagnostic

plots are included in Figure 3.2.12. Standard assumption checking for linearity,


●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

● ●●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

● ●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●●

●●

●

●

●

●

●

●●

●●

●

●

●

●●

●

●

●

●●

●

●

●●

●

●

●

●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●●

●

●

●

●●

●

●

●●

●●

●

●

●

●

●●

●

●

●

●

●

● ●

●

●

● ●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

● ● ●

● ●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●●

●

●

●

●●●●

●

●

●

●

●

●

●

●

20

30

40

50

60

70

40 50 60Model

Raw

Figure 3.2.11: Linear regression of original raw mean school e�ects against the �ttedoriginal model mean school e�ects (the black line represents the line Raw = Model).

homoscedasticity and normality can be made from these diagnostic plots, and there

is no reason to suppose that the linear regression assumptions are invalid as a result.

However, when we plot the residuals against school size, there is stark heteroscedas-

ticity, and the data points form the shape of a funnel in Figure 3.2.13. As school

size increases, variability in the di�erence in the original raw and model mean school

e�ects decreases. It would appear that school size can explain some of the variance

observed in the residuals, and we investigate this conjecture further. Our objective is

to �nd a weighted transformation by school size of the original raw and model mean

school e�ects such that heteroscedasticity in the residuals is reduced somewhat, if

not completely.


35 40 45 50 55 60 65

−20

−10

010

Fitted values

Res

idua

ls

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●●

● ●

●

●

●

●

●●●

●

●●

●

●

●●

●

●

●

●

●●●●

●

●●

●●● ●

●●●

●●●

●

●●●

●

●●

●

●

●

●●

●●

●●

●

●

●

●

●

●

●●●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

● ●

●●

● ●●

●

●

●●

●

● ●●

●●

●

● ●

●

● ●

●

●●

●

●●●

●

●● ●●

●●●

● ●

●

●

●●

●●●

●

●

●

●

●●

●●

●

●

●●●

●● ●●

●●

●

●

●

●●

●●

●

●

●

●

●●

●●●

●●●

●●●

●●●

●●●●

●

●●●●

●●

●

●●●●

●

●

●

●●

●●

●●

● ●●

●

●

●

●●

●●

●●●●

●

●

●

●

●

●

●

●

●

●●●●●●

●●

●

●

●

●

●●●

●●

●

●●●

●●

●

●

●

●

●●

●●

●

● ●●●

●

●

●●●●● ●

●

●●

●●●●

●

●●●

●

●

●●

●

●●●●

●

●●●

●

●●●

●●●●

●

●

●●

●

●●

●

●●●

●●

●●●●

●

●

●

●

●●

●

●●

●●

●●●●●

●

●

●●

●● ●

●

●●

●

●

●

●●●●

●●

●

●●

●

●●●●

●●

●

●

● ●

●●●●

Residuals vs Fitted

105

14 249

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

● ●

●

●

●

●

● ●●

●

●●

●

●

●●

●

●

●

●

●●

●●

●

●●

●●

●●●

●●

●●●

●

●●●

●

●●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●●●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●●

●●●

●

●

●●

●

● ●●

●

●

●

●●

●

●●

●

●●

●

●●

●

●

● ●●●

● ●●

●●

●

●

●●

●●●

●

●

●

●

●●

●●

●

●

●

●●

●●●●

●●

●

●

●

●●

●●

●

●

●

●

●●

●●●

●●

●

●●●

●●

●

●●

● ●

●

●●●●

●●

●

●●

●●●

●

●

●●

●●

●●

●●●

●

●

●

●●

●●

● ●●●

●

●

●

●

●

●

●

●

●

●●●●

●●

●●

●

●

●

●

●●●

●●

●

● ●●

●●

●

●

●

●

●●

●●

●

●●●●

●

●

●●

●●●●

●

●●

●●

●●

●

●●●

●

●

●●

●

●●●●

●

●●●

●

●●●

●●●●

●

●

●●

●

●●

●

●●●

●●

●●●●

●

●

●

●

●●

●

●●

●●

●●●●●

●

●

●●

●● ●

●

●●

●

●

●

●● ●●

●●

●

●●

●

●● ●●

●●

●

●

●●

●●●●

−3 −2 −1 0 1 2 3

−6

−4

−2

02

4

Theoretical Quantiles

Stan

dard

ized

res

idua

ls

Normal Q−Q

105

14249

35 40 45 50 55 60 65

0.0

0.5

1.0

1.5

2.0

2.5

Fitted values

Stan

dard

ized

res

idua

ls

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

● ●●●

●

● ●

●

●

●●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●● ●●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

● ●

●

● ●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

● ●

●

●

●

●

●●

●

●

● ●

●

●

●

●

●●

●●

●

●

●

● ●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

● ●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

● ●

●

●

●●●

●

●●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●●●

●●

●

●

●

●

●

●

● ●

●

●●

●●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●●●

●

●

●●●

●

●

●

●●●

●

●●

●●●

●

●●

●

●

●

●

● ●●

●

●

●●●

●

●

●

●

●

●●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●●

●

●

● ●

●●

●●

Scale−Location105

14 249

0.00 0.02 0.04 0.06

−8

−4

02

4

Leverage

Stan

dard

ized

res

idua

ls

●●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●●

●

●

●

●

●●●

●

●●

●

●

●●

●

●

●

●

●●●●

●

●●

●●●●

●●●

●●●

●

●●●

●

●●

●

●

●

●●

●●●●

●

●

●

●

●

●

●●●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●●

●●

●●●●

●

●●

●

●●●●

●

●

●●

●

●●

●

●●

●

●●●

●

●●●●●●

●●●

●

●●●

●●●

●

●

●

●

●●

●●

●

●

●●●●●●●●

●

●

●

●

●●

●●

●

●

●

●

●●

●●●

●●●

●●●●●●

●●●●

●

●●●●

●●

●

●●●●

●

●

●

●●●

●

●●●●

●●

●

●

●●

●●●●●●

●

●

●

●

●

●

●

●

●

●●●●●●

●●

●

●

●

●

●●●●

●

●

●●●

●●

●

●

●

●

●●●●

●

●●●●

●

●

●●●●●●

●

●●●

●●●

●

●●●●

●

●●

●

●●●●

●

●●●

●

●●●

●●●●

●

●

●●

●

●●●

●●●

●●

●●●●

●

●

●

●

●●●

●●

●●

●● ●●●

●●

●●

●●●●

●●

●

●

●

●●●●

●●

●

●●●

●●●●

●●

●

●

● ●

●●●●

Cook's distance

1

0.5

0.5

1

Residuals vs Leverage

105

14

4

Figure 3.2.12: Diagnostic plots to assess validity of assumptions in the linear regres-sion of Raw ∼ Model.


●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

● ●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

● ●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

● ●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

● ●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●●

●

●●

●

●

●●

● ●

●●

−20

−10

0

10

0 100 200 300 400 500School Size

Lin

ear

Reg

ress

ion

Res

idua

ls

Figure 3.2.13: Plot of residuals versus school size.

Statistical Theory

For homoscedastic data, the linear model is mathematically represented in matrix

form, as

Y = Xβ + ε

where ε is a random vector with

E(ε) = 0 and V ar(ε) = σ2I.

In the case that data is heteroscedastic, the assumption

V ar(ε) = σ2I

becomes

V ar(ε) = σ2V

where V is a known n× n positive de�nite, symmetric matrix.


Suppose we wish to estimate β. The ordinary least squares (OLS) estimate is

βOLS = (XTX)−1XTy.

βOLS is an unbiased linear estimator, but because the assumption of the Gauss-

Markov Theorem does not hold, it is not the best linear unbiased estimator. To

derive the best linear unbiased estimator, the problem must be transformed into

one for which the Gauss-Markov Theorem can be applied directly.

Theorem 3.2.1. Gauss-Markov Theorem [11]

Consider the linear regression model Y = Xβ + ε. Suppose E(Y ) = η = Xβ and

V ar(Y ) = σ2I. If aTY is an unbiased linear estimator for λTη then

V ar(aTY ) ≥ V ar(λT η)

with equality if and only if

a = X(XTX)−1XTλ.

Consider the regression model

Y = Xβ + ε

where ε is a random vector with

E(ε) = 0 and V ar(ε) = σ2V.

Pre-multiplying by V −12 gives

Y∗ = X∗β + ε∗

where

Y∗ = V −12Y , X∗ = V −

12X, and ε∗ = V −

12ε.

Now applying the rules for linear transformations of random variables, we �nd

E(ε∗) = E(V −12ε)

= V −12E(ε)

= V −120

= 0


and

V ar(ε∗) = V ar(V −12ε)

= V −12V ar(ε){V − 1

2}T

= σ2V −12V V −

12

= σ2I.

Hence, the Gauss-Markov Theorem applies, and the best linear unbiased estimator

is therefore

β = (XT∗ X∗)

−1XT∗ y∗

with

V ar(β) = σ2(XT∗ X∗)

−1.

Substituting for X∗ and y∗ produces the generalised least squares estimates

βGLS = (XTV −1X)−1XTV −1y

and

V ar(βGLS) = σ2(XTV −1X)−1.

Transformed Data

To account for the possible in�uence of school size on the model, we multiply both

the original raw mean school e�ects and the original model mean school e�ects by

the square root of school size to give the transformed raw mean school scores and

transformed model mean school scores. The linear regression of the transformed data

is given in Table 3.2.13. Now when the residuals are plotted again against the school

size (Figure 3.2.14), we observe the roughly constant variance of the residuals and

the reduced heteroscedasticity.

We have observed variance due to school size, however, school size does not com-

pletely explain this observed variance. Alternative methods like generalised least

squares could be used to transform the data and reduce the heteroscedasticity, but

ultimately, this variance should be modelled to identify all in�uential variables, not

only school size. This variation can be represented and modelled as a random e�ect

in a mixed e�ects model, which will be explained and implemented in Chapter 4.


Table 3.2.13: Summary output of linear regression Raw*sqrt(school size) ∼Model*sqrt(school size)

Estimate Std. Error t-value P -valueIntercept 1.2545 2.9206 0.43 0.6678transformed model 0.9975 0.0055 180.57 < 2e-16

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

● ●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●●● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

−50

0

50

100

0 100 200 300 400 500School size

Res

idua

ls

Figure 3.2.14: Plot of residuals versus school size for linear regression on data trans-formed by multiplying with square root of school size.

Signi�cant Schools

Our point of interest is to be able to identify schools which are under-performing or

over-performing compared to the other �average� schools at a statistically signi�cant

level. These schools would be of interest to investigate and see what programs are

being implemented to the advantage of students in the over-performing schools and

what can be done to aid under-performing schools.


From the original linear regression model in Table 3.2.12, 95% prediction bands are

calculated and plotted in Plot (A) of Figure 3.2.15. From this plot, it is clear that

the majority of schools fall within the 95% prediction bands and are represented

by the red dots in Figure 3.2.15. Eight schools are signi�cantly over-performing

and lie above the prediction interval (green dots in Figure 3.2.15) and ten schools

are signi�cantly under-performing and lie below the prediction interval (blue dots).

In comparison, the 95% prediction bands from the linear regression on the trans-

formed data identify nine signi�cantly over-performing schools (green dots), three

of which are signi�cant schools from the original linear regression. In addition, thir-

teen schools are signi�cantly under-performing (blue dots), and there is an overlap

of seven schools in both the original linear regression model and the transformed

linear regression model. Plot (B) of Figure 3.2.15 is Plot (A) of Figure 3.2.15 but

with the colours of the data points representing the signi�cant schools under the

transformed linear regression. The transformed linear model better satis�es the

assumption of homoscedasticity and has narrower prediction bands on the trans-

formed scale and more signi�cant schools. Table 3.2.14 is a 3×3 table of the numberof signi�cant schools under each of the two linear regression models, and Plot (C)

of Figure 3.2.15 highlights the three and seven schools which are signi�cant under

both linear regression models.

Table 3.2.14: The number of schools which are average, statistically signi�cantlyover-performing and under-performing under the original linear regression modeland the transformed linear regression model

Transformed ModelOriginal Model Average Over UnderAverage 371 6 6Over 5 3 0Under 3 0 7

To try and explain any common, underlying reasons or characteristics shared by

the signi�cant schools, we test whether the school covariates have a signi�cant re-

lationship with the signi�cance of schools - over, average and under - using χ2

tests to test for association. The school covariates under consideration are gpokm,

isolation, spatial_ar, staff_metr, school size and school type. Schools are clas-


●

●

●

●

● ●

●

●

●

●

●●

●

●

●

● ●

●

●●

●

●● ●

●

●

●●

●●●

●●

●

●

●●

●

●●●

●

●

●

●

●

●● ●●

●

●

●●

●●

●

●● ●

●●

●

● ● ●●● ●●

●

●●● ●

●

●

●

●

●●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●●●

●

●

●●

●

●

●●

●

●

●

●●

● ●●

●

●●

●●

●

●●

●

●

●

●

●

●●

● ●●

●●

●●●

●●

●

●

●●●●

●

●●

●●

●●

●

●●

●●● ●

●●

●

●

●

●

●

●

●●●

●

●

●●●●

●●

●

●●●

●

●●●

●●●●

●

●

●●●

●●●

●

●

●●●

●●●

● ●●●

●

●● ●●

●●

●●

●●●●

●●

●● ●

●

● ●●

●

●

●●●

●●

●● ●●●●

●

●

●

●●

●

●

●

●

● ●●●

●●●●

●

●

●●

●●●

●

● ●●●

●

●●

●● ●●

●● ●

●●● ●

●●●

●●

●●●●

●

●

●

●

● ●●●

● ●●●●

●

● ●●●●●

●

●

●●

●

●

●●

●

● ● ●●

●

●●●

●

●●●●●

●

●●●●●●●

●

●

●

●●

● ●●

● ●

●●

●

●●

●

●

●●

●

●●

●●●

●

●● ●●●●

●●

●

● ●●●●●● ●●

●●

●

●●

●●●

20

40

60

40 50 60Model

Raw

(A) Performance●

●

●

Average

Over

Under

●

●

●

●

● ●

●

●

●

●

●●

●

●

●

● ●

●

●●

●

●● ●

●

●

●●

●●●

●●

●

●

●●

●

●●●

●

●

●

●

●

●● ●●

●

●

●●

●●

●

●● ●

●●

●

● ● ●●● ●●

●

●●● ●

●

●

●

●

●●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●●●

●

●

●●

●

●

●●

●

●

●

●●

● ●●

●

●●

●●

●

●●

●

●

●

●

●

●●

● ●●

●●

●●●

●●

●

●

●●●●

●

●●

●●

●●

●

●●

●●● ●

●●

●

●

●

●

●

●

●●●

●

●

●●●●

●●

●

●●●

●

●●●

●●●●

●

●

●●●

●●●

●

●

●●●

●●●

● ●●●

●

●● ●●

●●

●●

●●●●

●●

●● ●

●

● ●●

●

●

●●●

●●

●● ●●●●

●

●

●

●●

●

●

●

●

● ●●●

●●●●

●

●

●●

●●●

●

● ●●●

●

●●

●● ●●

●● ●

●●● ●

●●●

●●

●●●●

●

●

●

●

● ●●●

● ●●●●

●

● ●●●●●

●

●

●●

●

●

●●

●

● ● ●●

●

●●●

●

●●●●●

●

●●●●●●●

●

●

●

●●

● ●●

● ●

●●

●

●●

●

●

●●

●

●●

●●●

●

●● ●●●●

●●

●

● ●●●●●● ●●

●●

●

●●

●●●

20

40

60

40 50 60Model

Raw

(B) Performance●

●

●

Average

Over

Under

●

●

●

●

● ●

●

●

●

●

●●

●

●

●

● ●

●

●●

●

●● ●

●

●

●●

●●●

●●

●

●

●●

●

●●●

●

●

●

●

●

●● ●●

●

●

●●

●●

●

●● ●

●●

●

● ● ●●● ●●

●

●●● ●

●

●

●

●

●●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●●●

●

●

●●

●

●

●●

●

●

●

●●

● ●●

●

●●

●●

●

●●

●

●

●

●

●

●●

● ●●

●●

●●●

●●

●

●

●●●●

●

●●

●●

●●

●

●●

●●● ●

●●

●

●

●

●

●

●

●●●

●

●

●●●●

●●

●

●●●

●

●●●

●●●●

●

●

●●●

●●●

●

●

●●●

●●●

● ●●●

●

●● ●●

●●

●●

●●●●

●●

●● ●

●

● ●●

●

●

●●●

●●

●● ●●●●

●

●

●

●●

●

●

●

●

● ●●●

●●●●

●

●

●●

●●●

●

● ●●●

●

●●

●● ●●

●● ●

●●● ●

●●●

●●

●●●●

●

●

●

●

● ●●●

● ●●●●

●

● ●●●●●

●

●

●●

●

●

●●

●

● ● ●●

●

●●●

●

●●●●●

●

●●●●●●●

●

●

●

●●

● ●●

● ●

●●

●

●●

●

●

●●

●

●●

●●●

●

●● ●●●●

●●

●

● ●●●●●● ●●

●●

●

●●

●●●

20

40

60

40 50 60Model

Raw

(C) Significance●

●

Once or none

Twice

Figure 3.2.15: (A) 95% prediction interval for original linear regression. Colours in-dicate schools which are considered average, statistically signi�cant over-performingand under-performing. (B) 95% prediction interval for the transformed scores.Colours indicate schools which are considered average, statistically signi�cant over-performing and under-performing. (C) Plot of which schools are signi�cantly di�er-ent in both the original linear regression model and the transformed linear regressionmodel.


si�ed into three di�erent types - boys-only, girls-only and co-education - and we

de�ne a boys-only school to be one which has the male gender recorded for over

90% of its students. The analogous de�nition is used for girls-only schools, and all

other schools are classi�ed as co-education. The χ2 values, degrees of freedom and

P -values are given in Table 3.2.15. At a 5% signi�cance level, we retain the null

hypothesis of independence between the signi�cance of schools and the covariate,

and none of the covariates seem to be outstanding in explaining the classi�cation of

schools into signi�cantly under-performing, average or signi�cantly over-performing

schools.

Table 3.2.15: Output from χ2 tests for association between the signi�cance groupsof schools and school covariates

Covariate χ2-value df P -valueisolation 23.4402 24 0.494spatial_ar 8.2187 8 0.4124staff_metr 1.7381 2 0.4194school type 6.2077 4 0.1842

For the continuous variables of gpokm and school size, the results of logistic regression

state that gpokm is not a signi�cant predictor of the signi�cance of schools (P -value

of 0.353) but school size is a signi�cant predictor (P -value of 0.00077) at the 5%

signi�cance level.

3.3 Discussion

We have established that the school-number model should not be considered as it

does not adequately model the school e�ect. The school-covariates model has its own

issues as the assumption of homoscedasticity is not satis�ed between the residuals

and school size. However, taking a suitable transformation of the school-covariates

model, we can identify the performance of schools based on mean scores and compare

how a school performs to how we expect them to perform from the model. This

mean-scores approach is similar to the comparison and ranking of statistically similar

schools on theMy School website. A more comprehensive and fundamental approach

118 3.3. Discussion

to the performance of schools is to look at a student-centric measure of performance.

Looking at how a school improves an individual student highlights the suitability

of a school for the student's characteristics and could potentially aid parents in

knowing which school is best for their child. This student-centric approach to the

performance of schools shall be discussed in Chapter 6.

The limitations of the school-number and school-covariates models can be addressed

by a hierarchical model which incorporates both models through having a school-

level regression in addition to a student-level model. The structure of a hierarchical

model can also model the high variability at the school level which we identi�ed

by the heteroscedasticity with school size. As a result, a hierarchical model is an

improvement and shall be discussed in Chapters 4 and 5.

Chapter 4

Hierarchical Modelling: Linear

Multilevel Mixed E�ects Models

The structure of students in classes in schools and then in education systems under

educational authorities naturally evokes multilevel or hierarchical modelling (Figure

4.0.1). The measured variable is the test result, and there are various factors at

the student, school and system levels. In the Basic Skills data set, we have student

covariates which are at the individual level and school covariates at the group level.

This structure of individuals within prescribed groups can be modelled using linear

multilevel mixed e�ects models. A vast array of literature can be found on mixed

e�ects models, and some selected papers and books include Gelman et al. [18],

Harville [24, 25], Laird and Ware [30], Snijders [43], Willms and Raudenbush [47],

Pinheiro and Bates [37] and Venables and Ripley [46].

The advantage of a hierarchical model is that the model combines both the school

number and the school covariates into a single model, and no information is lost at

either the student or school level. Thus, the school e�ect is explained and modelled

by the school covariates in a hierarchical model.

119

120 4.1. Theory of Linear Multilevel Mixed E�ects Models

SYSTEM

SCHOOL

CLASS

STUDENT

TEST

TEST RESULT

PARENT

Figure 4.0.1: Hierarchy of the school education system.

4.1 Theory of Linear Multilevel Mixed E�ects Mod-

els

We start with multiple linear regression as explained in Section 3.2.1 and to recap,

the matrix form of multiple linear regression is

y = Xβ + ε

ε ∼ Nn(0, σ2In)

where y = (y1, y2, . . . , yn)′ is the response vector; X is the model matrix with typ-

ical row xi = (xi1, xi2, . . . , xip); β = (β1, β2, . . . , βp)′ is the vector of regression

coe�cients; ε = (ε1, ε2, . . . , εn)′ is the vector of errors; Nn represents the n-variable

multivariate-normal distribution; 0 is an n× 1 vector of zeros and In is the order-n

identity matrix.

For any linear regression, the regression coe�cients are classi�ed as either �xed or

random e�ects. A �xed e�ects model is one in which the coe�cients are constant

across individuals [18]. The random e�ects coe�cients are modelled using proba-

bility distributions of a random variable. A mixed e�ects model is one which has a

combination of �xed and random e�ects.

As the simplest case, consider a single predictor. As stated before, students are

the individuals, and they are nested within groups, or schools, in our case. We

could �t a linear regression across all schools which collates all student information

Chapter 4. Hierarchical Modelling: Mixed E�ects Model 121

together and loses vital information about individual schools and their di�erentiating

characteristics which could help explain students' results. This is known as complete-

pooling

yi = α + βxi + εi

for student i (i = 1, . . . , N where N is the total number of students in the data)

and constants α and β. Another method would be to �t a linear model within each

school and have 426 linear models, one for each school. This is a no-pooling model

yi = αj[i] + βj[i]xi + εi

where i = 1, . . . , N , j[i] represents the group j of individual i and the αj[i]'s and

βj[i]'s are classic least squares estimates.

Instead, we use the nested structure of the data - students nested in schools - to �t

a linear regression where the intercept and/or slope can vary with each school and

incorporate the school e�ect into the model. The varying intercept and/or slope

can be modelled as a probability distribution or as a linear model itself based on

the school covariates. This is a mixed e�ects model and speci�cally, a hierarchical

model.

A mixed e�ects model is classi�ed as a hierarchical model when it is a random

coe�cient model with nested random coe�cients. The group-level model has pa-

rameters of its own which are known as the hyper-parameters of the model and are

estimated from the data. The data is structured into groups and coe�cients vary by

group. With grouped data, a regression that includes indicators for groups is called

a varying-intercept model because it can be interpreted as a model with a di�erent

intercept within each group

yi = αj[i] + βxi + εi (4.1.1)

where j[i] represents the group j of individual i. The slope can vary with constant

intercept to give the varying-slope model, and the varying slopes are interactions

between the continuous predictor x and the group-level indicators

yi = α + βj[i]xi + εi. (4.1.2)

The combination of equations (4.1.1) and (4.1.2) gives the varying-intercept, varying-

122 4.2. Hierarchical Model Formulation

slope model

yi = αj[i] + βj[i]xi + εi.

The model we consider is the varying-intercept model

yi = αj[i] + βxi + εi

where the αj's can be modelled by assigning a probability distribution, in this case,

the normal distribution

αj ∼ N(µα, σ2α),

for j = 1, . . . , J , with mean µα and standard deviation σα estimated from the data.

However, the group-level model is not solely restricted to a probability distribution

but can also be written as a separate regression in the form

αj = γ0 + γ1uj + ηj with ηj ∼ N(0, σ2α),

and uj is a group-level predictor. Adding a group-level predictor improves inference

for group coe�cients αj.

In summary, the model formulation is

yi = αj[i] + βxi + εi for i = 1, . . . , N

αj = γ0 + γ1uj + ηj for j = 1, . . . , J

εi ∼ N(0, σ2y)

ηj ∼ N(0, σ2α).

4.2 Hierarchical Model Formulation

The model we consider is one where the varying intercept represents the school e�ect

and the individual-level predictors are the student covariates. The school e�ect is

then modelled at the group-level by the school covariates. Considering an individual

student i, the multilevel model is


NNRaschi = schoolj[i] + β1procyeari + β2gradedyeari + β3atsii + β4lbotei + β5genderi

+ β6aboriginali + β7disabilityi + β8school_cari + β9occupationi

+ β10school_edui + β11non_schooli + εi

schoolj = γ0 + γ1gpokmj + γ2isolationj + γ3spatial_arj + ηj (4.2.1)

εi ∼ N(0, σ2y)

ηj ∼ N(0, σ2α)

for student i = 1, . . . , N and school j = 1, . . . , J , with N = 20 124 and J = 401.

The beauty of hierarchical modelling is that we model the student predictor vari-

ables and take into account variation by school. Therefore, we can also incorporate

the school covariates into modelling the school variation. This hierarchical model

combines both the school number and school covariates into a single model, and no

information is lost at either the student or school level.

However, in such a hierarchical model, the school is a random e�ect and is modelled

by a distribution. As a result, this hierarchical model can no longer identify school

e�ects, over and above those already modelled by the school covariates, and in

particular, signi�cant school e�ects. To identify signi�cantly under-performing or

signi�cantly over-performing schools, school would need to be treated as a �xed or

treatment e�ect in the model. This was represented in the school-number model

(Chapter 3), but the disadvantage of the school-number model was its inability to

explain school e�ect using the school covariates. The collinearity between schoolno

and the school covariates (Section 3.2.5) meant that we could not combine both the

schoolno and the school covariates into a single model. Should we continue with

the school-number model, we would now be interested in modelling students' scores

over time at the same school, rather than between schools. This would eliminate

the need for the explanatory nature of the school covariates of location as a school

does not generally change its location over time.

A solution in the form of modelling both the student and school covariates in a

hierarchical model introduces a random e�ect for school but restricts us to no longer

being able to distinctly identify under-performing or over-performing schools.

Our aim now is to use statistical modelling techniques to achieve a well-�tting model

124 4.3. Hierarchical Model Selection

which explains the relationship between NN Rasch scores and the covariates. With

this model, we hope to identify the signi�cant predictors of NN Rasch scores and

build a comprehensive and reliable model of a school's performance based at the

individual-level of a school's students' scores.

4.3 Hierarchical Model Selection

We �t this model using the linear mixed e�ects regression function lmer in the lme4

package in R. The random e�ect is schoolno and the summary lmer regression

output is given in Table 4.3.1.

We can see from Table 4.3.1 that the summary of a linear mixed e�ects model �t

by lmer provides estimates of the �xed e�ects parameters, standard errors for those

parameters and t-values, but no P -values have been given to compare and assess

the signi�cance of variables. Douglas Bates, the author of lmer, argues in [4, 32]

why it is not correct to state P -values for a linear mixed e�ects model. Bates is of

the opinion that calculating the P -values for �xed e�ects terms in a mixed e�ects

model is not reliable. That is because with unbalanced, multilevel data, the degrees

of freedom of the denominator used to penalise uncertainty are unknown, that is, we

are uncertain about how uncertain we should be. Bates contends that alternative

inferential approaches make P -values unnecessary.


Table 4.3.1: Output of �xed and random e�ects from linear mixed e�ects model

Fixed E�ects Estimate Std. Error t-valueIntercept 49.587147 0.348477 142.30procyear2001 0.155524 0.272450 0.57procyear2002 1.161221 0.260959 4.45procyear2003 0.570628 0.262967 2.17procyear2004 -0.083015 0.284865 -0.29gradedyear5 8.836288 0.120601 73.27gradedyear7 16.477341 0.418110 39.41atsi1 -2.135234 0.901806 -2.37atsiInconsistent -2.313706 0.777260 -2.98lbote1 -0.916066 0.266568 -3.44lboteInconsistent -0.501266 0.195160 -2.57genderM 1.089922 0.096272 11.32aboriginalY -1.518347 0.866750 -1.75disabilityY -8.182025 0.213560 -38.31school_carY -0.827144 0.189066 -4.37occupation1 0.771733 0.292393 2.64occupation2 0.677161 0.245864 2.75occupation3 0.673836 0.238969 2.82occupation4 0.168770 0.236223 0.71occupation8 0.079052 0.242659 0.33school_edu1 -1.723365 0.372996 -4.62school_edu2 -1.370120 0.322740 -4.25school_edu3 -0.409514 0.311441 -1.31school_edu4 0.311442 0.313765 0.99non_school5 0.653118 0.276486 2.36non_school6 1.544894 0.303423 5.09non_school7 2.274865 0.324388 7.01non_school8 0.632840 0.266736 2.37gpokm -0.007462 0.004947 -1.51isolation1.5 -0.104003 2.095854 -0.05isolation2 -2.062810 2.264896 -0.91isolation2.5 0.814912 2.251190 0.36isolation3 1.978547 2.479518 0.80isolation3.5 0.126070 2.425124 0.05isolation4 0.855229 2.589191 0.33isolation4.5 1.772173 2.908747 0.61isolation5 2.137099 3.498097 0.61isolation5.5 1.943485 3.666863 0.53isolation6 -0.043261 4.498165 -0.01isolation6.5 9.709074 7.077226 1.37isolation7 -1.888134 6.969833 -0.27spatial_ar2.2.1 0.634194 2.054559 0.31spatial_ar2.2.2 0.567557 2.270862 0.25spatial_ar3.1 1.609032 2.527954 0.64spatial_ar3.2 3.014347 2.973478 1.01

Random E�ects Name Variance Std Devschoolno Intercept 5.8022 2.4088Residual 45.2202 6.7246


4.3.1 Model Selection by Markov Chain Monte Carlo Sam-

pling

An alternative method to calculate and estimate the P -values is to use Markov Chain

Monte Carlo (MCMC) sampling. The function pvals.fnc (languageR package)

calculates the P -values which are given in the pMCMC column, and the highest

posterior density intervals for the �xed and random e�ects coe�cients (Table 4.3.2).

A 100(1 - α)% highest posterior density (HPD) interval is a region for which the

posterior probability of that region is 100(1 - α)%, and the minimum density of

any point within that region is equal to or larger than the density of any point

outside that region, for signi�cance level α. For the �xed e�ects parameters, anti-

conservative P -values based on the t-value calculated from the upper bound for the

degrees of freedom, are also included in the column P -value.

From Table 4.3.2, the signi�cant �xed e�ects are procyear, gradedyear, atsi,

lbote, gender, disability, school_car, occupation, school_edu and non_school

at the 5% signi�cance level.

Hence, the �nal model from MCMC sampling is

NN Rasch = procyear + gradedyear + atsi + lbote + gender + disability +


To compare which variables are signi�cant, it is a standard �rule-of-thumb� practice

[18] to consider a variable whose t-value has an absolute value less than two, as

signi�cant. Based on this measure, the �signi�cant� predictors in Table 4.3.1 agree

with the �nal MCMC sampling model.


Table 4.3.2: Estimates and P -values of �xed and random e�ects estimated byMarkov Chain Monte Carlo sampling

Estimate MCMCmean HPD95lower HPD95upper pMCMC P -valueIntercept 49.59 49.60 48.92 50.24 0.00 0.00procyear2001 0.16 0.15 -0.36 0.69 0.58 0.57procyear2002 1.16 1.16 0.67 1.69 0.00 0.00procyear2003 0.57 0.57 0.06 1.08 0.03 0.03procyear2004 -0.08 -0.08 -0.64 0.46 0.78 0.77gradedyear5 8.84 8.83 8.60 9.07 0.00 0.00gradedyear7 16.48 16.46 15.69 17.34 0.00 0.00atsi1 -2.14 -2.16 -3.87 -0.35 0.01 0.02atsiInconsistent -2.31 -2.35 -3.87 -0.82 0.00 0.00lbote1 -0.92 -0.92 -1.44 -0.41 0.00 0.00lboteInconsistent -0.50 -0.50 -0.89 -0.13 0.01 0.01genderM 1.09 1.09 0.91 1.28 0.00 0.00aboriginalY -1.52 -1.53 -3.20 0.16 0.08 0.08disabilityY -8.18 -8.19 -8.62 -7.79 0.00 0.00school_carY -0.83 -0.83 -1.20 -0.45 0.00 0.00occupation1 0.77 0.77 0.20 1.33 0.01 0.01occupation2 0.68 0.69 0.21 1.16 0.00 0.01occupation3 0.67 0.68 0.23 1.16 0.01 0.00occupation4 0.17 0.17 -0.30 0.62 0.47 0.48occupation8 0.08 0.06 -0.42 0.53 0.80 0.74school_edu1 -1.72 -1.75 -2.45 -0.98 0.00 0.00school_edu2 -1.37 -1.39 -2.00 -0.75 0.00 0.00school_edu3 -0.41 -0.41 -1.00 0.22 0.18 0.19school_edu4 0.31 0.31 -0.30 0.92 0.33 0.32non_school5 0.65 0.66 0.11 1.21 0.02 0.02non_school6 1.54 1.55 0.96 2.15 0.00 0.00non_school7 2.27 2.29 1.67 2.94 0.00 0.00non_school8 0.63 0.64 0.12 1.18 0.02 0.02gpokm -0.01 -0.01 -0.02 0.00 0.09 0.13isolation1.5 -0.10 -0.08 -3.92 3.62 0.97 0.96isolation2 -2.06 -2.05 -6.00 2.12 0.32 0.36isolation2.5 0.81 0.86 -3.04 5.13 0.67 0.72isolation3 1.98 2.06 -2.24 6.76 0.37 0.42isolation3.5 0.13 0.21 -3.95 4.81 0.92 0.96isolation4 0.86 0.94 -3.74 5.61 0.70 0.74isolation4.5 1.77 1.88 -3.35 7.12 0.47 0.54isolation5 2.14 2.25 -4.02 8.59 0.48 0.54isolation5.5 1.94 2.07 -4.57 8.66 0.53 0.60isolation6 -0.04 0.04 -7.88 8.18 0.99 0.99isolation6.5 9.71 9.90 -3.21 23.08 0.14 0.17isolation7 -1.89 -1.49 -13.80 11.05 0.82 0.79spatial_ar2.2.1 0.63 0.59 -3.30 4.15 0.75 0.76spatial_ar2.2.2 0.57 0.51 -3.53 4.68 0.81 0.80spatial_ar3.1 1.61 1.55 -3.16 6.07 0.50 0.52spatial_ar3.2 3.01 3.01 -2.40 8.41 0.27 0.31Groups Name Std.Dev. MCMCmedian MCMCmean HPD95lower HPD95upperschoolno Intercept 2.41 2.17 2.17 1.99 2.37Residual 6.72 6.74 6.74 6.67 6.80


4.3.2 Model Selection by Likelihood Ratio Test

One alternative inferential method to calculate P -values is to perform a likelihood

ratio test for each term in the lmer object. The function p.values.lmer (written by

Christopher Moore [31]) takes a lmer object and iteratively �ts models via maximum

likelihood which are reduced by each �xed e�ect and compares them to the full

model, yielding a vector of P -values based on χ2(1). Note that the accuracy of the

resulting P -values depends on having a large sample.

From Table 4.3.3, the signi�cant �xed e�ects are procyear, gradedyear, atsi,

lbote, gender, disability, school_car, occupation, school_edu and non_school,

the same variables as from the Markov Chain Monte Carlo sampling. This gives the

�nal model of

NN Rasch = procyear + gradedyear + atsi + lbote + gender + disability +



Table 4.3.3: Output of �xed and random e�ects from linear mixed e�ects regressionwith P -values calculated from comparing nested models �t by maximum likelihood

Fixed E�ects Estimate Std. Error t-value P -valueLRTIntercept 49.587147 0.348477 142.300000 0.000procyear2001 0.155524 0.272450 0.570000 0.566procyear2002 1.161221 0.260959 4.450000 0.000procyear2003 0.570628 0.262967 2.170000 0.029procyear2004 -0.083015 0.284865 -0.290000 0.776gradedyear5 8.836288 0.120601 73.270000 0.000gradedyear7 16.477341 0.418110 39.410000 0.000atsi1 -2.135234 0.901806 -2.370000 0.017atsiInconsistent -2.313706 0.777260 -2.980000 0.003lbote1 -0.916066 0.266568 -3.440000 0.001lboteInconsistent -0.501266 0.195160 -2.570000 0.010genderM 1.089922 0.096272 11.320000 0.000aboriginalY -1.518347 0.866750 -1.750000 0.079disabilityY -8.182025 0.213560 -38.310000 0.000school_carY -0.827144 0.189066 -4.370000 0.000occupation1 0.771733 0.292393 2.640000 0.008occupation2 0.677161 0.245864 2.750000 0.006occupation3 0.673836 0.238969 2.820000 0.005occupation4 0.168770 0.236223 0.710000 0.474occupation8 0.079052 0.242659 0.330000 0.757school_edu1 -1.723365 0.372996 -4.620000 0.000school_edu2 -1.370120 0.322740 -4.250000 0.000school_edu3 -0.409514 0.311441 -1.310000 0.187school_edu4 0.311442 0.313765 0.990000 0.321non_school5 0.653118 0.276486 2.360000 0.018non_school6 1.544894 0.303423 5.090000 0.000non_school7 2.274865 0.324388 7.010000 0.000non_school8 0.632840 0.266736 2.370000 0.018gpokm -0.007462 0.004947 -1.510000 0.123isolation1.5 -0.104003 2.095854 -0.050000 0.961isolation2 -2.062810 2.264896 -0.910000 0.352isolation2.5 0.814912 2.251190 0.360000 0.709isolation3 1.978547 2.479518 0.800000 0.411isolation3.5 0.126070 2.425124 0.050000 0.953isolation4 0.855229 2.589191 0.330000 0.732isolation4.5 1.772173 2.908747 0.610000 0.530isolation5 2.137099 3.498097 0.610000 0.530isolation5.5 1.943485 3.666863 0.530000 0.586isolation6 -0.043261 4.498165 -0.010000 0.991isolation6.5 9.709074 7.077226 1.370000 0.164isolation7 -1.888134 6.969833 -0.270000 0.788spatial_ar2.2.1 0.634194 2.054559 0.310000 0.755spatial_ar2.2.2 0.567557 2.270862 0.250000 0.803spatial_ar3.1 1.609032 2.527954 0.640000 0.518spatial_ar3.2 3.014347 2.973478 1.010000 0.300



4.3.3 Model Selection by glmulti

An R package for easy automated model selection with generalised linear mod-

els is glmulti [9, 10]. From a list of explanatory variables, the function glmulti

investigates all possible unique models including main e�ects and up to pair-wise

interaction terms. In the speci�cation of these models, restrictions can be set to

exclude speci�c terms, enforce marginality or control the complexity of the candi-

date models. The function glmulti returns the best n models and the value for the

chosen criterion, whether it be Akaike's Information Criterion (AIC) or the Bayesian

Information Criterion (BIC). One can then directly select the �best� model or build

a con�dence set of models from which model-averaged parameter estimates and pre-

dictions can be made. In addition, glmulti can not only implement exhaustive

screening but can also select models using a compiled genetic algorithm method

which is particularly useful in the event of large candidate model sets.

Applying glmulti to the following main e�ects model

NN Rasch = procyear + gradedyear + atsi + lbote + gender + aboriginal +

disability+ school_car + occupation + school_edu + non_school + gpokm

+ isolation + spatial_ar + (1 | schoolno),

where (1| schoolno) is the notation for a varying intercept for each school, a set of

eight models are deemed the best after an exhaustive search.

• NN Rasch = gradedyear + disability + occupation + non_school +

isolation + gpokm,


isolation ,

• NN Rasch = gradedyear + disability + occupation + non_school,


gpokm,


spatial_ar + gpokm,



spatial_ar,


isolation + spatial_ar + gpokm,


isolation + spatial_ar.

The Akaike Information Criterion pro�le or the ranked AIC values of the models

are plotted in Figure 4.3.1. The horizontal line is drawn at two AIC units above

the best model. A common rule of thumb is that models below this horizontal line

are worth considering as highlighted in the important points about using stepAIC

(Section 3.2.3). We see from Figure 4.3.1 that there are eight best models which are

the ones given above, and all other models have a much larger AIC value.

Figure 4.3.2 plots for each term, its estimated importance or relative evidence weight

computed as the sum of the relative evidence weights of all models in which the term

appears. We see from this plot that the test variable gradedyear and the student

variables of disability, occupation and non_school appear in all of the eight best

models and hence, have an importance of one. The school covariates of isolation,

gpokm and spatial_ar are also included in the best models but in only four out of

the eight models for each individual variable. This gives these variables a relative

importance of 0.5, and all other variables have a relative importance of zero.

When glmulti is applied to the main e�ects plus pair-wise interactions model, there

are hundreds of models with the same minimum AIC value in the con�dence set of

�best� models. Since there are too many models to sensibly analyse and interpret as

the optimal model, we achieve no further useful information from the model with

pair-wise interactions.


●●●●●●●●

●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

0 20 40 60 80 100

1356

0013

5800

1360

0013

6200

1364

0013

6600

1368

00

IC profile

Best models

Supp

ort

( ai

c )

Figure 4.3.1: Value of information criterion (AIC) for the best 100 models in theexhaustive search of the main e�ects model.

school_car

gender

aboriginal

lbote

procyear

atsi

school_edu

isolation

spatial_ar

gpokm

gradedyear

disability

occupation

non_school

Model−averaged importance of terms

0.0

0.2

0.4

0.6

0.8

1.0

Figure 4.3.2: The relative weights or importance of model terms.


4.4 Analysis of Results

From the P -values calculated by MCMC sampling and the likelihood ratio test,

we identi�ed the signi�cant variables to be procyear, gradedyear, atsi, lbote,

gender, disability, school_car, occupation, school_edu and non_school. Note

that the school covariates have large standard errors which indicate high variability

between the Rasch scores of di�erent schools. Recall from Section 2.5, that our

univariate analysis established the benchmark of six Rasch marks being equivalent

to, or worth, two years of education. Looking at the coe�cients for gradedyear5

and gradedyear7, it seems like the hierarchical model �t by lmer equates two years

of education to approximately eight Rasch marks. We can now interpret the regres-

sion coe�cients in terms of the number of years of education, for example, having a

disability is equivalent to being behind by two years of education.

As discussed in Section 4.3, model selection is not conclusive from the lmer out-

put through the usual procedure of comparing P -values to a signi�cance level. We

can use alternative methods like Markov Chain Monte Carlo simulation or the like-

lihood ratio test to calculate posterior probabilities and P -values as illustrated.

Using Markov Chain Monte Carlo simulation and sampling is essentially a Bayesian

technique, and so rather than restricting ourselves to just using Markov Chain

Monte Carlo techniques to calculate posterior probabilities, we can take advantage

of Bayesian statistics to �t a Bayesian hierarchical model. We have just considered

hierarchical modelling using linear multilevel mixed e�ects models but now con-

sider a Bayesian approach, not only to assess model �t and selection but to �t the

hierarchical model itself.

Chapter 5

Bayesian Hierarchical Modelling

We wish to apply a multilevel or hierarchical model using Bayesian techniques such

as Bayesian Inference Using Gibbs Sampling (BUGS) [36]. The challenge in �tting

multilevel models is estimating the data-level regression, along with the group-level

model. The most direct way of doing this is through Bayesian inference, a statistical

method that treats the group-level model as �prior information� in estimating the

individual- or student- level coe�cients.

The individual- or student-level coe�cients are estimated using Markov Chain Monte

Carlo simulation and sampling methods. Some common and reasonably simple

methods are the random walk Metropolis-Hastings algorithm or Gibbs sampling.

5.1 Theory

5.1.1 Bayesian Statistics

Bayesian statistics is based on Bayes' Theorem which states that the joint probability

mass or density function for parameter θ and data y can be written as the product

of the prior distribution p(θ) and the likelihood p(y|θ)

p(θ,y) = p(θ)p(y|θ).

Using the de�nition of conditional probability, Bayes' Theorem can be expressed in

135

136 5.1. Theory

terms of the posterior distribution p(θ|y)

p(θ|y) =p(θ,y)

p(y)

=p(θ)p(y|θ)

p(y)

∝ p(θ)p(y|θ).

This form of Bayes' Theorem eliminates the need to deduce the joint probability

distribution which is often quite complicated - only the prior distribution and like-

lihood need to be known. We wish to calculate the posterior distribution of the

parameters of interest, and inferences are typically summarised by random draws

from the posterior distribution.

The simplest form of Bayesian inference uses an un-informative prior, often in the

form of a prior distribution which is uniformly distributed. However, if information

is known about p(θ), an informative prior can be speci�ed and will improve the

estimate of p(θ|y).

5.1.2 Markov Chain Monte Carlo Simulation and Sampling

Markov Chain Monte Carlo (MCMC) simulation is an algorithmic method which

is based on drawing values of parameter θ from a prior distribution and then up-

dating those draws until they converge to an equilibrium distribution which is the

target posterior distribution p(θ|y). The samples are drawn sequentially, with the

distribution of the sampled draws depending on the last value drawn; hence, the

draws form a Markov Chain. Recall that a Markov Chain is a sequence of random

variables θ(1), θ(2), . . . , for which, at any step n, the distribution of θ(n) given all

previous θ's depends only on the most recent value, θ(n−1). This is known as the

memoryless property. By this method, the distributions are improved at each step

in the simulation and converge to the true distribution.

Two important cases of MCMC simulation are the Metropolis-Hastings algorithm

which takes a random walk through the space of the parameters, and the Gibbs

sampler, a special case of the Metropolis-Hastings algorithm, which updates the

parameters one at a time, or in batches, using conditional distributions.

We mainly use the Gibbs sampler for our analysis.

Chapter 5. Bayesian Hierarchical Modelling 137

Gibbs Sampling

Gibbs sampling is the name given to a family of iterative algorithms that are used in

the program BUGS (Bayesian Inference Using Gibbs Sampling) [36] (Section 5.2.2)

and other programs to �t Bayesian models. The basic idea of Gibbs sampling is

to partition the set of unknown parameters and then simulate them one at a time,

or one group at a time, with each parameter or group of parameters simulated

conditionally on all the others. The algorithm is e�ective because in a wide range of

problems, estimating separate parts of a model is relatively easy, even if it is di�cult

to see how to estimate all the parameters simultaneously.

Consider the simple two-variable case where we have a pair of random variables

(X, Y ). The Gibbs sampler generates a sample from the marginal distributions

fX(x) (and then fY (y)) by sampling instead from the conditional distribution fX|Y (x|y)

(and then fY |X(y|x)), assuming that they are easily known. This is done by gener-

ating a �Gibbs sequence� of random variables

Y ′0 , X′0, Y

′1 , X

′1, . . . , Y

′k , X

′k.

The initial value Y ′0 = y′0 is chosen, and the rest are obtained iteratively by alter-

nately generating values from

X ′j ∼ fX|Y (x|Y ′j = y′j)

Y ′j+1 ∼ fY |X(y|X ′j = x′j).

This is known as Gibbs sampling, and under certain regularity conditions, the dis-

tribution of X ′k converges to fX(x), the true marginal distribution of X as k →∞.

Thus, for k large enough, the �nal observation X ′k = x′k is e�ectively a sample point

from fX(x).

This algorithm can be extended to the estimation of a parameter vector which is

divided into components or sub-vectors. For the interested reader, texts such as

Gelman [17] and many others, explain this topic to a greater depth.

138 5.2. Hierarchical Modelling Using BUGS

5.2 Hierarchical Modelling through Bayesian Infer-

ence Using Gibbs Sampling (BUGS)

5.2.1 The Hierarchical Model

In Chapter 4, we discussed the structure of a hierarchical model in general. That

structure still applies here, but in a Bayesian hierarchical model, the regression

parameters are now estimated using Bayesian methods.

To recap, in a hierarchical model, there are the individual or student level and the

group or school level. We have the e�ect of school j being modelled and explained by

a linear regression of the spatial covariates (and potentially other school-level vari-

ables) for each school. The hierarchical model given in equation (4.2.1) of Chapter

4 is

NNRaschi = schoolj[i] + β1procyeari + β2gradedyeari + β3atsii + β4lbotei

+ β5genderi + β6aboriginali + β7disabilityi + β8school_cari

+ β9school_edui + β10non_schooli + β11occupationi + εi


εi ∼ N(0, σ2y)

ηj ∼ N(0, σ2α)

for student i = 1, . . . , N and school j = 1, . . . , J where N = 20 124 and J = 401.

It is natural to model this problem hierarchically, with observable outcomes mod-

elled conditionally on certain parameters which themselves are given a probabilistic

speci�cation in terms of hyper-parameters. More speci�cally, the di�erence between

the hierarchical model in equation (5.2.1) and a Bayesian hierarchical model is that

all parameters are now modelled by distributions, and we wish to estimate these

distributions. The regression coe�cients, βs and γs, are given independent normal

prior distributions, and the hyper-parameters of σy and σα are given independent

uniform prior distributions from which their posterior distributions are computed

using the conditional probability de�nition of Bayes' Theorem and Gibbs Sampling.

These random variables σα and σy are then simulated from their posterior distri-

butions and used in the sampling from the posterior distribution for the Bayesian


model. We assign uniform un-informative prior distributions to the hyper-parameter

standard deviations following Gelman [18].

In mathematical notation, the Bayesian hierarchical model is





εi ∼ N(0, σ2y)

ηj ∼ N(0, σ2α)

σy ∼ U(0, 1000)

σα ∼ U(0, 1000).

5.2.2 The Program

To implement this Bayesian hierarchical model, we use BUGS [36], a software pack-

age for performing Bayesian Inference Using Gibbs Sampling. In BUGS, the user

speci�es a complex, statistical model by stating the relationships between the re-

lated variables and the prior distributions for parameters. The software's system

then determines an appropriate MCMC scheme based on the Gibbs sampler for

analysing the speci�ed model. The power of BUGS is that it can break down the

analysis of arbitrarily large and complex structures into a sequence of relatively

simple computations which is then solved using a range of algorithms in BUGS.

There are two main versions of BUGS - WinBUGS and OpenBUGS. One of the main

di�erences between OpenBUGS and WinBUGS is the way in which the respective

systems make algorithmic decisions. WinBUGS de�nes one algorithm for each pos-

sible computation type, whereas there is no limit to the number of algorithms that

OpenBUGS can make use of, allowing for greater �exibility and extensibility. For

this reason, OpenBUGS is our software of choice, and we only state the OpenBUGS

results, as for every model and analysis we consider, the equivalent WinBUGS out-

put is very similar.


5.2.3 Directed Acyclic Graphs

In BUGS, models are speci�ed as Directed Acyclic Graphs (DAGs). A directed

acyclic graph is a graph formed by a collection of vertices (nodes) and directed edges

(arrows), each edge connecting one vertex to another, such that it is not possible

to return to a vertex after leaving it. DAGs are used to describe pictorially a wide

class of statistical models through describing the essential structure of the model

and the local relationship between quantities, without using numerous equations.

This is achieved by abstraction, as the details of distributional assumptions and

deterministic relationships are hidden.

The notation for a DAG in BUGS is that each quantity or variable in the model cor-

responds to a node, and edges between nodes represent direct dependencies between

variables. Rectangular nodes denote known constants, which may be in the form

of data, and elliptical nodes represent either deterministic relationships or stochas-

tic quantities which require a distributional assumption. Stochastic dependence

and functional dependence are denoted by single-edged arrows and double-edged

arrows, respectively. Repetitive structures, such as for-loops, are represented by

`panels' which may be nested if the model is hierarchical.

The DAG for our general hierarchical model in equation (5.2.2) is given in Figure

5.2.1.

From this diagram, we see that we have data for the variables of gradedyear, atsi,

lbote, gender, aboriginal, disability, school_car, school_edu, non_school,

gpokm, isolation and spatial_ar. The panels also clearly de�ne the school and

student variables and illustrate the repetitive structure of the data and model.

Although we have data for procyear, according to the theory of Rasch modelling,

procyear should not have a signi�cant e�ect on the NN Rasch scores. This issue

has been discussed previously in Section 1.2.4 and Section 3.2.4, for your reference.

As a result, we �rstly consider the model with the exclusion of procyear in equation

(5.2.3) and then incorporate it as a �xed e�ect which is normally distributed with

mean zero and a hyper-parameter standard deviation σp. The theory of Rasch

modelling justi�es the zero mean for procyear, and we see what e�ect this has on

the �t of the model in the later section of this thesis.


for(school j IN 1 : J)

for(student i IN 1 : N)

sigma.p

sigma.a

spatial[j]

isolation[j]gpokm[j]

a.hat[j]

school card[i]

occupation[i]aboriginal[i]

non school[i]

school edu[i]

gender[i]

disability[i]

lbote[i]atsi[i]

gradedyear[i]

procyear[i]sigma.ymu[i]

a[school[i]]

NN Rasch[i]

Figure 5.2.1: Directed acyclic graph for hierarchical model in equation (5.2.2).

NNRaschi = schoolj[i] + β1gradedyeari + β2atsii + β3lbotei + β4genderi

+ β5aboriginali + β6disabilityi + β7school_cari



εi ∼ N(0, σ2y)

ηj ∼ N(0, σ2α)

σy ∼ U(0, 1000)

σα ∼ U(0, 1000).

5.2.4 Analysis of BUGS output

Using OpenBUGS with three chains of 2000 iterations each and a burn-in of 1000

iterations, we �t the model in equation (5.2.3), and the summary output is given in

Table 5.2.1. Table 5.2.1 gives the estimated mean and standard deviation for the


model parameters. For these estimates, the 95% posterior density interval is given

by the 2.5% and 97.5% columns.

There are two di�culties associated with MCMC methods that are not present

with an independent sampling method. The �rst problem is determining when a

randomly initialized Markov chain has converged to its equilibrium distribution.

The second problem is whether the subsequent draws from the Markov chain are

correlated. Hence, we need to �rst answer whether the model is valid by considering

the convergence of the Markov chain, before looking at the results of the �tted

model.


Table 5.2.1: Summary output of hierarchical model without procyear using Open-BUGS

mean sd 2.5% 25% 50% 75% 97.5% R Ne�

Intercept 50.13 0.25 49.64 49.97 50.13 50.30 50.64 1.01 170gradedyear5 8.81 0.10 8.61 8.74 8.80 8.88 9.02 1.00 1 900gradedyear7 15.85 0.40 15.09 15.58 15.85 16.12 16.66 1.00 840atsi1 -2.04 0.93 -3.96 -2.64 -2.01 -1.39 -0.29 1.03 96atsiInconsistent -2.40 0.80 -3.98 -2.93 -2.39 -1.87 -0.84 1.00 600lbote1 -1.02 0.26 -1.54 -1.20 -1.02 -0.85 -0.51 1.00 1 400lboteInconsistent -0.42 0.20 -0.79 -0.55 -0.42 -0.29 -0.04 1.00 3 000genderM 1.10 0.10 0.91 1.04 1.10 1.16 1.29 1.01 450aboriginalY -1.61 0.89 -3.28 -2.23 -1.66 -1.04 0.19 1.03 100disabilityY -8.18 0.21 -8.60 -8.32 -8.18 -8.04 -7.76 1.01 330school_carY -0.92 0.18 -1.27 -1.04 -0.92 -0.80 -0.58 1.00 1 200occupation1 0.80 0.28 0.25 0.61 0.80 1.00 1.34 1.00 1 100occupation2 0.69 0.24 0.22 0.54 0.70 0.86 1.14 1.00 960occupation3 0.69 0.23 0.22 0.52 0.69 0.85 1.13 1.00 970occupation4 0.19 0.23 -0.28 0.03 0.20 0.36 0.62 1.01 340occupation8 0.09 0.24 -0.39 -0.08 0.09 0.25 0.54 1.00 570school_edu1 -1.69 0.38 -2.44 -1.95 -1.69 -1.43 -0.94 1.04 64school_edu2 -1.35 0.33 -1.97 -1.58 -1.35 -1.11 -0.70 1.05 51school_edu3 -0.40 0.32 -1.01 -0.62 -0.41 -0.17 0.22 1.06 45school_edu4 0.33 0.32 -0.28 0.10 0.33 0.56 0.97 1.06 44non_school5 0.63 0.28 0.09 0.43 0.63 0.81 1.19 1.03 86non_school6 1.51 0.31 0.91 1.30 1.51 1.72 2.10 1.02 96non_school7 2.23 0.32 1.61 2.00 2.23 2.45 2.87 1.02 110non_school8 0.61 0.27 0.10 0.42 0.63 0.79 1.15 1.02 90gpokm -0.01 0.00 -0.02 -0.01 -0.01 -0.01 0.00 1.08 33isolation1.5 0.41 1.80 -3.90 -0.65 0.52 1.66 3.63 1.38 9isolation2 -1.52 1.97 -5.77 -2.74 -1.48 -0.20 2.18 1.32 10isolation2.5 1.37 1.90 -3.09 0.23 1.52 2.73 4.66 1.41 9isolation3 2.66 2.03 -1.87 1.37 2.74 4.09 6.37 1.41 9isolation3.5 0.81 1.96 -3.57 -0.38 1.00 2.14 4.14 1.47 8isolation4 1.58 2.12 -3.03 0.22 1.81 3.12 5.03 1.44 8isolation4.5 2.54 2.48 -2.99 0.95 2.98 4.29 6.43 1.41 9isolation5 3.00 3.06 -3.52 1.09 3.36 5.20 8.18 1.27 12isolation5.5 2.73 3.15 -4.10 0.73 3.19 4.91 8.11 1.30 11isolation6 0.83 3.93 -7.44 -1.68 1.01 3.51 8.11 1.17 16isolation6.5 10.56 6.27 -2.02 6.61 10.77 14.67 22.18 1.04 63isolation7 -0.87 6.16 -13.52 -4.88 -0.58 3.44 10.36 1.15 18spatial_ar2.2.1 0.12 1.72 -2.97 -1.01 0.02 1.08 4.28 1.37 10spatial_ar2.2.2 -0.02 1.74 -3.21 -1.17 -0.13 1.07 3.66 1.42 9spatial_ar3.1 1.01 1.96 -2.45 -0.39 0.89 2.25 5.21 1.26 13spatial_ar3.2 2.61 2.45 -1.91 0.83 2.49 4.24 7.74 1.07 35σy 6.74 0.03 6.67 6.71 6.74 6.76 6.81 1.00 1 800σα 2.41 0.12 2.19 2.33 2.41 2.48 2.65 1.00 2 900


Validity of the Model: Diagnosis of Convergence

Markov Chain Monte Carlo methods generate samples from a target distribution

only after it has converged to an equilibrium distribution. In practice, we look at

diagnostics to monitor whether the Markov chains have converged. Methods to do

this include the numerical measures of R and Ne� which are given in Table 5.2.1, as

well as diagnostic plots.

Potential Scale Reduction

One way to monitor whether a chain has converged to the equilibrium distribution

is to compare its behaviour to other randomly initialised chains. This is the moti-

vation for the potential scale reduction statistic R as de�ned by Gelman and Rubin

[19]. The R statistic measures the ratio of the average variance of samples within

each chain to the variance of the pooled samples across chains. If all chains are at

equilibrium, these variances will be the same and R will be one, but if the chains

have not converged to a common distribution, the R statistic will be greater than

one.

Suppose we have M Markov Chains denoted by θm for m = 1, . . . ,M , each of which

has N simulation draws to give θ(n)m (n = 1, . . . , N). The between-sample variance

estimate is

B =N

M − 1

M∑m=1

(θ(•)m − θ(•)• )2,

where

θ(•)m =1

N

N∑n=1

θ(n)m and θ(•)• =1

M

M∑m=1

θ(•)m ,

and the within-sample variance is

W =1

M

M∑m=1

s2m,

where

s2m =1

N − 1

N∑n=1

(θ(n)m − θ(•)m )2.

The variance estimator is then de�ned to be

ˆvar+(θ|y) =n− 1

NW +

1

NB,


where y is the data. From this, the potential scale reduction statistic is de�ned by

R =

√ˆvar+(θ|y)

W.

This potential scale reduction approaches 1 as N →∞.

Looking at Table 5.2.1, the R values for the student variables are close to one,

but for the school variables, R ranges from 1.08 to 1.59. The discrepancy from a

preferred value of one indicates that the estimates of the school parameters are not

very reliable.

E�ective Sample Size

The second technical di�culty posed by MCMC methods is that the samples will

typically be autocorrelated within a chain. This leads to under-estimation of the

standard error of the posterior parameters. The amount by which autocorrelation

within Markov Chains leads to under-estimation of the standard error can be mea-

sured by the e�ective sample size.

Suppose we have the same situation of M Markov Chains denoted by θm for m =

1, . . . ,M , each of which has N simulation draws to give θ(n)m (n = 1, . . . , N). We start

with the observation that if the N simulation draws within each chain were truly

independent, then the between-sample variance B would be an unbiased estimate

of the posterior variance var(θ|y), and we would have a total of MN independent

simulations for the M chains. In general, however, the simulations of θ within each

chain will be autocorrelated. We thus de�ne the e�ective number of independent

draws or e�ective sample size as

Ne� = MNˆvar+(θ|y)

Bwith ˆvar+(θ|y) and B de�ned as for the potential scale reduction. If M is small,

B will have a high sampling variability, and Ne� is a fairly crude estimate. We

actually report min(Ne�,MN) to avoid claims that the simulation is more e�cient

than random sampling. If desired, more precise measures of simulation accuracy

and the e�ective number of draws could be constructed based on autocorrelations

within each chain.

Looking at Table 5.2.1, the e�ective sample size is reasonable for most of the student

parameters and both standard deviation parameters, but only ranges from eight to

sixty-three for the school parameters. The parallels between Ne� and R agree to give


the message that the school parameters are not accurately estimated by OpenBUGS.

Visualisation of Results

As part of the analysis of the OpenBUGS model, various diagnostic plots are pro-

duced for each of the estimated parameters. In the OpenBUGS model, the param-

eters are the model coe�cients for the student variables and the school variables,

the mean school e�ect for each school (αj) and the standard deviation for student

and school e�ect, σy and σα respectively. For each type of diagnostic plot, a brief

description and an example of an undesirable plot versus a desirable plot will be

given along with a discussion of the results.

Histograms: Histograms of the posterior distributions of all estimated model pa-

rameters, combined from all chains, are plotted to visualise the sampling distribution

of the parameters (Figure 5.2.2). All histograms are roughly symmetric with varying

means.

schoolno27

schoolno28

schoolno29

0

100

200

300

0

100

200

0

100

200

300

44 48 52 56

44 48 52 56

48 50 52 54 56value

coun

t

Figure 5.2.2: Histogram of parameters' posterior distribution.

Density Plots: Similar to the histograms, density plots of the posterior distribution

of the parameters allow for ease of comparison between chains - an individual density

is plotted for each chain (Figure 5.2.3). The density plots of all except seventeen


of the school e�ects from school 576 to school 639, are nearly identical for all three

chains.

schoolno30

schoolno31

schoolno33

0.0

0.1

0.2

0.0

0.1

0.2

0.0

0.1

0.2

0.3

42.5 45.0 47.5 50.0 52.5

48 51 54

47.5 50.0 52.5 55.0 57.5value

dens

ity

Chain

1

2

3

isolation4

isolation4.5

isolation5

0.0

0.1

0.2

0.3

0.00

0.05

0.10

0.15

0.20

0.00

0.05

0.10

0.15

−3 0 3 6

0 5

0 5 10value

dens

ity

Chain

1

2

3

(a) good (b) bad

Figure 5.2.3: Density plot of parameters' posterior distribution (each chain is adi�erent colour).

Traceplots: Traceplots give a time series representation of the estimated param-

eters. From the traceplots, we wish to see good mixing between the three chains.

Some examples are given in Figure 5.2.4, and the left hand plot is representative

of well-mixed chains as the traceplots of all three chains are plotted on top of each

other. The right hand plot though shows a parameter whose estimates are not well-

mixed as the three coloured chains are distinctly separate. Such plots are for schools

576 to 639, which corresponds to our observations about the di�erent density plots.

Running Means: Another diagnostic plot is the running means plot, which is a

time series of the running mean of a chain for each of the individual chains. The

horizontal line denotes the mean of the chain, and convergence is shown by the chain

converging to the mean line as the number of iterations increase (Figure 5.2.5). By

1000 iterations, some parameters still have not converged to the mean but are slowly

approaching convergence.


schoolno35

schoolno37

schoolno39

45.0

47.5

50.0

52.5

30

35

40

45

48

50

52

54

56

0 250 500 750 1000

0 250 500 750 1000

0 250 500 750 1000Iteration

value

Chain

1

2

3

isolation4

isolation4.5

isolation5

−3

0

3

6

0

5

0

5

10

0 250 500 750 1000

0 250 500 750 1000

0 250 500 750 1000Iteration

value

Chain

1

2

3

(a) good (b) bad

Figure 5.2.4: Traceplots of parameters (each chain is a di�erent colour).

1 2 3

49.0

49.5

50.0

50.5

51.0

46.0

46.5

47.0

47.5

46

47

48

49

schoolno54schoolno55

schoolno57

0 250 500 750 10000 250 500 750 10000 250 500 750 1000Iteration

Runn

ing M

ean

Chain

1

2

3

1 2 3

−4

−2

0

2

−2

0

2

4

−4

−2

0

2

isolation7spatial2.2.1

spatial2.2.2

0 250 500 750 10000 250 500 750 10000 250 500 750 1000Iteration

Runn

ing M

ean

Chain

1

2

3

(a) good (b) bad

Figure 5.2.5: Running means of parameters in each chain.


Autocorrelation Plots: Autocorrelation plots of each parameter, bounded be-

tween -1 and 1 are given in Figure 5.2.6. We expect an exponential decrease in

autocorrelation as lag increases. However, some parameters are very highly corre-

lated even at a lag of �fty which raises concerns about the reliability of these results

from OpenBUGS. High autocorrelation within chains is an indication that the pa-

rameter space may not be thoroughly explored by Gibbs Sampling, and so, estimates

of the parameters are unreliable. For this reason, in Section 5.3, we investigate an

alternative method which reduces the correlation in the sampling method.

1 2 3

−1.0

−0.5

0.0

0.5

1.0

−1.0

−0.5

0.0

0.5

1.0

−1.0

−0.5

0.0

0.5

1.0

schoolno50schoolno51

schoolno52

0 10 20 30 40 50 0 10 20 30 40 50 0 10 20 30 40 50Lag

Autoc

orrela

tion

Chain

1

2

3

1 2 3

−1.0

−0.5

0.0

0.5

1.0

−1.0

−0.5

0.0

0.5

1.0

−1.0

−0.5

0.0

0.5

1.0

occ4occ8

edu1

0 10 20 30 40 50 0 10 20 30 40 50 0 10 20 30 40 50Lag

Autoc

orrela

tion

Chain

1

2

3

(a) good (b) bad

Figure 5.2.6: Autocorrelation plots of parameters.


Crosscorrelation Plot: Previously, we looked at the autocorrelation plots of each

parameter in each chain - now the correlations between all the parameters are given

in Figure 5.2.7. We observe that there is high correlation between the school vari-

ables, and it occurs in blocks for both the school and student variables.

gradedyear5gradedyear7

atsi1atsiInconsistent

lbote1lboteInconsistent

genMaborY

disYschoolcardY

occ1occ2occ3occ4occ8edu1edu2edu3edu4non5non6non7non8

Interceptgpokm

isolation1.5isolation2






spatial2.2.1spatial2.2.2

spatial3.1spatial3.2

grad

edye

ar5

grad

edye

ar7

atsi

1at

siIn

cons

iste

ntlb

ote1

lbot

eInc

onsi

sten

tge

nMab

orY

disY

scho

olca

rdY

occ1

occ2

occ3

occ4

occ8

edu1

edu2

edu3

edu4

non5

non6

non7

non8

Inte

rcep

tgp

okm

isol

atio

n1.5

isol

atio

n2is

olat

ion2

.5is

olat

ion3

isol

atio

n3.5

isol

atio

n4is

olat

ion4

.5is

olat

ion5

isol

atio

n5.5

isol

atio

n6is

olat

ion6

.5is

olat

ion7

spat

ial2

.2.1

spat

ial2

.2.2

spat

ial3

.1sp

atia

l3.2

−0.5

0.0

0.5

value

Figure 5.2.7: Crosscorrelations of parameters.


Highest Posterior Density Interval (HPD) Plot: Recall from Section 4.3.1

that the highest posterior density interval is the interval of values that contains

100(1 - α)% of the posterior probability and also has the characteristic that the

density within the region is never lower than that outside. The 95% highest posterior

density intervals of the school and student parameters are plotted in Figure 5.2.8,

also known as a caterpillar plot. These could be used to assess the signi�cance of

variables depending on whether the value of zero is contained in the HPD interval.


●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

Intercept

gpokm

isolation1.5

isolation2

isolation2.5

isolation3

isolation3.5

isolation4

isolation4.5

isolation5

isolation5.5

isolation6

isolation6.5

isolation7

spatial2.2.1

spatial2.2.2

spatial3.1

spatial3.2

0 20 40Value

Par

amet

er

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

gradedyear5

gradedyear7

atsi1

atsiInconsistent

lbote1

lboteInconsistent

genM

aborY

disY

schoolcardY

occ1

occ2

occ3

occ4

occ8

edu1

edu2

edu3

edu4

non5

non6

non7

non8

0 10Value

Par

amet

er

Figure 5.2.8: Highest posterior density plot for the school and student parameters.


We could go on to interpret the coe�cients of the OpenBUGS output and their

signi�cance from the HPD intervals, but based on the validation of the model,

the estimates may not be very reliable. So, we investigate a way to avoid high

autocorrelation.

5.3 Hierarchical Modelling Using Stan

In 2012, the software Stan was released [5]. Stan is based on the No-U-Turn Sam-

pler (NUTS), an extension of the Hamiltonian Monte Carlo (HMC) method which

is a MCMC algorithm that avoids the random walk behaviour and sensitivity to

correlated parameters which a�ect the more common MCMC methods.

5.3.1 Theory

Hamiltonian Monte Carlo

For complicated models with many parameters, simple methods such as the random-

walk Metropolis-Hastings algorithm or Gibbs sampling may require an unacceptably

long time to converge to the target equilibrium distribution. This is due to the ten-

dency of these methods to explore the parameter space via ine�cient random walks.

Hamiltonian Monte Carlo (HMC) is able to suppress such ine�cient behaviour by

transforming the problem into one of simulating Hamiltonian dynamics, rather than

sampling from a target distribution.

HMC is a MCMC method based on simulating the Hamiltonian dynamics of a �c-

tional physical system in which the parameter θ represents the position of a particle

in K-dimensional space, and potential energy is de�ned to be the negative, unnor-

malised log probability. Each sample in the Markov Chain is generated by starting

at the last sample, applying a random momentum to determine initial kinetic en-

ergy, then simulating the path of the particle in the �eld. This is the evolution over

time of the Hamiltonian dynamics of this system and requires the gradient of the

log-posterior probability. Although HMC is more e�cient than standard MCMC

algorithms, the e�ciency of HMC is reliant on the choice of at least two parameters,

a step size ε and the number of steps L for which to run a simulated Hamiltonian

154 5.3. Hierarchical Modelling Using Stan

system. In particular, if L is too small, the algorithm exhibits undesirable ran-

dom walk behaviour, while if L is too large, the algorithm wastes computation. A

poor choice of either of these parameters will result in a dramatic drop in HMC's

e�ciency.

No-U-Turn Sampler

The No-U-Turn Sampler (NUTS) is a MCMC algorithm which extends HMC and

eliminates the need to set a number of steps L. Standard HMC runs the simulation

for a �xed number of discrete steps of a �xed step size, but NUTS adjusts the number

of steps on each iteration and allows varying step size per parameter. NUTS uses a

recursive algorithm to build a set of likely candidate points that span a wide swath

of the target distribution, stopping automatically when it starts to double back and

retrace its steps. NUTS uses a geometric criterion that stops a trajectory when

it begins to head back in the direction of the initial state; hence the name of no

U-turns. This prevents correlation between estimates. This criterion is based on the

dot product between the current momentum and the vector from the initial position

to the current position. Once a trajectory is stopped, NUTS uses slice sampling to

select a state along the trajectory as the next proposal. The details of the NUTS

algorithm can be found in Ho�man and Gelman [26].

NUTS eliminates HMC's dependence on the number-of-steps parameter but retains

and sometimes improves HMC's ability to generate e�ectively independent samples

e�ciently. With NUTS, it is possible to obtain the e�ciency of HMC without

spending time and e�ort hand-tuning HMC's parameters.

The advantage of NUTS is its e�ciency in e�ectively generating independent sam-

ples. A quantitative comparison of NUTS and Metropolis and Gibbs sampling al-

gorithms was done on a highly-correlated 250-dimensional multivariate normal dis-

tribution in Ho�man and Gelman [26]. The samples are generated from the data

in Ho�man and Gelman [26], and the �rst two dimensions are plotted in Figure

5.3.1. The right-most plot is of 1000 independent draws from the highly-correlated

distribution, and we observe the e�cient exploration of the parameter space. One

million samples by Metropolis and Gibbs sampling are given in the �rst and second

left plots, and the Metropolis samples are very concentrated with the Gibbs samples


being slightly more extensive. These plots were of one million samples, but with

only 1000 samples generated by NUTS (second-from-the-right plot), NUTS is able

to generate many e�ectively independent samples and explore the space relatively

well compared to Metropolis and Gibbs sampling. The extent to which the param-

eter space is explored by independent samples and those by NUTS is very similar

and supports the claim that NUTS can generate e�ectively independent samples.The No-U-Turn Sampler

Figure 5.3.1: Comparison of NUTS with Metropolis and Gibbs sampling. (the �rsttwo dimensions are plotted on the two axes) [26]

NUTS's ability to operate e�ciently without user intervention makes it well suited

for use in inference programs such as BUGS which until now, has largely relied

on much less e�cient algorithms, such as Gibbs sampling. NUTS is used as the

fundamental inference algorithm for parameters in the program Stan.

5.3.2 Results

Validity of the Model: Diagnosis of Convergence

After using Stan to �t the hierarchical model with procyear as a �xed main e�ect,

the results from three chains of 2000 iterations each, are given in Table 5.3.1. The

measures for convergence are again, R which is exactly or close to one for all vari-

ables, and Ne� is adequately large, the smallest e�ective sample size being 306. All

the diagnostic plots indicate the Markov Chains have converged to the equilibrium

distribution, and the autocorrelation is now reasonable for all variables. The diag-

A NOTE:



nostic plots are not included here, but to illustrate the reduction in autocorrelation,

Figure 5.3.2 is the plot of the previously highly-correlated variable in Figure 5.2.6(b)

but now from Stan, and we observe the improvement. The overall conclusion is that

convergence can be assumed, and the estimates are reliable.

1 2 3

−1.0

−0.5

0.0

0.5

1.0

−1.0

−0.5

0.0

0.5

1.0

−1.0

−0.5

0.0

0.5

1.0

occ2occ3

occ4

0 10 20 30 40 50 0 10 20 30 40 50 0 10 20 30 40 50Lag

Aut

ocor

rela

tion

Chain

1

2

3

Figure 5.3.2: Autocorrelation plot of parameters from Stan.


Table 5.3.1: Summary output of hierarchical model with procyear as �xed e�ectusing Stan

mean se mean sd 2.5% 25% 50% 75% 97.5% Ne� RIntercept 51.1 0.9 1.3 45.7 47.6 49.1 50.8 52.3 1 025 1.01procyear2001 0.02 0.01 0.24 -0.41 -0.14 0.01 0.18 0.54 443 1.01procyear2002 1.01 0.01 0.24 0.59 0.85 1.00 1.16 1.53 332 1.01procyear2003 0.43 0.01 0.23 0.02 0.28 0.42 0.58 0.92 306 1.01procyear2004 -0.20 0.01 0.25 -0.64 -0.36 -0.21 -0.04 0.32 381 1.01gradedyear5 8.84 0.00 0.12 8.61 8.76 8.84 8.92 9.08 1 055 1.00gradedyear7 16.39 0.01 0.42 15.58 16.10 16.39 16.68 17.21 1 006 1.00atsi1 -2.00 0.04 0.89 -3.67 -2.62 -1.99 -1.37 -0.22 506 1.00atsiInconsistent -2.30 0.02 0.77 -3.79 -2.83 -2.30 -1.80 -0.76 1 061 1.00lbote1 -0.96 0.01 0.26 -1.44 -1.14 -0.96 -0.78 -0.42 1 023 1.00lboteInconsistent -0.52 0.01 0.19 -0.91 -0.65 -0.51 -0.39 -0.15 1 484 1.00genderM 1.09 0.00 0.10 0.90 1.03 1.09 1.16 1.29 3 000 1.00aboriginalY -1.84 0.04 0.85 -3.47 -2.43 -1.86 -1.26 -0.19 504 1.00disabilityY -8.17 0.01 0.22 -8.58 -8.31 -8.17 -8.02 -7.75 1 368 1.00school_carY -0.84 0.00 0.19 -1.20 -0.97 -0.83 -0.70 -0.49 1 553 1.00occupation1 0.79 0.01 0.28 0.24 0.59 0.78 0.98 1.35 1 077 1.00occupation2 0.69 0.01 0.24 0.24 0.53 0.68 0.86 1.15 1 036 1.00occupation3 0.69 0.01 0.23 0.23 0.53 0.69 0.85 1.13 815 1.00occupation4 0.19 0.01 0.23 -0.25 0.03 0.19 0.35 0.65 804 1.00occupation8 0.12 0.01 0.24 -0.36 -0.05 0.11 0.28 0.57 911 1.00school_edu1 -1.81 0.01 0.37 -2.52 -2.06 -1.82 -1.57 -1.05 709 1.01school_edu2 -1.42 0.01 0.32 -2.03 -1.63 -1.42 -1.21 -0.79 517 1.01school_edu3 -0.47 0.01 0.30 -1.06 -0.67 -0.47 -0.27 0.17 453 1.01school_edu4 0.27 0.01 0.31 -0.33 0.05 0.27 0.47 0.89 447 1.01non_school5 0.70 0.01 0.27 0.17 0.51 0.70 0.87 1.23 379 1.01non_school6 1.60 0.01 0.30 1.00 1.40 1.59 1.80 2.18 411 1.01non_school7 2.32 0.02 0.31 1.68 2.11 2.33 2.52 2.92 363 1.01non_school8 0.66 0.01 0.26 0.17 0.50 0.67 0.84 1.18 373 1.01gpokm 3.10 0.24 9.55 -15.51 -3.43 3.17 9.35 22.28 1 539 1.00isolation1.5 2.34 0.26 9.30 -16.34 -3.76 2.46 8.53 20.48 1 237 1.01isolation2 3.12 0.30 10.10 -16.26 -3.68 2.97 9.86 23.30 1 171 1.00isolation2.5 2.51 0.32 9.04 -15.47 -3.40 2.38 8.82 19.75 802 1.00isolation3 2.55 0.26 9.85 -16.12 -4.18 2.27 8.85 22.83 1 397 1.00isolation3.5 3.10 0.26 9.74 -15.17 -3.60 3.22 9.75 22.32 1 396 1.00isolation4 2.50 0.33 9.59 -16.08 -4.24 2.62 8.81 21.81 863 1.00isolation4.5 2.98 0.37 10.20 -17.43 -3.75 3.01 9.77 23.09 775 1.00isolation5 2.75 0.29 10.13 -16.49 -4.25 2.32 9.50 23.35 1 184 1.00isolation5.5 2.00 0.30 9.73 -17.60 -4.52 1.92 8.68 21.93 1 052 1.01isolation6 2.82 0.31 9.51 -15.70 -3.77 2.98 9.12 21.69 916 1.00isolation6.5 2.96 0.28 9.21 -15.02 -3.25 2.88 9.20 20.42 1 080 1.00isolation7 2.93 0.25 9.86 -15.78 -3.96 2.86 9.75 22.01 1 550 1.00spatial_ar2.2.1 2.68 0.30 9.85 -16.34 -3.85 2.47 9.06 22.62 1 051 1.00spatial_ar2.2.2 2.95 0.29 10.04 -17.86 -3.36 3.10 9.38 22.22 1 166 1.00spatial_ar3.1 2.67 0.27 9.54 -15.90 -3.75 2.74 9.11 21.13 1 214 1.00spatial_ar3.2 3.01 0.29 9.57 -15.96 -3.47 3.31 9.21 21.57 1 109 1.00σα 2.47 0.00 0.10 2.31 2.40 2.47 2.54 2.69 1 416 1.00σy 6.73 0.00 0.03 6.66 6.70 6.73 6.75 6.80 3 000 1.00σp 0.92 0.02 0.64 0.34 0.55 0.74 1.08 2.44 805 1.00


Interpretation of Regression Coe�cients

Looking at the regression coe�cients (Table 5.3.1), we can determine which predictor

variables are signi�cant by considering the 95% highest posterior density interval

given by the 2.5% and 97.5% bounds and seeing whether zero is contained within the

interval. Based on this, the variables of procyear, occupation and school_edu are

signi�cant, as well as all the school covariates, gpokm, isolation and spatial_ar.

However, all the school covariates have large standard errors, creating 95% HPD

intervals with widths of approximately 37 units.

Comparing the Stan regression coe�cients (Table 5.3.1) to those given by lmer

(Table 4.3.1), the student predictor variables of gradedyear, atsi, lbote, gender,

aboriginal, disability, school_car, occupation, school_edu and non_school

have very similar estimates, some of them di�ering at the second decimal place.

The school predictor variables, however, vary greatly between lmer and Stan, but

recall that the standard errors are very high and notice that the estimated lmer

coe�cients are contained within the 95% HPD intervals. We can also estimate the

between-students and between-schools standard deviation by σy = 6.73 and σα =

2.47 respectively.

Recall from Section 2.5 that our univariate analysis established the benchmark of six

Rasch marks being equivalent to two years of education. Looking at the coe�cients

for gradedyear5 and gradedyear7, it seems like it is now approximately eight Rasch

marks which are equivalent to two years of education. We can now interpret the

regression coe�cients in terms of years of education.

Chapter 6

Model Validation

At this point, we have �t the hierarchical model





εi ∼ N(0, σ2y)

ηj ∼ N(0, σ2α)

for student i = 1, . . . , N and school j = 1, . . . , J where N = 20 124 and J = 401,

using two di�erent methods - lmer's linear multilevel mixed e�ects and Stan's No-

U-Turn sampler. In Chapter 5, we �tted the models, assessed model �t and the

signi�cance of variables, and interpreted regression coe�cients. However, we wish

to validate the models and use them to predict students' scores.

6.1 Student-level Prediction

One method of assessing the e�ect of school is to analyse the performance of the

students in the school. From our �tted hierarchical model, suppose we calculate a

�tted score for a student. The error associated with this �tted score is represented

by a prediction interval calculated for a student's score in their speci�ed school and

with their student characteristics as given in the data set. In the education system,

159

160 6.1. Student-level Prediction

we would expect a narrow expected range of marks around a student's observed

score. As a result, we calculate a 50% prediction interval, compare the observed

NN Rasch score to the 50% prediction interval, and say that if a student's observed

score lies below the lower bound, they are under-performing, if their observed score

lies above the upper bound, they are over-performing and if their observed score

lies within the interval, they are performing as expected. This classi�es a student

as lying under, over or within their prediction interval, and for each school, the

number of students in each of these categories can be counted. From the de�nition

of prediction intervals, we expect approximately 50% of students to lie within their

50% prediction intervals.

6.1.1 Prediction Intervals for lmer

To calculate the prediction intervals for students from the lmer model, �rst recall

the statistical theory of prediction intervals based on linear regression.

Consider the model

Y = Xβ + Zb+ ε

where Xβ represents the �xed e�ects of the student variables, Zb represents the

random e�ects of the school variables and ε is noise. From the lmer output, we

have the estimated parameters β, b, ˆV ar(b) and ˆV ar(ε). Suppose we wish to

predict values for an individual with data X0 and Z0. The predicted value would be

given by

Y0 = X0β + Z0b+ ε

and the estimated expected value is

E[Y0] = X0β + Z0b.

Assuming independence, the estimated variance is

ˆV ar(Y0) = V ar(X0β + Z0b+ ε)

= V ar(X0β) + V ar(Z0b) + V ar(ε)

= X0V ar(β)XT0 + Z0V ar(b)Z

T0 + V ar(ε)

where we have estimates for V ar(β), V ar(b) and V ar(ε). The approximate 50%

prediction interval is taken to be 0.67 standard deviations from the predicted value

(Y0 − 0.67

√ˆV ar(Y0), Y0 + 0.67

√ˆV ar(Y0)).

Chapter 6. Model Validation 161

The cut-o� value of 0.67 corresponds to a 25%-50%-25% strati�cation of a standard

normal distribution.

From the hierarchical model �tted in lmer (Section 4.3), we calculate prediction

intervals for each student in our data set. In reference to the statistical theory

outlined above, the data for the predicted values is X0 = X since we are predicting

for the same data on which the model was �tted. We then classify each student into

under-performing, over-performing or within, depending on whether their observed

NN Rasch score falls below, above or in the 50% prediction interval. The counts

and proportions of students in each of the three categories is given in Table 6.1.1.

Table 6.1.1: Counts and proportions of students in each performance category fromthe lmer model

Category Count ProportionOver 4 429 0.22Within 11 131 0.55Under 4 564 0.23

From the column of proportions in Table 6.1.1, we observe that approximately 50%

of students' observed scores fall within their 50% prediction bands which is to be

expected. It is important to note that the �z-score� of 0.67 is only an approximation,

as it is not reasonable to calculate a t-test statistic for the same reason why P -values

are not calculated in lmer (Section 4.3).

6.1.2 Prediction Intervals from Stan

To calculate prediction intervals for students' scores in Stan, we simulate 3000 scores

from the model for each of the 20 124 students in the data set. These simulations

give a posterior distribution for the score of each student, and from the posterior

distribution, we take the 50% posterior density interval as the 50% prediction interval

for a student's score.

The same classi�cation of students in Section 6.1.1 as under-performing, over-performing

or within, is applied to the observed data and the prediction intervals from Stan.

Table 6.1.2 gives the counts and proportions of students in each of the three cate-

gories.


Table 6.1.2: Counts and proportions of students in each performance category fromthe Stan model

Category Count ProportionOver 4 653 0.23Within 10 686 0.53Under 4 785 0.24

As before, approximately 50% of students' observed scores fall within their 50%

prediction bands.

6.1.3 Comparison of lmer and Stan

We have results for the performance of students from lmer and Stan and now wish

to assess model agreement. Table 6.1.3 gives the 3×3 table of the counts of studentsunder each of the performance categories for both models.

Table 6.1.3: 3×3 table of the counts of students for performance categories underlmer and Stan models

Stanlmer Over Within UnderOver 4 403 26 0Within 250 10 641 240Under 0 19 4 545

The values along the diagonals of Table 6.1.3 are those students who are classi�ed

into the same group under both models. The o�-diagonal entries are of interest as

they are students for which their classi�cation di�ers depending on lmer or Stan.

This identi�es a di�erence between the models in the calculation of �tted values or

prediction intervals, and hence, the classi�cation of certain students.

From the calculated prediction intervals, we obtain the number of students in the

performance categories of Over, Within and Under for each individual school. We

then use this data and a χ2 test to identify whether the school has a signi�cant e�ect

on the number of over-performing, under-performing and within students. A more


appropriate test would be Fisher's exact test because of the small counts for some

variables in schools, but Fisher's exact test is intractable for the large 3 × 401 table

created by three performance categories and 401 schools.

For the lmer model, Pearson's χ2 test gives a test statistic of 688.92 and a P -value of

0.9981. Similarly for the Stan model, Pearson's χ2 test gives a test statistic of 680.93

and a P -value of 0.9991. We conclude that school is not a signi�cant predictor of

the counts in each of the performance categories since we retain the null hypothesis

at a 5% signi�cance level for both models.

Another way to compare models is to compare the �tted values for student scores

under the lmer and Stan models. Figure 6.1.1 plots the Stan �tted values against

the lmer �tted values, and we observe the close alignment between the �tted values.

The colours of Figure 6.1.1 are de�ned from Table 6.1.3 where students are coloured

depending on whether they are classi�ed as Over (O), Within (W) or Under (U) by

the lmer (L) and Stan (S) models.

The top plot in Figure 6.1.1 is of all students who have no discrepancy in classi�ca-

tion, and the data points lie very close to the line of equality. Looking at the bottom

plot which contains the students whose classi�cations di�er, we see that the �tted

values of lmer and Stan agree well in most cases, with only a few points standing

out. These plots indicate that the lmer and Stan models agree in general in terms

of �tted values for students. It is the calculated prediction intervals which appear

to change the classi�cation of students, and since we have commented on the fact

that the lmer prediction intervals are only approximate, we choose the Stan model

over lmer, and all further analysis is with the results from the Stan model.


●●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

● ●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●

●●●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●●●

●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●●●

●

●

●

●

●

●●

●●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●●

●●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●●

●

●

●

●●

●●

●

●

●●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●●●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●●

●

●

●

●●

●

●●●

●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●●

●

●

●

●●●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●●

●

●

●

●

●

●●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●●

●●●●

●●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●●●●

●

●

●

●

●

●

●●●●

●

●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●●

●●

●●

●●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●

●●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●●

●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●●

●

●●

●●

●

●

●

●

●

●●●

●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●●●

●

●

●

●●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●

●●●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●●

●●

●●

●

●

●

●

●

●●●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●

●

●

●

●

●

●

●●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●●●

●

●●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●●

●

●

●

●●●●

●

●

●

●●●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●●●

●

●

●●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●●

●●

●●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●●

●

●

●●

●●

●

●

●

●

●

●

●●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●●●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●

●

●

●

●●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●●

●●

●

●

●

●●●

●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●●

●●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●●●

●

●

●●

●

●●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●●●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●●

●

●●

●

●●

●

●

●●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●●

●●●●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●●●

●

●

●

●●

●

●

●●

●

●●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●●●●

●

●

●

●

●●●

●

●●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●●●

●

●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●

●

●

●●

●●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●●●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●●●

●

●●

●

●

●

●

●

●●●

●

●

●●●●

●

●

●

●●

●

●

●●

●

●

●●

●●

●

●

●●

●

●

●●●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●●●●

●

●

●●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●●

●

●

●

●

●

●

●●●

●

●

●●●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●●

●

●

●●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●●●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●●●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●

●●

●

●

●●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●●

●

●

●●

●

●

●

●

●

●

●

●●●●

●●

●

●

●

●

●●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●●●●

●

●

●●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●●

●●

●

●●●●

●

●

●●

●

●

●

●

●●

●●

●

●

●

●

●

●●●●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●●

●

●

●●●●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●●

●

●

●

●●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●●●

●●

●●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●●

●●

●

●

●

●

●

●●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●●

●●

●

●●●

●

●

●

●●

●●

●

●

●●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●●

●

●

●●●

●

●

●●

●

●

●●●

●

●

●●

●

●

●●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●●●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●●

●●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●●●

●

●●

●

●

●

●

●

●

●●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●●●●

●

●●

●

●

●

●

●●

●●

●●●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●●●

●●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●●

●●●●

●

●

●

●

●●

●

●●

●●

●

●

●

●●

●

●

●

●

●●

●

●●●

●

●

●

●

●

●

●●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●●

●

●

●

●●

●

●●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●●

●

●

●●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●●

●

●

●●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●●●●

●●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●●

●

●●●

●

●

●●

●●

●

●●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●●

●

●

●●

●

●

●●

●

●

●

●

●●

●●●

●●●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●●

●

●●

●

●

●●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●●●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●●

●●

●●

●●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●

●

●

●

●●●

●

●

●

●●

●

●●●

●

●●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●●

●

●●

●

●

●

●

●

●●●

●

●

●●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●●●●

●

●

●

●●

●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●●

●

●●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●●

●

●●

●

●

●

●

●

●

●●●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●●

●

●●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●●●●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●●●

●●

●

●●

●●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●●●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●●

●

●●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●●

●●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●●●

●

●●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●●●

●

●

●

●

●●●●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●●

●

●

●

●

●

●

●

●

●●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●●

●

●

●

●

●

●

●●

●●

●

●

●

●●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●●●

●

●●

●

●●

●●●

●●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●●●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●●●

●●

●

●●●●●●●

●●●●●

●

●

●●

●

●●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●●

●●●●

●●

●●●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●●●

●

●●●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●●●

●

●

●●

●

●●●

●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●●●

●

●

●

●

●

●

●●

●●

●

●

●●●●

●

●●

●

●●

●

●●

●●

●●●●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●●

●●

●

●

●●

●●

●

●

●●

●●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●

●

●

●

●

●●

●●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●●

●●

●●

●

●●

●

●

●●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●●●●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●●

●

●●●

●

●

●●●

●●●

●●●

●

●

●●●●

●

●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●●

●

●

●●

●●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●●

●

●

●

●

●●●

●

●●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●●

●

●●

●

●●

●

●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●●●●

●

●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●●●●

●●

●

●

●

●

●●●

●●

●●

●

●

●●

●

●

●

●

●

●●

●

●

●●

●

●●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●●●

●

●●●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●●●

●

●

●

●●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●●

●

●

●

●

●

●●

●

●

●

●

●

●

●●●

●●

●

●

●

●●

●

●

●

●

●●

●●

●●●

●●●

●

●●

●

●

●

●

●

●

●

●

●●●

●●

●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●●

●●

●

●

●

●

●

●

●●

●

●

●●

●●●

●●●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●●

●

●●●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●●

●

●●●

●

●●

●

●

●

●

●

●

●

●●

●●●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●

●●●

●

●●

●

●●●

●

●

●●

●

●●

●

●

●●●

●

●

●●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●●

●

●●●

●

●

●●

●

●

●

●

●

●●

●●●

●

●●●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●●●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●●●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●●

●

●●

●●

●

●

●

●

●

●

●●●

●

●●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●●

●

●

●

●

●

●●●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●●

●●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●

●

●

●●

●●●●●

●

●

●

●

●

●

●

●

●●●●

●

●

●

●

●

●

●●●

●●

●●●

●●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●●

●●

●●●

●

●

●●

●

●

●

●

●●●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●●

●●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●

●

●●●

●●

●

●●

●

●●

●

●

●

●

●●●

●

●●●

●

●

●●

●●

●●

●

●●

●

●

●

●

●

●●●

●

●

●●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●●

●

●

●

●

●

●

●

●●●●

●

●

●●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●●

●●

●

●

●

●

●

●●

●

●

●

●●●

●●●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●●●

●●

●●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●●●

●●●

●●●

●

●●

●●●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●●●

●

●

●

●●

●

●●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

●

●●●●●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●●

●●

●

●

●

●●●

●●●

●

●

●

●

●●●●

●●

●

●●●

●●

●

●

●

●

●●●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●●

●●

●

●●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●●

●

●●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●●

●

●

●

●●

●

●●

●

●●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●●

●

●

●

●

●

●●

●

●●

●

●

●

●●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●●

●

●●

●●

●

●

●

●●

●

●

●●

●

●

●

●●●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●●

●

●

●

●●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●●

●

●

●

●

●●

●

●●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●●

●

●

●●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●●●

●

●

●

●

●●

●

●

●●

●●

●●

●

●

●

●

●●

●

●

●

●●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●

●●

●

●

●

●●

●

●

●

●●

●

●

●

●●

●

●

●

●●●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●●

●●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●●

●

●●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●●

●

●●

●●

●

●

●

●●

●

●

●

●

●●

●

●●

●●

●

●

●

●

●

●

●

●

●●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●●●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●●

●

●

●

●●

●

●

●●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●●

●

●

●

●

●●●

●●

●

●●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●

●●

●●●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●●

●

●●

●

●

●

●●

●●

●●

●●

●

●

●●●●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●●

●

●

●

●

●●●●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●●

●●

●

●●

●

●

●●

●

●●●

●

●●

●●

●

●

●

●

●

●●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●●●

●

●

●

●

●●●

●●

●●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●

●●●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●●

●●

●●

●

●●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●●

●

●●●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●●●

●

●●

●

30

40

50

60

70

30 40 50 60 70lmer fitted values

Stan

fitt

ed v

alue

s

Categories●

●

●

SOLOSULUSWLW

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●

●

●

●●

●

●

●●

●●

●●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●●●

●●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

30

40

50

60

70

30 40 50 60 70lmer fitted values

Stan

fitt

ed v

alue

s

Categories●

●

●

●

SOLWSULWSWLOSWLU

Figure 6.1.1: Plot of the Stan �tted values against the lmer �tted values colouredby classi�cation cateogories.


6.2 Analysis of School E�ect

We have divided students into three categories - over-performing, within and under-

performing. If we look more closely at the proportion of students in these categories

for each school, we can rank schools in order of their proportion of over-performing

students. Figure 6.2.1 plots these ranked non-zero proportions and the 95% con�-

dence intervals of the proportion.

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

100

200

300

400

−0.2 0.0 0.2 0.4 0.6 0.8 1.0 1.2Proportion of Students

Ran

k

Stan: Over−performing Students of Schools

Figure 6.2.1: Ranked proportion of students who are over-performing in a schoolplotted with a 95% con�dence interval. The line y = 0.25 is given for reference andthe colour green highlights schools whose con�dence interval lies completely above0.25 while the colour red highlights schools whose con�dence interval lies completelybelow 0.25.

We would expect that approximately 25% of students in a school would be over-

performing, deduced from the 50% prediction interval, and a vertical line is drawn

at 0.25 to illustrate how schools perform with respect to this expectation. If a

school's 95% con�dence interval lies above the line at 0.25, we can conclude that that

166 6.2. Analysis of School E�ect

●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●●

100

200

300

400

−0.2 0.0 0.2 0.4 0.6 0.8 1.0 1.2Proportion of Students

Ran

kStan: Under−performing Students of Schools

Figure 6.2.2: Ranked proportion of students who are under-performing in a schoolplotted with a 95% con�dence interval. The line y = 0.25 is given for reference andthe colour red highlights schools whose con�dence interval lies completely above 0.25while the colour green highlights schools whose con�dence interval lies completelybelow 0.25.

particular school has a signi�cant proportion of over-performing students. These

schools are highlighted in green in Figure 6.2.1, and there are three such schools.

Note that these three schools have over-performing students as their total student

population, giving a proportion of one and a non-existent con�dence interval. In

the case where the proportion of over-performing students is either zero or one, the

exact binomial calculation can be used instead.

If a school's 95% con�dence interval for the proportion of over-performing students

lies completely below the reference point of 0.25, we conclude that that particular

school has a signi�cantly low number of over-performing students. These schools

are coloured red in Figure 6.2.1, and there are 43 such schools.


We also consider the proportion of under-performing students in schools in Figure

6.2.2. There are four red schools with a signi�cantly higher proportion of under-

performing students, and 32 green schools which have a signi�cantly lower proportion

of under-performing students. The four red schools are small schools, and either

75% or all the students are under-performing. Minimising the number of under-

performing students in a school is an aim, like having a signi�cant number of over-

performing students.

In Section 6.1, we performed a χ2 test on all schools and their counts of Over, Within

and Under students. The P -value of 0.9991 led us to the conclusion that school

does not have an overall signi�cant e�ect on the strati�cation of students in the

performance categories. We then look at a χ2 test on the proportion of counts being

(Over, Within, Under) = (0.25, 0.5, 0.25) in each individual school. This is to assess

the goodness of �t of a multinomial distribution for the proportion of counts in each

school. To perform multiple hypothesis testing, there are a number of methods, one

of which is to calculate Bonferroni adjusted P -values. Bonferroni adjusted P -values

correct for familywise error rate, but are very conservative. Another method is to

convert the P -values to q-values which take into account the positive false discovery

rate (see Storey [44] for the statistical theory of q-values). The calculated q-values

are all larger than 0.05, so at the 5% signi�cance level, we conclude that there is

no signi�cant association between school and the number of over-performing, within

and under-performing students.

This method answers the question of whether the proportion of over-performing or

under-performing students in a school is signi�cantly di�erent from the expected

value of 0.25. From the χ2 test however, we cannot conclude that signi�cant pro-

portions are due to a school e�ect. This then provides the avenue of investigating

more subtle issues and the e�ect of other student and school covariates.

The conclusion that school does not have a signi�cant e�ect is an indication that the

model is working and �tting the data well. The validation of the model con�rms that

as we expect, schools are performing as they should, and the model has satisfactorily

captured and explained the relationship between students' NN Rasch scores and their

student and school characteristics. As stated, each school has a random e�ect to

explain its e�ect on a student's expected performance, the variation of this random

168 6.2. Analysis of School E�ect

e�ect (standard deviation of 2.41) could be further explained by predictive factors

not presently contained in the data set, for example, pedological philosophies of the

teachers at the schools. This unexplained random e�ect varies between schools and

is de�ned to be γ0 in equation (6.0.1), that is, the di�erence in the observed and

estimated school-level linear regression. These γ0s can be extracted from the model

�t and a normal Q-Q plot of the random e�ects estimates is given in Figure 6.2.3.

●

●

●●

●

●●●●●●●●

●●●●●

●●●●●●●

●●●●●●●

●●●●●●●●●●

●●●●●●●●●●

●●●●●●●●●

●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●

●●●●●●●●●●●●●

●●●●●●●●●●●●●●●

●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●

●●●●●●●●●●●●●●●●

●●●●●●●●●●

●●●●●●●●●

●●●●●●●●●

●●●●●●●●

●●●●●●●

●●●

●

●●

●

●

−4

0

4

8

−2 0 2Theoretical Quantiles

Sam

ple

Qua

ntile

s

Figure 6.2.3: Normal Q-Q plot of the random e�ects estimates.

We observe that the points in the normal Q-Q plot follow quite a strong linear

relationship, especially for the majority of the points. There is the possibility of

the highest, rightmost point being an outlying school, and such a school could be

investigated further.

The fact that the random e�ects for school are normally distributed, as assumed for

the model, is further con�rmation that we have a well-�tting and accurate model.

We wish to explain this currently �unexplained� random e�ect, and to do so, would


require more data. Ideally, the most meaningful data would be data on variables

which can be changed and controlled. Our current data is of variables over which

students, schools or education authorities have no control, for example, they cannot

change or control the disability status of a student, their parental occupation or the

location of the school. It would be of great interest to have policy-controlled data,

for example, the level of sta�ng at a school, teacher mobility or the implementation

of a particular teaching program, so that we can analyse the e�ect these investments

and controlled factors have on student performance.

6.3 Heteroscedasticity and School Size

In Section 3.2.5, we looked at the heteroscedastic nature of the residuals from the

model �tted on transformed data and school size. A similar analysis is done here

where the variance of the student residuals for each school is plot against school size

(Figure 6.3.1). For each school, the variance of the student residuals is de�ned to

be

MSE =

∑nj

i=1(yi − yi)2nj − 1

for student i = 1, . . . , nj and school j = 1, . . . , J where nj is the number of students

in school j, yi is the observed student score and yi is the �tted or expected student

score from the model.

There is a distinct funnel-shape to the plot which indicates heteroscedasticity be-

tween the residuals and school size. This relationship between the residuals and

school size suggests that we need to account for school size in the model. One

solution is to stratify schools into small schools and large schools and then �t the hi-

erarchical model on each of these groups of schools separately to reduce the in�uence

of school size. Another idea would be to incorporate school size into the original

model to hopefully weight the model or compensate for the di�erence in school size.

Finally, the predicted quartile interval for the students of a school may be adjusted

by the school's variance of residuals to account for the variation between schools.

All of these suggested methods are avenues for future research in addressing this

heteroscedasticity.

170 6.3. Heteroscedasticity and School Size

●

●●●

●

●●

●●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●● ●

●●●

●

●

●

●●

● ●

●

●

●●●

●●

●● ●

●

●

●

●

●

●

●

●

●●

●●

●●

●

●

●● ●

●●

● ●

●

●

●

●

●

●

●

●

● ●

●

●●

●

●●

●

●

● ●●

●

●

●●

●

●●

●

●

●

●●

●

●

●

●

●●

●

● ●

●● ●●

●

●

●●

●

●

● ●

● ●●●

●

●●

●

●

●

●● ●

● ●● ● ●●

● ●●●

●

●

●●

●

●

●

●●

●●●

●●

●●

●●

●●

●

● ●

●●

●●

●●

●●

●

●●

●●●

●

●

●

●●●

●● ●

●

●

●

●

●●

●●

●

●●

●

●●

●●●

●● ● ● ● ●●

●

●

●

●●

●

●

● ● ●●

●●

●●●

●● ● ●●● ●

●

●●●

●

●●

●

●

●●

●● ●●

● ●●

●●

●●● ●

●●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●●● ●

● ●●●

●

●

● ●●

●●●

●●●

●●

● ●●

●

●

● ● ●●●●●

●●●●

●●

●●

●●

●●

●●● ●●●

●

●● ●

●

●●●

●●

●

●● ●

●●

●

●●

●

●

●● ●● ●

●●

●●

●

●

●●

●●●

●●

●

●●● ● ●●

●●

●● ●

●●

●●

●●

●

●●

●

●

●●

●●●

0

200

400

600

800

0 50 100 150 200School Size

Var

ianc

e of

Res

idua

lsStan

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

● ●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●● ●

●

●

●

●

●

●

●

●●

●●

●●

●

●

●

●

●

●

●

●●

●

●● ●●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●● ●

● ●

●

●

●

●

●

●

●

●

● ●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

● ●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●

●

●

●

●

● ●

●

●●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

0

50

100

150

0 50 100 150 200School Size

Var

ianc

e of

Res

idua

ls

Stan

Figure 6.3.1: Plot of the variance of the residuals for each school against school size(the bottom plot has the point at (1,778) removed to zoom in on the other points).


6.4 Conclusion and Impact

It is also important to note that the classi�cation of students can highlight the

exceeding of expectations due to background factors. An over-performing student is

not necessarily the student with the highest score in a school; it could be a student

who has a socially disadvantaged background, but their scores are not hindered and

they are classed as over-performing. Figure 6.4.1 gives the example of school 115

where there is a student who is over-performing with a score of 67.34.

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

40

50

60

70

80

StanModel

Scor

e Indicator●

●

●

WithinOverUnder

Figure 6.4.1: Student scores for school 115 coloured by whether the students areclassi�ed as over-performing, within or under-performing.

From this analysis, we can accurately predict a student's score given their student

and school characteristics. This student-centric approach would be of particular

use to anyone who wishes to determine which school is best for an individual, for

example a parent for their child, as we can calculate their predicted score for each

school under the �nal Stan model and compare results.

172 6.4. Conclusion and Impact

By the end of this chapter, we have achieved a model which is the �best�, out of

all we considered. The di�erence between the results which are reported from NA-

PLAN scores by the My School website's rankings is that for each school, My School

agglomerates all the students' results into a single measure or average and then

compares the school to other �statistically similar� schools. In stark contrast, our

methods takes an individual student, compares them to other students, classifying

them into over-performing, within or under-performing and then identi�es whether

a school has a signi�cant proportion of students in each category. The power of this

model is its emphasis on students and their individual performance to give a person-

alised model aggregated to schools. This makes the model very versatile as it models

and assesses the performance of students and schools, based on the performance of

their students.

Chapter 7

Initial Longitudinal Analysis

In Chapter 1, we discussed the motivation for value-added measurement. This

concept is related to the longitudinal aspect of the data. It is the usual practice for

students to remain in the school system and progress from Grade 3 to Grade 5 to

Grade 7, and during the course of their education, sit all three tests. Hence, there is

an underlying commonality or correlation between tests if they are sat by the same

person. Longitudinal modelling incorporates this extended structure of repeated

measures over time. For these students, we can take the di�erence in sequential

scores and analyse the improvement of students over time to see whether the �value�

added to their education is due to a school e�ect.

In the Basic Skills Test data set, individual students are uniquely identi�ed by the

combination of their school ID number and student number (Section 2.2). As a

result, we cannot track students who move between schools but are restricted to

tracking students only within a school.

7.1 Summary Statistics of Data from Sequential Tests

Students can either sit two sequential tests - Grades 3 and 5 or Grades 5 and 7 -

or three sequential tests - Grades 3, 5 and 7. First, we clean the data by removing

students who have multiple scores recorded for the same grade. For example, a

student has a Grade 3 result in 2000 and a Grade 3 result in 2001 as they have

re-sat the Grade 3 test. As we are primarily interested in the progress of students

173

174 7.2. Grade 3 and Grade 5 Tests

between sequential tests, that is, between Grades 3 and 5 and Grades 5 and 7, we

disregard these repeated scores. Some students have scores recorded for Grade 3

and Grade 7 but not Grade 5. The di�erence in these scores are not comparable

to the di�erence in scores of sequential tests and so are removed from the data set.

Finally, there are 31 students who sit sequential tests which do not occur biennially,

for example, they skip a year and sit tests in consecutive years, or they are held

back a year and sit another test three years after the previous one. These students

are also removed during the process of cleaning the data.

From the cleaned data set, 8 357 students from 401 schools have two appropriate

sequential test entries, and the break-down into whether they are Grade 3 and Grade

5 scores or Grade 5 and Grade 7 scores is given in Table 7.1.1.

Table 7.1.1: Counts of students who sat two appropriate sequential tests

Type CountsGrades 3 & 5 8 143Grades 5 & 7 214

Total 8 357

In the cleaned data set, there are now 487 students from 158 schools with three

appropriate recorded scores. These scores can be broken down into two sets of two

sequential tests, or by taking a more fundamental longitudinal analysis approach,

another recorded score at a third time point (Grade 7) can be added to the longi-

tudinal model. This data will be addressed in Section 7.3.

For the moment, we shall use only the data of Grade 3 and Grade 5 test scores to

illustrate statistical methods for the simplest case as proof-of-concept for how this

time-dependent data can be analysed.

7.2 Grade 3 and Grade 5 Tests

In the data of all students who sat a Grade 3 and Grade 5 test two years apart, there

are 8 143 students from 400 schools. There are seventy-nine schools which have less

than �ve students recorded in this data set of matched Grade 3 and Grade 5 scores,

Chapter 7. Initial Longitudinal Analysis 175

and as it may be di�cult to estimate the school e�ect from such small sample sizes,

we remove these schools from the data to give 7 907 students from 321 schools. The

univariate summary statistics of this data set are given in Appendix B.2.

7.2.1 Individual Scores

The most important relationship to plot for longitudinal data on multiple subjects is

the trend of the response over time by student. We plot the actual Rasch scores for

each individual student in each school to see the trend of scores over grades (Figure

7.2.1). Each panel is a school, and the connected data points are for each individual

student. The axes are constant for all the panels, which allows for examination

of the time trends within students and for comparison of these patterns between

schools. Figure 7.2.1 plots the data for only schools 33 to 213, but similar plots can

be done for all schools and their students. An observation is that Rasch scores can

sometimes decrease over time and between tests. These are highlighted in red in

Figure 7.2.1, and in total, there are 458 students who exhibit this decrease in the

scores of two sequential tests.

Reasons for this decrease in Rasch scores are various, but we are interested in

whether school e�ect is a cause. We investigate this further by using the di�er-

ence in scores for each student.

7.2.2 Di�erence in Scores

For each student, we calculate the di�erence in NN Rasch scores between Grade 3

and Grade 5. This is a measure of a student's change over time. In accordance with

the theory of Rasch modelling, we expect an increase in Rasch scores from Grade 3

to Grade 5 to Grade 7.

Figure 7.2.2 plots for each individual student, the di�erence in the Grade 3 and

Grade 5 scores against the average of their Grade 3 and Grade 5 scores. From this

plot, we see that there is no observable trend between how well students do on tests

and how much they improve between tests. Good students can still improve greatly,

and overall improvement does not seem to be directly related to the ability of the

student, as measured by their average Rasch scores.


●●

●

●

●●

●

●●

●● ●

●

●●

●

●

●● ●●

●

● ●

●

●

● ●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

● ●●

●

●●

●

●

●●

●●

●

●

●

●

●

●

●●●

●

●

●●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●

●●

●

● ●●

●

●

●

●

●

●

●●

●

●

●●

●

●

● ● ●

●

●●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●●

●●

● ●●

●

● ●●

●

●

●

●

●●

●

●●

●

●

●

● ●●

●

●

●

●

●

●

●●

●●

●

●

●

●

●● ●

●

●●

●●

●●

●

●

●

●

●●

●

●●

●

●

●●

●● ●

●

●● ●

●●● ●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●● ●● ●

●

●●

●●●

●●●

●

●●

●

●●

●

●

●

●

●

●

●

● ●

●●●●

●

●

●

●

●

●●

●

● ●●

●

● ●

●

●

●●

●

●

● ●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

● ●● ●

●

●●

●

●

●

●

●●

●

●

●

●

●●●

●

●●

●

●

●●

● ●●

●

●

●

●●

●

●

●●

●●

●●

●

●

●●

●

●

●

●

●

● ●

●

●●●

●

●

●

●

●

●●

●

●●

●●●

●

●

●●

●●

●

●

●

●

●●

●

●

●● ●●

●

●

●

●

●●

●

●

●● ●●

●

●

●

●

●

●●

●

●

● ●

●

●

●●

●

●●

●

● ●●

●

●●

●

●

●●

●

●● ●

● ●●●

●

●

●●

●

●

●

●● ●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

● ●●

● ●

●

●

●

●

●

●

● ●●●●

●

●

●

●

●● ●●

●

●

●●

●

●●

●

●●

●●

●

●

●

●

●●

●

●●

●

●●

●

● ●

● ●

●●● ●

●●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●●

●●●●

●

●

●●●

● ●

●

●●●

●●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●●

●●

●●

● ●

●

●

●●

●

●●

●

●●

●

●

●●

●

●●

●

●

●

●

●●

●

●

●

●

●●

●●

●

●●● ●●●

●

●

●

●●

●

●

●

●

●

● ●

●

●

●

●

●●

●●

●

●

● ●● ●●

●

●

●

●

●

●

●

●●

●●

●●

●●

●

●

●

●

●

●●

●

●●

●●●

●●

●

●●

●

●●

●

●

●

●

●

●

●

●●

● ●●●

●●

●●

●

●●●

●

●

●

●

●●

●

●

●

●●

●●● ●

●●

●●

●

●●

●●

●

●●

●

●

●●

●

●

●

●

●

●● ●

●

●●

●●

●●

●

●●

●

●

●

●

●

●

●

●●

●●

●

●●

●● ●●

●●

●

●

●

●

●●

●

●

●

●●

●

●

●●●

●

●

●●

●

●

●

●

●● ●

●●

●

●

●

●●

●

●

●●

●

●●● ●

● ●●

●

●●

●●●●

●

●●

●

●

●●

●

●●

●●

●

●●

●

●

●

●

●● ●●

●● ●

●

●●

●

●

●

●

●●●

●

●

●

●

●●

●

●

●

●

●

●

●

●● ●

●●

● ●●

●

●

●●

●●

●

●●

●●

●●

● ●● ●

● ●●●

●

●

●●

●

●

●

●

●

●

● ●

●

●

●

●

●●

●●

●

●

●

●

●●

●

●

●

●

● ●

● ●

●

●

● ●

●●

●

●

●

●

●

●

● ●

●

● ● ●●

●

●

●●

●●●

● ●

●

●

●

●

●

●● ●●

●

●

●

●●

●●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●●●●

●

●

●

●●

●

●

●

●

●

●

●● ●●●

●

●●●

●

●

● ●

●●

●

●

● ●

●●

●

●

●●

●

●

●

●

●

●●

●● ●

● ●●

●●

●●●

●

●

●

●

● ●

●

●

●●

●●

●●

●

●●

●

●

●● ●● ●● ●●

●

●

●

●

●

●

●● ●

●

●

●

●

● ●

●●

●

●●

●

●

●● ●

●

●●

●

●

●●

●

●●

●

●●

●

●

●

●

●

●●

●●

●●

●

●

●

●●

●

●

●

● ●

●

●

● ●

●

●

●

●

●

●

●

●●●

●

● ●

●

● ●●

●

●

●●

●

●

●

●

● ●

●● ●

●

●●

●●

●

● ●●

●

●●● ●●●

● ●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●

●

●

●●●

●●

●

●

●

●

●

●

●

●

●

●

33 41 49 61 67 73 75 77 79 82

84 85 89 91 94 95 96 97 102 105

106 110 113 114 120 123 125 127 128 130

134 135 142 146 147 148 149 151 152 153

155 157 160 161 162 164 167 168 169 171

173 182 183 184 185 186 187 188 189 190

191 192 194 199 200 201 205 206 207 213

40

60

80

40

60

80

40

60

80

40

60

80

40

60

80

40

60

80

40

60

80

3 5 3 5 3 5 3 5 3 5 3 5 3 5 3 5 3 5 3 5Graded year

NN

Ras

ch Indicator●

●

Decrease

Increase

Figure 7.2.1: Plot of the Rasch scores of students where each panel is a school andthe colour of the points denote an increase or decrease from Grade 3 to Grade 5.

As each individual student sits two tests and has two scores recorded, it is a matched-

pairs design. We then take the di�erence to form a single sample of the di�erences

and test to see whether the school means are signi�cantly di�erent from each other

using ANOVA (Table 7.2.1). Figure 7.2.3 is a plot of the mean di�erence in each

school, and from this plot, there are no schools which have a negative mean di�er-

ence.

From Table 7.2.1, the P -value is less than 0.05, and at the 5% signi�cance level,

schoolno is a signi�cant predictor of the di�erence in Grade 3 and Grade 5 scores.


●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●●

●

●

●●

●

●● ●

●

●

●●

●

●

●

●● ●

●

●

●

●

●

●

●

●

●

● ●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

● ●

●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

● ●●

●

●

●●

●

●

●

●

● ●

●

●

●●●

●●

●

●

●

●

●

●

●●

●

●

●●●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

● ●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●●

●

●

●

●

● ●

●

●●

●

●

●

●

● ●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●● ●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●●●

● ●

●●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●●

●

●●●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●●

●

●●

●

●

●

●●●

●

●

●

●

●●

●●

●

●

● ●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

● ●

● ●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

● ●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●

●●

●

●

●

●●

●

●

●

●●

● ●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

● ●

●

●

●

●●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●●

●

●●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●●

●

●

●

● ●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

● ●

●●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

● ●

●● ●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

● ●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

● ●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

● ●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●●

●●

●●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

● ●

●

●

●

●●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

● ●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●●

●●

●● ●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

● ●

●

●

●●

●

●

●

●

● ●

●

●

●

●●

●●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●●

●

●●

●

●●

●

●

●

●

●

●●

●

●●● ●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

● ●

●

●●

●

●

●

●

● ●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

● ●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

● ●●

●

● ●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

● ●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●●

●●

● ●

●

●

●●

●

● ●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

● ●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

● ●●

●

●

●

● ●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●●

●●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●●

●

●●

●

●

●

●

●●

● ●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

● ●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●●

● ●●

●

●

●●

●

●●

●

●

●

●

●●

●

●

●

●

●

●●

●●

●

●

●

●

●

●●

●

●

●

●

●●

●●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

● ●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

● ●

●●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

● ● ●

●

●●

●

●

●

●

●

●

●

●

●

●● ●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

● ●

●

●

●●

● ●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

● ●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●● ●

●

●

●●

●

●

●

●

●

●●

●

●

● ●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●●

● ●

●

●

●

●●

●●●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●●

●●

●

●

●

●

●

●

●

●

●● ●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●● ●

●

●

●

●

●●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●

●

●

●

●

● ●

●●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●● ●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●●

●

●

●

● ●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●

●

●● ●

●

●

●

●

●

●

●●

●

●

●

●

● ●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●●

●

●

●

●

●

●●

●●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

● ●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●●

●●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●● ●

●

●

●

●

●

●

●

●

●

●

●

●●

● ●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●●

●

●

●

●●

●

●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●● ●

●

●

●

●

●

●

●

●

●●●

●●

●●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

● ●

●

●

● ●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

● ●●

●

●

● ●

●

●

●

●●

●

● ●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●●

●●

●

● ●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●●

●

●●

●

●

●●

●●

●●

●

●●

●

●

●

●

●

● ●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●

●●

●●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●●

●

●

●

● ●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●

●

●

●●

●

●●

●

●

●

● ●

●●

●

●● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●●

●

●

●●

●

●●

●

●

●

●●

●

●●

●

●●

●

●

●● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●●

●●

●●

●

●

● ●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

● ●

●

●●

●

●

●●

●

●

●

●

●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

● ●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

● ●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●● ●

●

●

●

●

●

●

●

●●

●

●●

●

●●

● ●

●

●

●

● ●●

●

●

●

●

●

●

●●

●●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●●

● ●

●

●

●

●●

●

●

●

●

●●

●

●

●

● ●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

● ●

●●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

● ●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

● ●

●

●

●

●

●

●

● ●

●

●●

●

●

●

●

●

●

●

●

●

●

● ●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

● ●

●

●

●

●

●

●

●●

●

●●

●●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●●

●

●

●●

●

●

●

●●

●●

●

●●

●●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

● ●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

● ●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

● ●

●

●

●

● ●

●

●

●

●

●

●

●

● ●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●●●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

● ●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●●

●

●

●

●

●

●

●

●●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

● ●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●●●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●● ●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

● ●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●●

● ●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

−20

0

20

40

20 40 60 80Average in Grade 3 and 5 scores

Diff

eren

ce in

Gra

de 3

and

5 s

core

s

Figure 7.2.2: Di�erence versus Average in Grade 3 and Grade 5 scores.

Table 7.2.1: Summary output of ANOVA between Grade 3 and 5 scores andschoolno

Df Sum Sq Mean Sq F -value P -valueschoolno 320 35 087 109.65 3.301 < 2.2e-16Residuals 7 586 251 997 33.22


●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●●●

●

●

●

●●

●●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●●

●

●●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

5

10

15

100 200 300 400 500 600schoolno

mea

nGrades 3−5

Figure 7.2.3: Plot of the mean di�erence in Grade 3 and Grade 5 values in eachschool.

Simple Linear Regression

Having established that schoolno is a signi�cant predictor, we wish to assess which

other student, school or test predictor variables are signi�cant in predicting this

di�erence in scores. We �t a simple linear regression as given in the model below

where Grade 3 Rasch is the Grade 3 NN Rasch score of the student and proc is the

value for procyear for a student's Grade 3 test.

(Grade 5 - Grade 3)= Grade 3 Rasch+ schoolno+ proc+ gpokm+ isolation+

spatial_ar+ staff_metr+ atsi+ lbote+ gender+ aboriginal+ disability

+ school_car + occupation + school_edu + non_school + p_g_gender +

p_g_nesb + home_langu.


As in Chapter 3, there is collinearity between the schoolno variable and the school

covariates of gpokm, isolation, spatial_ar and staff_metr which results in the

un-identi�ability of the above model. We then split the model up into the school-

number model and the school-covariates model, and the summary output of the

linear regressions are given in Table C.2.1 (Appendix C) and Table 7.2.2 respec-

tively.

School-Number Model

Under the school-number model, all of the regression coe�cients are compared to

the baseline category or intercept of a female student in school 33 who sits the

Grade 3 test in 2000 and does not come from an Aboriginal, Torres Strait Islander

or non-English speaking background, English is their home language and they are

not identi�ed as having a disability or School Card. Their primary guardian or

parent is female from an English speaking background whose status of occupation,

school education and non-school education are not stated. From Table C.2.1 in

Appendix C, there are six schools out of the 321 schools which are signi�cant, and

the signi�cant predictors are Grade 3 Rasch, proc, gender and disability at the

5% signi�cance level. This is another example of how the test variables of Grade 3

Rasch and proc are signi�cant even though there should be no statistically signi�-

cant di�erence according to the theory of Rasch modelling. The other two variables

which are signi�cant are disability and gender. It is reasonable that having a

disability is signi�cant as it is a condition which greatly a�ects a student's progress

educationally and has the potential to either �uctuate over time or have a greater

in�uence at di�erent stages of their lives. It could be possible that a disability has a

greater e�ect on younger students as they are still developing and learning to cope

with their disability. From the regression coe�cient of -1.9197 in Table C.2.1 in

Appendix C, having a disability results in lower Rasch scores by almost two units,

as expected. Gender is also statistically signi�cant with males having Rasch scores

which are 0.342 units higher on average (P -value of 0.0326) than females. Recall

that we are considering NN Rasch scores in this analysis, and boys generally perform

better at Numeracy compared to females. The signi�cance of gender also indicates

that over time, boys not only perform better, but their rate of improvement is also

higher - the development rate di�ers between males and females.



Under the school-covariates model, all of the regression coe�cients are compared to

the baseline category or intercept of a female student who sits the Grade 3 test in

2000 and does not come from an Aboriginal, Torres Strait Islander or non-English

speaking background, is not identi�ed as having a disability or School Card and their

primary guardian or parent is female from an English speaking background whose

status of occupation, school education and non-school education is not stated. The

characteristics of the reference school is one with an isolation factor of 1, spatial

value of 1.1 (metropolitan) and metro sta� classi�cation. From Table 7.2.2, the

signi�cant variables are Grade 3 Rasch, proc, aboriginal, disability, school_car

and non_school at the 5% signi�cance level.

As before, we can combine both the school-number model and the school-covariates

model using hierarchical modelling, whether through linear multilevel mixed e�ects

models or Bayesian techniques.


Table 7.2.2: Linear regression output of school-covariates model

Estimate Std. Error t-value P -valueIntercept 28.0276 0.6411 43.71 0.0000Grade 3 Rasch -0.3986 0.0109 -36.55 0.0000proc2001 1.7782 0.2406 7.39 0.0000proc2002 -0.1400 0.2743 -0.51 0.6097atsi1 0.9698 1.5676 0.62 0.5362atsiInconsistent -0.4929 0.9614 -0.51 0.6082lbote1 -0.1969 0.6569 -0.30 0.7644lboteInconsistent -0.3536 0.3026 -1.17 0.2426genderM 0.2896 0.1596 1.81 0.0697aboriginalY -2.9556 1.4501 -2.04 0.0416disabilityY -1.8251 0.3821 -4.78 0.0000school_carY -1.5561 0.6101 -2.55 0.0108occupation1 0.3448 0.4905 0.70 0.4820occupation2 0.4459 0.4055 1.10 0.2715occupation3 0.2811 0.3940 0.71 0.4757occupation4 -0.1912 0.3905 -0.49 0.6244occupation8 -0.3818 0.4015 -0.95 0.3416school_edu1 0.2839 0.6268 0.45 0.6506school_edu2 -0.2707 0.5572 -0.49 0.6270school_edu3 0.2073 0.5294 0.39 0.6953school_edu4 0.5201 0.5358 0.97 0.3317non_school5 0.3512 0.4772 0.74 0.4618non_school6 0.5174 0.5201 0.99 0.3199non_school7 1.2320 0.5479 2.25 0.0246non_school8 0.1218 0.4611 0.26 0.7917p_g_genderM -0.1446 0.2196 -0.66 0.5103p_g_nesbY 0.0276 0.3895 0.07 0.9436home_languY 0.6151 0.4089 1.50 0.1326gpokm -0.0045 0.0032 -1.40 0.1620isolation1.5 0.5138 1.4396 0.36 0.7212isolation2 -3.0177 3.2234 -0.94 0.3492isolation2.5 -1.5420 3.2291 -0.48 0.6330isolation3 -1.2058 3.3116 -0.36 0.7158isolation3.5 -1.9639 3.2790 -0.60 0.5493isolation4 -2.7259 3.3290 -0.82 0.4129isolation4.5 -2.6123 3.4033 -0.77 0.4428isolation5 -2.0635 3.6523 -0.56 0.5721isolation5.5 -2.4696 3.6641 -0.67 0.5003isolation6 -4.1386 4.0324 -1.03 0.3048spatial_ar2.2.1 -0.5314 1.4191 -0.37 0.7081spatial_ar2.2.2 0.6797 1.5766 0.43 0.6664spatial_ar3.1 2.0616 1.7254 1.19 0.2322spatial_ar3.2 3.0254 2.0259 1.49 0.1354staff_metrC 2.1576 2.8412 0.76 0.4477


Hierarchical Modelling using Linear Multilevel Mixed E�ects Models

Similar to the hierarchical model in Section 4.2, the model we consider is one where

the varying intercept represents the school e�ect and the individual-level predictors

are the student covariates. The school e�ect is then modelled at the group level by

the school covariates. Considering an individual student i, the multilevel model is

(Grade5− Grade3)i = schoolj[i] + β1proci + β2atsii + β3lbotei + β4genderi

+ β5aboriginali + β6disabilityi + β7school_cari

+ β8school_edui + β9non_schooli + β10p_g_genderi

+ β11p_g_nesbi + β12home_langui + εi

schoolj = γ0 + γ1gpokmj + γ2isolationj + γ3spatial_arj + ηj

εi ∼ N(0, σ2y)

ηj ∼ N(0, σ2α)

for student i = 1, . . . , 7907 and school j = 1, . . . , 321. This model is �t using

lmer, and the summary output is given in Table 7.2.3. P -values can be calculated

using MCMC sampling (Table 7.2.4) or the likelihood ratio test (Table 7.2.5). From

these tables, the signi�cant variables are Grade 3 Rasch, proc, gender, aboriginal,

disability and school_car at the 5% signi�cance level, and these are similar to the

signi�cant variables of the school-covariates model. Under the school-number model,

we have discussed the signi�cance of Grade 3 Rasch, proc, gender and disability,

and the same reasoning would apply. Looking at the other signi�cant variables and

their regression coe�cients, being Aboriginal or having a School Card is associated

with lower NN Rasch scores. Scores will be lowered on average by 2.8 units if a

student is Aboriginal, by 1.9 units if a student has a disability, by 1.2 units if a

student has a School Card and by 0.3 if their gender is female. These variables

are similar to disability as they are characteristics which have a great impact on

the progress of students' education, and their e�ect could �uctuate over time, in

particular the allocation of a School Card.


Table 7.2.3: Output of �xed and random e�ects from linear mixed e�ects model

Fixed E�ects Estimate Std. Error t-valueIntercept 28.355383 0.660856 42.91Grade 3 Rasch -0.405707 0.010997 -36.89proc2001 1.773802 0.243842 7.27proc2002 -0.149153 0.277424 -0.54atsi1 0.795269 1.541512 0.52atsiInconsistent -0.415879 0.942732 -0.44lbote1 -0.110775 0.647401 -0.17lboteInconsistent -0.403400 0.297961 -1.35genderM 0.318460 0.156389 2.04aboriginalY -2.824746 1.422893 -1.99disabilityY -1.868215 0.376738 -4.96school_carY -1.234884 0.609795 -2.03occupation1 0.117941 0.486178 0.24occupation2 0.316111 0.405328 0.78occupation3 0.154364 0.394346 0.39occupation4 -0.200597 0.391764 -0.51occupation8 -0.363845 0.402761 -0.90school_edu1 0.433085 0.624708 0.69school_edu2 -0.117734 0.555700 -0.21school_edu3 0.295951 0.530276 0.56school_edu4 0.541175 0.535819 1.01non_school5 0.304333 0.470355 0.65non_school6 0.413619 0.512525 0.81non_school7 1.024595 0.541995 1.89non_school8 0.067121 0.453768 0.15p_g_genderM -0.263181 0.224004 -1.17p_g_nesbY -0.089162 0.388065 -0.23home_languY 0.642543 0.411745 1.56gpokm -0.004032 0.004364 -0.92isolation1.5 0.569176 1.946172 0.29isolation2 -0.271445 2.108274 -0.13isolation2.5 0.494194 2.103445 0.23isolation3 0.802914 2.312771 0.35isolation3.5 0.188738 2.261286 0.08isolation4 -0.674492 2.413397 -0.28isolation4.5 -0.389159 2.643973 -0.15isolation5 -0.106571 3.185713 -0.03isolation5.5 -0.400420 3.277458 -0.12isolation6 -2.481292 3.857674 -0.64spatial_ar2.2.1 -0.442645 1.915104 -0.23spatial_ar2.2.2 0.633213 2.128294 0.30spatial_ar3.1 2.109029 2.311667 0.91spatial_ar3.2 3.045106 2.729152 1.12



Table 7.2.4: Estimates and P -values of �xed and random e�ects estimated byMarkov Chain Monte Carlo sampling

Estimate MCMCmean HPD95lower HPD95upper pMCMC P -valueIntercept 28.36 28.30 27.01 29.60 0.00 0.00Grade 3 Rasch -0.41 -0.40 -0.43 -0.38 0.00 0.00proc 2001 1.77 1.77 1.30 2.27 0.00 0.00proc 2002 -0.15 -0.15 -0.67 0.41 0.58 0.59atsi1 0.80 0.82 -2.11 3.83 0.60 0.61atsiInconsistent -0.42 -0.44 -2.21 1.47 0.64 0.66lbote1 -0.11 -0.13 -1.37 1.14 0.84 0.86lboteInconsistent -0.40 -0.40 -0.99 0.20 0.19 0.18genderM 0.32 0.31 0.00 0.62 0.05 0.04aboriginalY -2.82 -2.84 -5.65 -0.07 0.05 0.05disabilityY -1.87 -1.86 -2.60 -1.10 0.00 0.00school_carY -1.23 -1.30 -2.51 -0.11 0.03 0.04occupation1 0.12 0.16 -0.80 1.10 0.75 0.81occupation2 0.32 0.34 -0.47 1.12 0.41 0.44occupation3 0.15 0.18 -0.59 0.95 0.65 0.70occupation4 -0.20 -0.19 -0.95 0.56 0.62 0.61occupation8 -0.36 -0.36 -1.16 0.42 0.36 0.37school_edu1 0.43 0.41 -0.83 1.58 0.51 0.49school_edu2 -0.12 -0.14 -1.22 0.93 0.80 0.83school_edu3 0.30 0.29 -0.76 1.29 0.59 0.58school_edu4 0.54 0.54 -0.54 1.53 0.31 0.31non_school5 0.30 0.30 -0.62 1.22 0.52 0.52non_school6 0.41 0.42 -0.56 1.43 0.42 0.42non_school7 1.02 1.05 -0.00 2.11 0.05 0.06non_school8 0.07 0.06 -0.84 0.92 0.88 0.88p_g_genderM -0.26 -0.25 -0.67 0.21 0.27 0.24p_g_nesbY -0.09 -0.07 -0.82 0.71 0.86 0.82home_languY 0.64 0.64 -0.17 1.44 0.12 0.12gpokm -0.00 -0.00 -0.01 0.00 0.31 0.36isolation1.5 0.57 0.58 -2.98 4.21 0.75 0.77isolation2 -0.27 -0.33 -4.21 3.51 0.87 0.90isolation2.5 0.49 0.55 -3.19 4.49 0.78 0.81isolation3 0.80 0.85 -3.46 5.00 0.69 0.73isolation3.5 0.19 0.27 -3.92 4.34 0.91 0.93isolation4 -0.67 -0.60 -5.07 3.71 0.77 0.78isolation4.5 -0.39 -0.37 -5.17 4.39 0.87 0.88isolation5 -0.116 -0.00 -5.89 5.66 0.99 0.97isolation5.5 -0.40 -0.33 -6.34 5.51 0.91 0.90isolation6 -2.48 -2.33 -9.39 4.85 0.52 0.52spatial_ar2.2.1 -0.44 -0.48 -3.95 3.08 0.78 0.82spatial_ar2.2.2 0.63 0.62 -3.30 4.59 0.76 0.77spatial_ar3.1 2.11 2.06 -2.21 6.50 0.33 0.36spatial_ar3.2 3.05 3.03 -1.91 8.19 0.24 0.26Groups Name Std.Dev. MCMCmedian MCMCmean HPD95lower HPD95upperschoolno Intercept 1.33 1.13 1.13 0.92 1.33Residual 4.71 4.74 4.74 4.63 4.85


Table 7.2.5: Output of �xed and random e�ects from linear mixed e�ects regressionwith P -values calculated from comparing nested models �t by maximum likelihood

Fixed E�ects Estimate Std. Error t-value P -valueLRTIntercept 28.355383 0.660856 42.910000 0.000Grade 3 Rasch -0.405707 0.010997 -36.890000 0.000proc 2001 1.773802 0.243842 7.270000 0.000proc 2002 -0.149153 0.277424 -0.540000 0.585atsi1 0.795269 1.541512 0.520000 0.601atsiInconsistent -0.415879 0.942732 -0.440000 0.651lbote1 -0.110775 0.647401 -0.170000 0.858lboteInconsistent -0.403400 0.297961 -1.350000 0.176genderM 0.318460 0.156389 2.040000 0.042aboriginalY -2.824746 1.422893 -1.990000 0.046disabilityY -1.868215 0.376738 -4.960000 0.000school_carY -1.234884 0.609795 -2.030000 0.039occupation1 0.117941 0.486178 0.240000 0.794occupation2 0.316111 0.405328 0.780000 0.425occupation3 0.154364 0.394346 0.390000 0.683occupation4 -0.200597 0.391764 -0.510000 0.607occupation8 -0.363845 0.402761 -0.900000 0.360school_edu1 0.433085 0.624708 0.690000 0.492school_edu2 -0.117734 0.555700 -0.210000 0.823school_edu3 0.295951 0.530276 0.560000 0.577school_edu4 0.541175 0.535819 1.010000 0.309non_school5 0.304333 0.470355 0.650000 0.514non_school6 0.413619 0.512525 0.810000 0.414non_school7 1.024595 0.541995 1.890000 0.055non_school8 0.067121 0.453768 0.150000 0.880p_g_genderM -0.263181 0.224004 -1.170000 0.246p_g_nesbY -0.089162 0.388065 -0.230000 0.828home_languY 0.642543 0.411745 1.560000 0.117gpokm -0.004032 0.004364 -0.920000 0.336isolation1.5 0.569176 1.946172 0.290000 0.765isolation2 -0.271445 2.108274 -0.130000 0.886isolation2.5 0.494194 2.103445 0.230000 0.804isolation3 0.802914 2.312771 0.350000 0.716isolation3.5 0.188738 2.261286 0.080000 0.930isolation4 -0.674492 2.413397 -0.280000 0.779isolation4.5 -0.389159 2.643973 -0.150000 0.880isolation5 -0.106571 3.185713 -0.030000 0.979isolation5.5 -0.400420 3.277458 -0.120000 0.905isolation6 -2.481292 3.857674 -0.640000 0.516spatial_ar2.2.1 -0.442645 1.915104 -0.230000 0.810spatial_ar2.2.2 0.633213 2.128294 0.300000 0.758spatial_ar3.1 2.109029 2.311667 0.910000 0.348spatial_ar3.2 3.045106 2.729152 1.120000 0.250


186 7.3. Grade 3, 5 and 7 Tests

7.3 Grade 3, 5 and 7 Tests

In the cleaned data of students with test results in Grades 3, 5 and 7, all separated

by two years, there are 487 students from 158 schools. When schools with less than

�ve students are removed, there are now 173 students from 20 schools - this is not

much data at all, but this is not surprising, as after the data has been cleaned and

missing data removed, it is highly unlikely that there will be many students in each

school who sat all three tests. The lack of data is also explained by only having one

cohort available with three tests (Table 2.3.7), and all the students sit the Grade 3

test in 2000, Grade 5 test in 2002 and Grade 7 test in 2004.

The univariate counts of the categorical covariates are given in Table 7.3.1. Some

groups of the categorical variables have a small number of data entries which is

important to keep in mind, when making inferences. The proportions of data in

each group of a category are reasonable and what we would expect based on what

the groups represent in real life and past counts from this data set (Section 2.5

and Section 7.2). Some groups of the categorical variables are not even included in

this relatively small data set, for example, some spatial_ar, isolation and lbote

categories.

Figure 7.3.1 includes a plot of the number of students in each school and histograms

of the di�erences in scores between Grade 3 and 5 scores and Grade 5 and 7 scores.

There are not many students in any school, with the maximum number of students

in a school being 19. We also observe that there are some negative di�erences in

scores for both the di�erence in Grade 3 and 5 tests and the Grade 5 and 7 tests.

These students who decrease in Rasch scores between at least one pair of tests over

time, are highlighted in Figure 7.3.2; there are eighteen such students.

We wish to identify students who have a decrease in results as they progress from

Grade 3 to Grade 5 and from Grade 5 to Grade 7. They are bucking the expected

trend of improvement and an increase in Rasch scores for further grades. For three

sequential grades, there are four di�erent possible �trends� (Figure 7.3.3) - the usual

and desired trend is the one in part (A) of Figure 7.3.3 where there is a progression

and increase in scores over time. Any of the other three trends are cause for concern.

As we are considering Rasch scores and they have a continuous scale, we do not

expect scores to remain equal in di�erent grades. Hence, we exclude these scenarios


Table 7.3.1: Counts of data for all the categorical predictors in the data of Grade 3,5 and 7 scores

Variable Group Countisolation 1 125

3.5 214 85 75.5 12

spatial_ar 1.1 1252.2.2 293.1 19

staff_metr M 125C 48

atsi 2 1681 3Inconsistent 2

lbote 2 147Inconsistent 26

gender F 88M 85

aboriginal N 170Y 3

disability N 168Y 5

school_car N 170Y 3

home_langu N 170Y 3

Variable Group Countoccupation 0 17

1 12 143 64 118 1NA 123

school_edu 0 171 22 33 164 12NA 123

non_school 0 185 86 87 28 14NA 123

p_g_gender F 132M 41

p_g_nesb N 166Y 7

as highly unlikely with Rasch scores and have four possible trends. These trends are

highlighted in Figure 7.3.2 and in total, there are 13 students which experience a

downward trend between Grade 3 and Grade 5 and four students who have a lower

Grade 7 score than in Grade 5. However, there are no students who have both a

decrease from Grade 3 to Grade 5 and Grade 5 to Grade 7 - that is, no occurrence

of trend (D).


●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

10

15

400 500 600School Number

Num

ber

of S

tude

nts

0

5

10

15

−10 0 10 20Difference in Grade 3 and 5 scores

coun

t

0

5

10

15

0 10 20Difference in Grade 5 and 7 scores

coun

t

Figure 7.3.1: Plots of the number of students in each school and the distribution ofthe di�erences in scores between Grade 3 and 5 and Grade 5 and 7.


●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

358 451 461 490 500 527 533 547 548 550

552 556 567 571 584 593 599 600 608 64930

40

50

60

70

80

90

30

40

50

60

70

80

90

3 5 7 3 5 7 3 5 7 3 5 7 3 5 7 3 5 7 3 5 7 3 5 7 3 5 7 3 5 7Graded year

NN R

asch Indicator

●

●

Decrease

Increase

Figure 7.3.2: Plot of the Rasch scores of students where each panel is a school andthe colour of the points denote an increase or decrease between grades over time.


(A) (B)

(C) (D)

Figure 7.3.3: (A) Increasing trend from Grade 3 to Grade 5 and Grade 7 (B) Increasebetween Grades 3 and 5 but a decrease between Grades 5 and 7 (C) Decrease betweenGrades 3 and 5 but an increase between Grades 5 and 7 (D) Decreasing trend fromGrade 3 to Grade 5 and Grade 7.


7.3.1 Longitudinal Modelling

For each grade, there is the hierarchical structure of the school and student covariates

in�uencing the NN Rasch score as discussed in Chapters 4, 5 and 6. Having three

tests, Grades 3, 5 and 7, there is the added element of time as this hierarchical

structure is repeated at each time point (Figure 7.3.4) in a longitudinal data set.

SYSTEM

SCHOOL

CLASS

STUDENT

TEST

TEST RESULT

PARENT

Grade 3SYSTEM

SCHOOL

CLASS

STUDENT

TEST

TEST RESULT

PARENT

Grade 5SYSTEM

SCHOOL

CLASS

STUDENT

TEST

TEST RESULT

PARENT

Grade 7

Figure 7.3.4: Longitudinal and hierarchical structure of the education system.

With students' results in Grades 3, 5 and 7, these scores can be broken down into

two di�erences between pairs of sequential tests and analysed in the same way as for

the Grade 3 and Grade 5 tests in Section 7.2. However, these separate individual

analyses would not incorporate the underlying factor of the student which links

Grades 3, 5 and 7. We can utilise longitudinal modelling techniques which exploit

the time-dependency structure of the data

The full longitudinal and hierarchical model is

NNRaschit = schoolj[i]t + β1tatsiit + β2tlboteit + β3tgenderit + β4taboriginalit

+ β5tdisabilityit + β6tschool_carit + β7toccupationit

+ β10tschool_eduit + β11tnon_schoolit + β12tp_g_genderit

+ β13tp_g_nesbit + β14thome_languit + εit

schooljt = γ0t + γ1tgpokmjt + γ2tisolationjt + γ3tspatial_arjt + ηjt (7.3.1)

εit ∼ N(0, σ2y)

ηjt ∼ N(0, σ2α)

for student i = 1, . . . , 173, school j = 1, . . . , 20 and time t = 1, 2, 3 where t = 1 is

Grade 3, t = 2 is Grade 5 and t = 3 is Grade 7.

However for the data we have, individual students are uniquely identi�ed by the


combination of their student ID and school number. This then means that within

this data set of students who have sat all three tests, a student cannot have moved

school in that time, else, they could not be tracked over time. Hence, the school-

level variables of schoolno, gpokm, isolation and spatial_ar are not dependent

on time, and the model becomes

NNRaschit = schoolj[i] + β1tatsiit + β2tlboteit + β3tgenderit + β4taboriginalit

+ β5tdisabilityit + β6tschool_carit + β7toccupationit

+ β10tschool_eduit + β11tnon_schoolit + β12tp_g_genderit

+ β13tp_g_nesbit + β14thome_languit + εit

schoolj = γ0 + γ1gpokmj + γ2isolationj + γ3spatial_arj + ηj

εit ∼ N(0, σ2y)

ηj ∼ N(0, σ2α).

This longitudinal model can be �t in lmer. However, with only 173 students in this

data set and nine student variables with 22 categories in total, in addition to three

school variables with 17 categories in total, there is a strati�cation problem when

trying to �t the longitudinal, hierarchical model or even a simple linear regression.

One illustrative example of the insu�ciency of the data in terms of quantity and

quality when trying to �t this complicated, multilevel and longitudinal model is give

in Table 7.3.2 which is of the counts in the intersection of each of the categories of

atsi (rows) and school_edu (columns). There are no students who have an atsi

value of 1 or Inconsistent and at the same time, a school_edu value of 0, 1, 2, 3 or

4. The only occurrences are when school_edu is missing (NA), and these missing

data are removed when �tting a linear regression model. There are other pair-wise

combinations of covariates which exhibit this lack of data in the intersection of

categories, not only atsi and school_edu. Hence, it is not possible to �t regression

models with all the covariates included in the model, due to the lack of data when

it is strati�ed over all the variables' categories.

Ideally, we wish to �t a longitudinal, hierarchical model using all the student and

school variables, but are not able to with our lack of data. We could reduce the

number of estimated parameters by excluding variables, and a simple linear regres-

sion of the student and school covariates


Table 7.3.2: Counts of students in the categories of atsi and school_edu

atsi 0 1 2 3 4 NA2 51 6 9 48 36 3541 0 0 0 0 0 9Inconsistent 0 0 0 0 0 6

NN Rasch = atsi + lbote + gender + aboriginal + disability + school_car

+ p_g_gender + p_g_nesb + home_langu + gpokm + isolation + spatial_ar

highlights the linear dependence between atsi1 and aboriginal which both in-

dicate the nine Aboriginal students. The variable aboriginal is then excluded

from the model, as it does not provide any further information about the Aboriginal

status of students which is not already contained in atsi. There is also collinearity

between spatial_ar and isolation. We would choose to include spatial_ar over

isolation as we do not need both school measures of location, and the de�nition of

spatial_ar is clearer and more meaningful. The issue with removing variables from

the model and then �tting the longitudinal, hierarchical model, if it were even possi-

ble with our data, is that the reliability of the estimated parameters is reduced. We

cannot be assured that this is a good model since variables are removed pre-model-

�tting and analysis, solely to reduce collinearity and the over-parameterisation of

the model. We would be limited by the variables which we can include in the model

to try and best explain the linear relationship with NN Rasch scores.

In the situation of having appropriate and su�cient data, we would �t the model

in equation (7.3.1) in lmer and then Stan, having established that Stan is the most

appropriate method compared to BUGS (Chapter 5). These methods and programs

would be applied to the longitudinal, hierarchical model to give and assess the

signi�cance of the parameter estimates. From the �tted models, we could validate

the model using methods as outlined in Chapter 6. Essentially, we would apply all

statistical methods as for the hierarchical model to the longitudinal, hierarchical

model.

The longitudinal, hierarchical model is an improvement on the hierarchical model

as it incorporates repeated measures and accounts for the common characteristics of


the student. Given su�cient data, we can use this more complicated, but improved,

model to measure the performance of schools through the improvement of their

students.

Chapter 8

Conclusion

8.1 Summary and Conclusions

The objective of this thesis was to accurately model and measure the school e�ect

on student improvement. This was achieved on an example data set, a subset

of the Basic Skills Test data which was the precursor assessment program to the

National Assessment Program - Literacy and Numeracy (NAPLAN) tests in South

Australia. An analysis of the basic descriptive statistics uncovered various structural

and incidental anomalies at di�erent levels of the data. The use of statistics in a

forensic manner proved to be integral in investigating these anomalies, trying to

explain what we could, infer with reason and caution and to �nally achieve a heavily

reduced data set which was clean and able to be used for modelling.

The process of statistical model selection pruned the variables down further to

�fteen variables - procyear, gradedyear, gpokm, isolation, spatial_ar, atsi,

lbote, gender, aboriginal, disability, school_car, occupation, school_edu

and non_school. The issue of collinearity between the schoolno variable and the

school covariates was resolved using hierarchical modelling which more importantly,

modelled the natural multilevel structure of students within schools. In addition,

the hierarchical model estimated the school e�ect as a linear regression of location

variables. This hierarchical model, however, could not identify individual schools

and their e�ects as statistically signi�cant.

The bulk of the modelling in this thesis was �tting hierarchical models using three

195

196 8.2. Practical Implications and Future Work

di�erent methods - linear multilevel mixed e�ects models and two Bayesian ap-

proaches. The hierarchical linear mixed e�ects model was �t using the lmer package

inR, and one of the three model selection techniques was to calculate P -values using

Markov Chain Monte Carlo sampling. This method identi�ed the signi�cant �xed ef-

fects to be procyear, gradedyear, atsi, lbote, gender, disability, school_car,

occupation, school_edu and non_school.

Since the Markov Chain Monte Carlo simulation and sampling method is essen-

tially a Bayesian technique, we decided to �t the hierarchical model itself using

Bayesian Inference using Gibbs Sampling (BUGS) and assessed the �t of the model.

Unfortunately, some of the variables were highly correlated, and this was causing

imprecise estimates of the parameters. To further improve model �t, we used the

recently-released program Stan, a package for obtaining Bayesian inference using the

No-U-Turn sampler, a variant of the Hamiltonian Monte Carlo method. This im-

proved model-�tting approach achieved converged parameter estimates of increased

reliability.

The �nal model from Stan was validated and shown to be a well-�tting model which

accurately predicts students' results, given student and school characteristics. From

this model, school is not a signi�cant predictor, and our conclusion is that from the

data, schools are performing as they should. The usefulness of the Stan model lies

in its explanatory ability to identify signi�cant predictors, account for the variation

in the scores and ultimately, model the data.

If the purpose of the model was to classify schools, more school data with variables

which are not correlated would allow for school to be a treatment e�ect so that we

can identify signi�cant schools. This is future work, given suitable data. It would be

best if these extra school variables were ones that are potentially subject to control

by the school, or education authorities.

8.2 Practical Implications and Future Work

To link this work on the Basic Skills Test data back to the NAPLAN tests, we have

developed techniques which can be applied to the NAPLAN data to predict the

performance of students. These techniques incorporate the hierarchical and longi-

Chapter 8. Conclusion 197

tudinal structure of the data, and so dependencies are accounted for and modelled,

leading to sensible and reliable conclusions. If these statistical methods and models

were applied to the current NAPLAN data, the relevant education system authori-

ties, for example, the Australian Curriculum, Assessment and Reporting Authority

(ACARA), could use the results to approach schools and see which programs are

being implemented to the advantage of students and what can be done to aid other

schools. Used appropriately, this information could be used to raise the performance

of many schools. Should a speci�c teaching initiative be implemented in a subset of

schools, these statistical models can identify the e�ect of these additional programs

and measure the resultant improvement of the students. This could then be used to

assess the �value� of the di�erent initiatives.

Recall that there is also the perspective of the parent who is primarily interested

in knowing, given the characteristics of their child, which school is best suited to

their child, and are there any particular schools which can be expected to produce

better subsequent achievements than other schools? The breakdown of the model

into student and school level explanatory variables, gives the e�ect of each of the

individual variables on the overall school mean. This personalises the school e�ect

to also incorporate an individual student's characteristics.

Having addressed both the school and student levels of the hierarchy of the education

system, the repeated measures of tests at increasing grades for each student is the

longitudinal aspect which is now available in the NAPLAN data, four years after

it �rst started in 2008. As students sit NAPLAN tests in Grades 3, 5, 7 and 9, an

individual student will only be recorded every two years, for example, a student who

sits the Grade 3 test in 2008 will have their Grade 5 test recorded in 2010. As of

December 2012, there are at most, only two recorded test results for each student.

This is very similar data to the Grades 3 and 5 longitudinal data in Chapter 7.

One restriction with the longitudinal Basic Skills Test data was only being able to

identify students uniquely by both their school and student ID numbers - we could

not identify students who had changed schools between tests. However, the mobility

of students is an important issue which can be modelled from the NAPLAN data

which assigns to each student, a unique ID number for the duration of their edu-

cation. For any longitudinal measure, how students who change and move between

198 8.2. Practical Implications and Future Work

schools are modelled must be resolved. For example, what proportion of a student's

results or improvement should be attributed to the current and the former school is

a question that should be addressed. Goldstein et al. [20] states

�There is also the problem of accounting for students who change schools,

who may have particular characteristics, and there is almost no research

into this problem.�

This student mobility is a vital consideration, because a large proportion of students

change schools at some point (especially the transition between Year 7 and 9 or Year

5 and 7, that is, from primary to secondary education, depending on the state or

territory), and so valuable information is lost if they are discarded from the data

analysis.

The focus of this thesis was primarily at the student and school levels because

of the data available. However, there are other levels in education's hierarchical

structure, and the statistical techniques discussed in this thesis can be widely applied

to compare the performance of students, whether it be between classes, between

the private and government school sectors or between denominational versus non-

denominational schools.

One �nal area of interest is the noticeable drop in student participation in certain

schools. An article by Save Our Schools [14] on November the 26th, 2012 says that

although the average percentage of students who have withdrawn from the NAPLAN

tests remains low in all states and for Australia overall, these low averages disguise

some very high withdrawal rates in many schools.

�The average withdrawal rates across Australia in 2011 and 2012 were

one to two percent for the di�erent Year levels tested. In contrast, 276

schools had withdrawal rates of 10% or more in 2011 according to the

My School website. Seventy-eight schools had over 25% withdrawn and

32 schools had withdrawal rates of between 75 and 100%. . .

Schools with high withdrawal rates include both government and private

schools . . . Private schools accounted for the large proportion of schools

with very high withdrawal rates. Seventy-three per cent of all schools that

had 25% or more of their students withdrawn from NAPLAN were private

schools.�

Chapter 8. Conclusion 199

This �worrying� trend may have roots in something deeper at an education level,

but at a data management and analysis level, the rapid growth in withdrawal rates

poses a threat to the reliability of NAPLAN results for inter-school comparisons,

inter-jurisdictional comparisons and trends in indicators of student achievement [13].

The increasing numbers of students being withdrawn or absent from NAPLAN will

a�ect the reliability of the results as exempt students, those students who are not

assessed, are deemed to be below minimum national standards and still included

in the NAPLAN results. Changes in participation rates could a�ect the results of

individual schools, sub-groups of students such as Indigenous and low socio-economic

status students and state or territory results as well as trends over time. This needs

to be taken into account in the interpretation of future NAPLAN results.

As shown in this thesis, statistical modelling techniques can be applied to an ed-

ucation data set, such as the Basic Skills Tests or NAPLAN data to draw reliable

conclusions about school performance and can potentially be used to answer future

research questions in this area.

Appendix A

Coding of Variables

(c) represents categorical variables

(q) represents quantitative, continuous variables

A.1 Test Variables

aspect literacy or numeracy aspect (c)

LL, LR, LS, LW, NN, NS, NU, NM

gradedyear grade of the test (c) - 3, 5 or 7

nocorrect raw test mark (q)

procyear calendar year (c) - 1997 - 2005

standardsc standardised score under Rasch scaling (q)

A.2 School Variables

x00y{4,5,6}_abs absentee rate in 2004, 2005 and 2006 respectively

(q)

x00y{4,5,6}_beh number of behavioral incidents in 2004, 2005 and

2006 respectively (q)

201

202 A.2. School Variables

cap Country Areas Program* (c)

Y Yes

x00y{4,5,6}_enr enrolment numbers in 2004, 2005 and 2006 respec-

tively (q)

gpokm distance from Adelaide General Post O�ce in km

(q)

isolation isolation index (c)

1, 1.5, 2, 2.5, 3, 3.5, 4, 4.5, 5, 5.5, 6, 6.5, 7

(further information about the categories of

this variable is not available)

x00y{4,5,6}_mob mobility of students in 2004, 2005 and 2006 respec-

tively (q)

schoolno code number of the school (c)

x00y{4,5,6}_scrd number of School Cards** in 2004, 2005 and 2006

respectively (q)

spatial_ar MCEETYA*** classi�cation for Rurality and Re-

moteness (c)

1.1 Metropolitan

2.2.1 Large Provincial Towns

2.2.2 Small Provincial Towns

3.1 Remote Areas

3.2 Very Remote Areas

* The Country Areas Program is an Australian government program which provides

�nancial help to rural schools.

(https://deewr.gov.au/country-areas-program)

** The School Card Scheme is a government initiative which provides �nancial

assistance towards educational expenses for eligible families.

(http://www.decd.sa.gov.au/goldbook/pages/school_card/schoolcard/?reFlag=1)

*** Ministerial Council on Education, Employment, Training and Youth A�airs

Appendix A. Coding of Variables 203

staff_metr classi�cation by DECS**** sta� (c)

M Metro

C Country

x00y{4,5,6}_tch number of teachers in 2004, 2005 and 2006 respec-

tively (q)

x00y{4,5,6}_tmob teacher mobility in 2004, 2005 and 2006 respec-

tively (q)

**** Department of Education and Children's Services

A.3 Student Variables

aboriginal Aboriginal status (c)

Y Yes

N No

atsi Aboriginal or Torres Strait Islander (c)

1 Yes

2 No

country_of country of origin (c) - country abbreviations (e.g. TAIW)

cultural_b cultural background (c) - country abbreviations (e.g.

SAUD)

date_of_bi date of birth

disability disability status (c)

Y Yes

N No

gender male or female (c)

M Male

F Female

204 A.3. Student Variables

home_langu English as home language (c)

Y Yes

N No

lbote language background other than English (c)

1 Yes

2 No

nesb_code non-English speaking background (c)

A Students of ATSI origin who identify as ATSI and

who speak an ATSI language (including Aborigi-

nal English). Exclude ATSI students who do not

speak an ATSI language.

P1 Permanent resident students born overseas with

at least one parent/guardian from a non-English

speaking background. (This includes children adopted

by English speaking families who have maintained

a cultural or linguistic link with their country of

origin.)

P2 Permanent resident students born in Australia with

at least one parent/guardian born overseas and

from a non-English speaking background.

P3 Permanent resident students born in Australia, not

included in the previous two de�nitions, who have

maintained an identity and family link with a non-

English speaking language or culture.

TR Temporary resident - students who are not per-

manent residents in Australia and who come from

non-English speaking countries

non_school parental non-school education (c)

0 Not stated/unknown

Appendix A. Coding of Variables 205

5 Certi�cate I to IV

6 Advanced diploma/Diploma

7 Bachelor degree or above

8 No non-school quali�cation

occupation parental occupation group (c)


1 Senior management in large business organisations,

government administration and defence and qual-

i�ed professionals

2 Other business managers, arts/media/sportspersons

and associate professionals

3 Trades and advanced/intermediate clerical, sales

and service sta�

4 Other occupations like machinery operators, hos-

pitality sta�, o�ce or sales assistants, labourers

and related workers

8 Not in paid work in the last 12 months

p_g_countr parental country of origin (c) - country abbreviations

(e.g. ZIMB)

p_g_cultur cultural background of parent/primary guardian (c) -

country abbreviations (e.g. RUSS)

p_g_gender gender of principal guardian or parent (c)

M Male

F Female

p_g_nesb parental non-English speaking background (c)

Y Yes

N No

206 A.3. Student Variables

school_car individual School Card (c)

Y Yes

N No

school_edu parental school education (c)


1 Year 9 or equivalent or below

2 Year 10 or equivalent



status current status of student within school (c)

A Active

B Bus only

C Exclusion placement

D Deceased

E External

F Future

H Home education

L Left

M Medical placement

N Not active

O Out of scope

R Not in census

T Alternative placement

V Permanent exemption

X Never attended

studentide student ID number (c)

visa_sub_c Australian permanent visa sub-class (c)

Appendix B

Plots

B.1 Boxplots of LL Rasch and NN Rasch for Cate-

gorical Variables

The sample size and sample mean are given above each boxplot. The na category

represents missing data for the categorical variable.

207

●●

●

●●

●●

●

●●●●

●●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●●

●●●

●

●

●●●

●

●

●

●●●

●●

●

●●

●

●●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●●

●●

●

●●

●

●●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●●●

●

●

●●

●

●●

●●●●●●●

●

●

●

●●●

●

●

●

●●

●●

●●

●●●●●●●●●●

●

●●●●●

●

●

●

●

●

●●

●

●●●●

●●

●●●●●●

●

●

●

●

●●●

●

●

●●●●

●

●●●

●

●●●●

●●●●●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●●

●

●

●

●●●

●●

●●●●

●

●●

●

●●

●

●

●●●

●

●

●

●

●

●●

●

●●●●

●

●

●●●●●

●●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●●●●●●●

●●

●●

●

●●

●

●

●

●

●●●●●●●

●●●

●

●●

●

●●●●●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●●

●

●●

●

●

●●●

●●

●●●●●

●

●

●

●

●●

●●●

●

●

●●●

●

●●●

●

●

●●●●●●●

●●●

●

●

●●

●

●

●

●

●●

●●

●

●

●

●●

●●

●

●

●

●

●●

●

●

●

●

●●

●

●

●●●

●

●

●

●

●

●

●

●

●

●●

●

●

●●●●●

●

●●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

7 8 68 8308 8259 17546 17819 7845 9

45.7 46.8 45.4 48.27 49.16 52.52 52.15 55.84 56.89

procyear

20

40

60

80

100

1997 1998 1999 2000 2001 2002 2003 2004 2005Categories

LL

Ras

ch

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●●

●

●

●

●

●

●●●

●

●

●●●

●

●

●

●●●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●●●

●●●●

●

●

●

●●

●

●

●●

●●

●

●●●●●●●●

●

●●●

●●

●●●●●

●

●●●

●

●●●●●●●

●

●

●●●●●

●●

●●

●

●●

●●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●●●

●●

●

●

●●

●●●●

●

●●●

●

●●

●●

●

●

●●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●●

●

●

●

●

●●

●●

●

●

●

●

●

●●

●

●

●

●

●●

●●

●

●

●

●●

●●

●

●

●

●●

●

●●●●●●

●

●

●

●

●

●

●

●

●●●

●

●

●●

●

●

●●●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●●

●●

●

●

●

●●

●

●●

●●

●

●●

●

●●

●●●●●

●

●●●●

●

●

●

●●

●

●

●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●●●●

●

●

●

●

●●●●●●

●

●

●●

●

●

●●

●

●

●●

●

●

●

●●

●

●

●

●●

●

●

●●

●

●

●●●

●

●●●●

●●

●

●●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●●

●

●●

●

●●●

●

●

●

●

●●

●

●

●

●

●

●

●●●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●●●●

●

●●

●

●●

●

●

●●●

●

●●●

●●●

●●

●

●●

●●

●●●●

●

●

●

●

●

●

●

●●●

●

●

●

●●

●●

●

●●

●●●

●

●

●●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●●

●●

●

●

●●●●

●

●

●

●●

●

●

●

●

●

●●●

●

●

●●●

●●

●

●

●●●

●●

●●●

●●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●●●

●

●

●

●

●

●

●●●●

●

●

●

●

●●

●●●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●●●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●●

●

●

●●●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●●●

●

●

●

●●

●●

●●●

●●

●

●

●

●

●●●

●

●

●

●

●●●●●●●

●

●

●●●

●

●●●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●

●

●●

●

●

●

●●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

6 7 59 8346 8261 17610 17935 7844 10

47.37 46.69 42.64 48.88 49.32 54.14 54.27 57.66 58.42

procyear

20

40

60

80

100

1997 1998 1999 2000 2001 2002 2003 2004 2005

Categories

NN

Ras

ch

208 B.1. Boxplots of LL Rasch and NN Rasch for Categorical Variables

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●●

●●

●

●●

●●

●

●●●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●●●●●

●

●

●●●●

●

●

●

●

●

●

●

●

●

●●●●

●●

●

●

●

●●●●●

●

●

●●●

●

●●

●●

●●

●

●

●

●●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●●●

●

●

●

●●

●●●●

●

●

●

●

●

●

●

●●●●

●●

●

●

●●

●

●●

●

●

●

●

●

●●

●

●●●●●

●

●●

●

●●●

●●●

●

●●

●●

●●

●

●

●

●

●●

●

●

●

●

●●

●●

●

●

●●

●●●●

●

●

●

●

●

●●

●

●●●

●

●●

●

●

●

●

●

●●●●●

●

●●

●●

●

●●

●

●●●●

●

●●●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●●

●

●●

●

●

●

●●●

●

●

●

●

●

●●

●

●●

●●●●●

●

●●

●

●

●●●

●

●

●●

●

●

●

●

●

●●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●●

●

●

●●

●●

●

●●

●

●●●●

●●

●

●●●●

●

●●

●

●

●

●●●●●

●

●

●●●

●

●

●●●

●

●

●

●●●

●

●

●

●

●

●●

●

●●

●●

●

●

●●

●

●

●●

●●

●

●●●●●

●

●●

●

●

●●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●●

●

●

●●

●●

●

●

●

●●●●●●

●

●●●

●

●

●

●●

●

●

●●

●

●

●

●

●

●●●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●●

●

●●●●●

●

●

●

●

●

●

36837 20477 2555

48.94 55.72 61.18

gradedyear

20

40

60

80

100

3 5 7Categories

LL

Ras

ch

●●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●●

●●

●●

●●

●

●

●●

●

●

●

●

●

●

●●●●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●

●

●●

●

●

●

●

●

●

●●●

●

●

●●

●

●●

●

●

●

●

●

●●●●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●●●

●●●●

●

●●●

●

●●●●●●●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●●

●●

●

●

●●

●●

●

●

●

●

●

●

●

●

●●

●

●

●●

●●●

●

●●

●

●

●

●

●

●

●

●●●●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●●●

●

●●

●

●

●

●●●

●

●

●●

●

●

●●

●

●

●

●●

●●●

●●●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●●●●

●●

●

●

●●

●

●

●

●

●

●

●●

●

●

●●●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●

●

●●

●

●

●

●●●●

●

●●●●

●

●●●

●●

●

●

●●

●

●

●

●●●●●●

●●●

●

●

●

●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●●

●

●

●●●●

●

●

●

●

●●●

●

●●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●●●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●●●●●●

●●

●●

●●

●●●●

●

●●

●

●●

●

●●

●●

●

●

●

●

●

●

●

●●●●●

●

●

●

●

●

●

●

●

●●

●

●●

●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●●

●●●

●

●

●

●

●

●●

●●

●●

●

●●

●

●

●

●

●●

●●●

●

●

●●

●

●

●

●

●

●●●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

●

●●●●

●●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●●●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●●

●

●

●●

●●

●

●

●●

●

●

●●●●●

●

●●

●●

●

●●

●●●●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●●●

●

●●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●●●

●

●●●

●

●

●

●

●

●

●

●●●

●●

●

●

●

●●

●●●

●●●●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●●

●

●

●

●

●

●

●

●●

●

●

●

●●●●●

●

●

●

●

37005 20490 2583

49.65 58.17 65.45

gradedyear

20

40

60

80

100

3 5 7Categories

NN

Ras

ch

Appendix B. Plots 209

●●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●●●●●

●

●

●

●

●

●●

●

●●●

●●

●

●●

●

●●●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●●●●●

●●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●●●●

●

●●

●

●

●

●●●●

●

●●

●

●

●●●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●●

●

●●●

●

●

●●●

●

●

●

●

●

●

●

●●●●

●

●●●●

●●

●

●

●●●●

●

●

●●●

●

●

●●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●●

●●

●

●●

●

●

●●

●

●

●

●●●●●

●●●●●

●

●

●

●●●●●

●

●

●

●

●

●

●

●

●

●●

●

●●

●●

●

●

●

●

●●●●

●

●

●●●

●

●

●

●

●

●●

●

●

●●

●

●●

●●

●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

● ●

●●

39848 1392 135 8 85 1174145 1029 1174 920 3545 2348 4213 910

52.15 50.05 46.28 43.74 37.91 51.7252.1 48.85 51.88 51.93 51.16 50.12 51.33 51.14

isolation

20

40

60

80

100

1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6 6.5 7 NACategories

LL

Ras

ch

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●●

●

●

●

●●●●●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●

●●

●●

●●●

●●

●●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●●●

●●

●

●●

●

●

●

●

●

●

●

●●

●●

●

●

●●

●

●

●

●●

●

●●

●

●

●

●

●●●

●●

●

●●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●●

●

●

●

●

●●●●

●

●●●

●●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●●

●

●

●

●●

●

●

●

●●

●

●●●

●●

●

●

●●

●●

●●●

●

●

●

●

●

●

●

●●●

●

●

●●

●

●●

●

●●●●

●●●

●

●

●●

●

●

●

●

●●●●●●●

●

●

●●

●●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●●

●

●

●●

●

●

●●●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●●

●

●

●

●

●●

●

●

●

●

●

●●●●

●

●●●

●

●

●

●

●

●

●●●●●

●

●

●

●●

●

●●

●

●●

●

●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●●

●

●●●

●

●

●

●

●

●

●●

●

●

●●

●

●

●●

●●

●

●●

●

●

●

●●

●

●

●●●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●●●●

●

●

●

●

●

●

●●●

●

●

●

●

●●●

●

●●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

39815 1423 141 9 117 1194164 1039 1194 932 3598 2331 4282 914

53.47 51.55 45.89 43.91 37.53 54.353.87 50.17 54.18 54.32 53.11 52.08 52.59 53.41

isolation

20

40

60

80

100

1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6 6.5 7 NACategories

NN

Ras

ch


●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●●●●●

●

●

●

●

●

●●

●

●●●

●●

●

●●

●

●●●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●●●●●

●●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●●●●

●

●●

●

●

●

●●●●

●

●●

●

●

●●●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●●

●

●●●

●

●

●●●

●

●

●

●

●

●

●

●●●●

●

●●●●

●●

●

●

●●●●

●

●

●●●

●

●

●●●

●●

●

●

●

●

●

●

●

●

●●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●●

●●

●

●●

●

●

●●

●

●

●

●

●

●●●

●●●●●

●

●

●

●

●

●

●●●

●●

●

●●

●

●

●

●

●

●

●●

●

●●●

●●

●

●

●●

●

●●

●●

●

●

●

●●

●●●

●

●

●

●

●

●

●

●

●

●●

●

●●●

●●

●

●

●

●●

●

●

●

●●

●●●

●

●

●

●●

●

●

●

●

●

●

●

●●

●●

39489 7581 9767 2451 568 13

52.16 51.53 50.97 50.84 47.11 52.8

spatial_ar

20

40

60

80

100

1.1 2.2.1 2.2.2 3.1 3.2 NACategories

LL

Ras

ch

●

●

●●

●

●●

●

●

●●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●●

●

●

●

●●●●●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●

●●

●●

●●●

●●

●●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●●●

●●

●

●●

●

●

●

●

●

●

●

●●

●●

●

●

●●

●

●

●

●●

●

●●

●

●

●

●

●●●

●●

●

●●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●●

●

●

●

●

●●●●

●

●●●

●●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●●

●

●

●

●●

●

●●●

●●

●

●

●●

●●

●●●

●

●

●

●

●

●

●

●●●

●

●

●●

●

●●

●

●●●●

●●●

●

●

●●

●

●

●

●

●●●●●●●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●●●●●

●

●●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

●

●

●●●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●●●

●

●●●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

39445 7646 9891 2470 612 14

53.49 53.14 52.77 52.7 47.71 54.61

spatial_ar

20

40

60

80

100

1.1 2.2.1 2.2.2 3.1 3.2 NACategories

NN

Ras

ch


●

●

●

●

●●

●

●

●

●

●

●●

●●

●●●

●

●●●

●

●●

●●

●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●●●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●●●

●●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●●●●●

●

●

●

●

●

●●

●

●●●

●●

●

●●

●

●●●

●

●

●

●●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●●●●●

●●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●●●●

●

●●

●

●

●

●●●●

●

●●

●

●

●●●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●●●●

●

●

●

●

●

●

●

●●

●

●●●

●

●

●●●

●

●

●

●

●

●

●

●●●●

●

●●●●

●●

●

●

●●●●

●

●

●●●

●

●

●●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

16049 43807 13

50.77 52.16 52.8

staff_metr

20

40

60

80

100

C M NACategories

LL

Ras

ch

●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●●●

●●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●●●

●

●

●

●

●

●

●●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●●●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●●

●

●

●

●●●●●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●●

●●●

●●

●●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●●●

●●

●

●●

●

●

●

●

●

●

●

●●

●●

●

●

●●

●

●

●

●●

●

●●

●

●

●

●

●

●●●

●●

●

●●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●●

●

●

●●

●

●●●●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●●●

●

●●

●

●

●

●●

●

●

●

●●

●

●●●

●●

●

●

●●

●●

●●●

●

●

●

●

●

●

●

●●●

●

●

●●

●

●●

●

●●●●

●●●

●

●

●●

●

●

●

●

●●●●●●●

●

●

●●

●●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●●

●

16275 43789 14

52.47 53.51 54.61

staff_metr

20

40

60

80

100

C M NACategories

NN

Ras

ch


●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●●●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●●●

●●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●●●●●

●

●

●

●

●

●●

●

●

●

●

●●

●●

●

●

●●

●

●●

●

●

●

●●●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●●●

●●

●

●

●

●

●

●

●

●

●

●

●

●●●●

●

●●

●

●

●

●●

●●

●

●●●●

●

●

●

●

●

●

●●

●

●●

●●●●●

●

●●

●●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●●

●

●

●●●

●

●

●

●

●

●●

●

●

●●

●

●●●

●

●

●

●

●

●

●●●

●

●

●●●●

●●

●

●

●●

●

●

●●●●

●

●

●

●

●●

●

●

●●●

●

●

●

●

●●●

●

●

●

●●●

●

●

●●

●

●

●

●

●

●

●

●●●●

●

●

●

1936 57529 404

45.77 52 49.82

atsi

20

40

60

80

100

1 2 NACategories

LL

Ras

ch

●

●

●

●

●●●●

●

●

●●

●

●

●

●

●

●●

●●

●

●

●

●●

●

●

●

●

●

●

●

●●●

●●

●

●●●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●●

●

●

●

●

●●●●●

●●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●●

●

●

●

●●●●●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●●●●

●●

●●●●

●

●

●

●

●

●

●

●

●●

●

●●●

●

●

●

●

●●

●

●

●

●

●

●●

●●●

●●

●●

●●

●

●

●

●

●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●●

●●

●●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●●●●

●●

●

●

●

●

●●●

●

●●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●●

●

●

●●

●

●●●●

●

●●

●

●

●

●

●

●

●

●●

●

●●●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●●

●●

●

●

●●

●●

●●●

●

●

●

●

●

●

●

●●●

●●

●

●

●

●

●●

●●

●

●●

●

●●●●

●●●

●

●

●●

●

●

●

●

●

●

●

●

●●●●●●●

●

●

●●

●●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

1934 57730 414

45.97 53.49 50.69

atsi

20

40

60

80

100

1 2 NACategories

NN

Ras

ch


●

●●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●●

●

●●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●

●●

●

●

●

●

●

●

●

●●●

●●

●

●

●●●

●

●

●●

●

●

●

●

●

●

●

●

●●●●●●●

●

●

●

●

●

●●●

●

●

●●

●●

●

●●

●●

●

●●

●

●

●

●●●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●●●●

●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●●

●

●

●

●●●●

●

●●

●●

●

●●●●

●

●

●●

●

●

●

●●

●

●●

●

●●●

●

●●

●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●

●●

●●●●

●

●

●

●

●

●●

●

●

●●●

●●

●

●

●

●

●

●●●●

●

●

●●●

●●

●

●●

●

●

●●●●●

●

●

●

●

●●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

2445 51606 5818

50.07 51.76 52.7

lbote

20

40

60

80

100

1 2 NACategories

LL

Ras

ch

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●●●

●●

●

●

●●

●●

●

●

●

●

●

●

●

●

●●●

●●

●

●●●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●

●

●●●●

●

●

●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●●

●

●●●●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●

●

●

●

●

●●●

●

●

●

●●

●

●

●

●

●

●

●

●●

●●●

●●

●●●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●

●●

●

●

●

●

●

●●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●●●●

●●

●

●

●

●

●●

●●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●●

●

●

●●

●

●●●

●

●●●

●

●

●

●

●

●

●

●●●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●●●

●●●●

●

●●●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●

●●

●●

●

●

●

●

●

●

●

●●●

●

●

●

●

●●●●

●

●

●

●●

●●●

●

●

●●

●

●

●

●

●

●

●

●

●●●●

●

●

●●

●●

●

●

●●

●●

●

●●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●●●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●●

●●●

●

●

●

●

●

●

2494 51712 5872

51.01 53.24 54.13

lbote

20

40

60

80

100

1 2 NACategories

NN

Ras

ch


●

●

●

●●

●

●

●

●

●

●●

●

●

●

●●●●

●

●

●●●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●●●

●

●

●

●

●

●●

●

●●●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●●●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●●●●

●

●

●

●

●●

●●

●

●●

●

●●

●●●

●

●

●

●

●

●●

●

●●

●

●●●●

●

●

●

●●●

●

●

●●

●●●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●●

●

●

●●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●●

●

●

●

●

●●

●●

●

●●

●

●

●●●

●

●

●

●

●

●

●

●●

●

●●

●

●●

●

●

●●●●

●

●

●●

●

●

●

●

●

●

●●

●

●●●

●

●●

●

●

●

● ●

●●

●

●●●●●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●●●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●●

●

●

●

●●●

●

●

●

●●●●

●

●●●

●

●

●●●

●●

●

●

●

●●

●●●

●

●●

●

●

●●

●

●●

●

●

●

●●●●

●

●

●●

●

●●

●

●●●

●

●

●

●

●●

11321 88 1 9 28179 32 2 5 20232

50.24 52.82 57.38 49.11 52.29 50.09 48.93 51.18 51.94

status

20

40

60

80

100

A B E H L N T X NACategories

LL

Ras

ch

●

●●

●

●

●●●●

●●

●

●

●

●

●

●

●●●

●●●

●

●●

●

●

●

●

●

●●

●

●●

●

●●

●

●

●

●

●

●

●

●●●

●

●●

●

●

●●

●

●

●●●●●

●

●

●

●

●●

●●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●●●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●

●

●●●

●

●

●

●

●●

●●

●

●●

●

●

●

●

●●

●●●

●●●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●●

●

●

●

●●

●

●●●●

●

●

●

●

●●

●

●●

●●●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●

●

●

●

●

●

●●

●●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●●

●●

●

●●

●●●

●

●

●

●

●

●

●●●

●●●

●

●

●

●●●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●●●

●●

●

●●

●

●

●

●

●

●

●

●●●

●●

●

●

●

●

●

●

●

●

●

●●●

●●

●

●

●

●

●

●●●

●●

●●●

●

●●●

●

●

●

●

●

●●●

●

●

●

●

●

●●

●●

●

●●●

●

●

●●

●

●

●

●

●●

●

●●●

●

●

●●

●●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●●

●

●

●

●

●

●●

●●

●●●

●

●

●

●

●

●

●

●●

●●

●●

●

●

●●

●

●●

●

●●

●

●●●●

●●●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●

●

●

●

●

●●

●●

●

●

●

●●

●●●

●

●

●●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●●

●

●●●●●●

●

●●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

11421 95 1 6 28140 32 2 6 20375

50.89 56.84 54.48 48.17 54.18 54.27 61.13 50.89 53.22

status

20

40

60

80

100

A B E H L N T X NACategories

NN

Ras

ch


●

●

●

●

●

●

●

●

●

●

●

●

●●●●●●●

●●

●●●

●

●

●

●

●

●

●●

●

●

●

●●●

●●

●●●

●●

●●

●

●

●

●●●

●

●●

●

●

●●

●●

●

●●

●

●●

●●

●

●

●●

●●

●

●

●

●

●

●●●

●

●●

●

●

●

●●

●

●

●

●

●●

●

●

●●

●

●

●●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●●

●●

●●

●

●

●

●●

●

●

●

●

●●

●

●●●●●

●●

●

●

●

●

●●●●

●

●

●

●

●

●●

●

●●

●

●

●

●●

●●

●

●

●

●

●●●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●●

●

●

●

●●●

●

●

●●

●

●

●

●●●

●

●

●

●

●

●●● ●

●●

●

●●●●●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●●●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●●

●

●

●

●●●

●

●

●

●●●●

●

●●●

●

●

●●●

●●

●

●

●

●●

●●●

●

●●

●

●

●●

●

●●

●

●

●

●●●●

●

●

●●

●

●●

●

●●●

●

●

●

●

●●

19400 20237 20232

52.75 50.7 51.94

gender

20

40

60

80

100

F M NACategories

LL

Ras

ch

●

●

●●

●●

●

●

●●●

●

●

●●

●

●

●

●●

●

●

●

●●●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●●●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●●●

●●

●

●

●

●

●

●

●

●

●●

●●

●

●

●●●

●

●

●

●●●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●●

●

●

●

●●

●

●●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●●

●●●

●

●

●●

●●

●

●

●

●

●

●●●

●●●

●

●

●●

●

●

●

●

●●●

●

●

●

●

●●●

●

●●

●

●

●

●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●

●

●

●●●

●

●

●

●

●

●

●●

●

●●

●

●

●

●●

●●

●●

●

●

●●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●●●●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●

●●

●

●

●●

●●

●●●●●●

●

●●

●

●

●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●●●

●●

●●

●●

●

●●

●

●

●●●●

●

●●

●

●

●

●

●

●

●

●

●●●●

●●●

●

●

●

●

●●●

●

●

●

●●

●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●

●●●●

●

●

●●

●

●●●●●●●

●

●●

●

●●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●

●

●

●

●

●●

●●

●

●

●

●●

●●●

●

●

●●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●●

●

●●●●●●

●

●●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

19315 20388 20375

52.75 53.7 53.22

gender

20

40

60

80

100

F M NACategories

NN

Ras

ch


●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●●

●●●●

●

●●

●

●●

●

●

●

●

●

●

●

●

●●●●

●

●

●

●

●

●

●

●

●

●●

●●

●

●●

●●

●

●●●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●●

●●

●

●

●●

●

●●

●

●●

●●

●

●

●●

●

●

●

●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●●

●

●

●●●

●

●

●

●

●

●

●

●●

●

●●

●●

●

●

●●

●

●

●●●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●●●●●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●●●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●●

●

●

●

●●●

●

●

●

●●●●

●

●●●

●

●

●●●

●●

●

●

●

●●

●●●

●

●●

●

●

●●

●

●●

●

●

●

●●●●

●

●

●●

●

●●

●

●●●

●

●

●

●

●●

38354 1283 20232

51.91 45.71 51.94

aboriginal

20

40

60

80

100

N Y NACategories

LL

Ras

ch

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●●●

●

●●

●

●

●

●●●

●

●

●●●

●

●●

●

●●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●●

●

●●

●●●●

●

●

●

●●

●

●●●

●●

●●

●

●

●

●

●

●

●

●●

●●

●

●

●●●

●

●●●

●

●●●

●

●●

●

●

●

●

●●

●

●

●

●

●

●●●

●

●

●

●

●●

●●

●

●●●

●

●

●

●

●

●●

●

●●

●

●●

●

●

●

●●

●●●●

●

●●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●●

●

●●

●

●●●●

●●●●●

●

●

●

●

●●

●●

●

●

●

●●●

●

●

●

●

●●●●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●●

●●●●●

●

●●●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●

●

●

●

●

●●

●●

●

●

●

●●

●●●

●

●

●●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●●

●

●●●●●●

●

●●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

38416 1287 20375

53.47 46.24 53.22

aboriginal

20

40

60

80

100

N Y NACategories

NN

Ras

ch


●

●

●

●

●

●●

●

●

●

●●●●●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●

●

●

●

●●●●●

●●

●

●

●●●

●●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●

●

●

●

●

●●

●

●

●●●

●

●

●

●

●●●●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●●●●●

●●

●●●

●

●

●●

●

●

●●●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●

●

●●

●

●

●

●

●

●●

●●

●

●

●

●

●●●

●

●

●

●●

●

●

●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●●●●●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●●●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●●

●

●

●

●●●

●

●

●

●●●●

●

●●●

●

●

●●●

●●

●

●

●

●●

●●●

●

●●

●

●

●●

●

●●

●

●

●

●●●●

●

●

●●

●

●●

●

●●●

●

●

●

●

●●

37236 2401 20232

52.24 43.38 51.94

disability

20

40

60

80

100

N Y NACategories

LL

Ras

ch

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●●●●●

●●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●●

●

●

●●

●

●

●

●●●●●

●

●●●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●●

●

●●●

●●

●●

●●

●

●

●

●●

●

●

●

●

●●

●●

●

●

●●●

●

●

●

●

●

●●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●●

●

●

●

●

●●

●

●●●

●

●●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●●●●

●●●

●

●

●

●

●

●

●●

●

●●

●

●

●●

●

●●●●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●●●

●

●

●

●

●

●

●

●

●●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●●●

●

●

●

●

●

●●

●

●

●●●

●●

●●

●

●

●●

●

●

●

●●

●

●●

●

●●●●

●●●●●

●

●

●

●

●●

●●

●

●

●

●●●●

●●

●

●

●●●●●

●

●●●

●●

●●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●●●

●●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●

●

●

●

●

●●

●●

●

●

●

●●

●●●

●

●

●●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●●

●

●●●●●●

●

●●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

37226 2477 20375

53.83 44.32 53.22

disability

20

40

60

80

100

N Y NACategories

NN

Ras

ch


●

●

●

●

●

●

●

●

●

●●●●

●

●

●

●

●●

●●

●

●●

●

●

●●

●●●●●

●

●

●●

●

●

●

●●

●

●●

●

●●●●

●

●

●

●

●

●●●

●

●●

●●

●

●●●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●●●

●

●

●●

●●

●

●●

●●

●●

●●●●

●

●

●

●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●●

●

●

●●●

●

●

●

●

●

●

●

●●●

●

●●

●●

●

●●

●

●

●●●●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●●●

●

●

●●

●

●

●

●

●

●●

●

●●

●

●

●●●

●

●●

●●

●

●

●●●●●

●

●

●●

●

●●●●●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●●●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●●

●

●

●

●●●

●

●

●

●●●●

●

●●●

●

●

●●●

●●

●

●

●

●●

●●●

●

●●

●

●

●●

●

●●

●

●

●

●●●●

●

●

●●

●

●●

●

●●●

●

●

●

●

●●

36852 2785 20232

51.99 47.98 51.94

school_car

20

40

60

80

100

N Y NACategories

LL

Ras

ch

●

●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●

●

●

●●●●●

●●

●

●

●

●

●●

●

●●●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●●

●●

●

●

●

●●

●

●

●

●

●●

●●

●●●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●

●

●●

●●

●●

●

●●

●

●●●

●

●

●

●

●

●

●

●

●●

●●

●

●●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●●

●●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●●

●●

●

●

●

●

●●

●

●

●

●●

●

●

●●

●●

●

●

●●●

●

●●●

●

●

●●●

●

●●

●

●●●

●

●●●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●●●

●

●

●●

●

●

●●

●

●●

●

●●

●

●

●

●

●●

●

●●●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●●●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●●●

●

●●

●

●

●

●

●

●●

●●

●●●

●●

●●

●

●

●●●●

●

●●

●

●●

●

●●●●

●●●●●

●

●

●

●

●

●●

●

●

●

●

●●●

●●

●

●

●●●●●

●

●●●

●●

●●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●

●

●●

●●

●

●

●●●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●

●

●

●

●

●●

●●

●

●

●

●●

●●●

●

●

●●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●●

●

●●●●●●

●

●●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

36896 2807 20375

53.59 48.65 53.22

school_car

20

40

60

80

100

N Y NACategories

NN

Ras

ch


●

●

●

●●

●

●

●

●

●

●

●

●●

●

●●●●●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●●

●

●●

●

●●●

●

●

●●

●

●

●

●●

●

●●

●

●

●

●●

●

●●

●

●●

●

●

●

●

●

●

●

●●

●

●●●●●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●●●●

●

●

●

●

●●

●

●

●●

●

●

●●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●●

●

●●

●

●●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●●

●

●

●

●

●●●●

●

●

●●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●●

●

●

●●●

●

●

●

●

●

●

●

●

●●●●

●

●

●

●

●

●●●

●

●

●

●●●

●●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●●

●

●

●●

●

●●

●

●

●

●

●

●●

●

●●

●

●●●

●

●

●

●●●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●●●

●●

●●

●

●

●

●●

●

●

●

●

●

●●●

3062 2100 3611 3853 4669 3581 38993

50.84 54.89 53.24 52.59 51.08 50.47 51.68

occupation

20

40

60

80

100

0 1 2 3 4 8 NACategories

LL

Ras

ch

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●

●

●●●

●

●

●

●●

●●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●●

●●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●●●●

●

●● ●

●●●

●●

●

●

●

●●●

●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●●●

●●

●●●

●●

●

●●●

●

●

●

●●●

●

●●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●●●

●●

●

●

●●●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●●

●●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●

●

●

●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●●

●●

●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●●

●

●

●●

●

●

●

●●●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●●

●

●●●●

●

●

●

●●

●

●

●

●

●

●

●

●

●●●●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●●

●

●●

●

●

●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●●●

●●

●●●●●

●

●●

●●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●●

●●

●

●

●

●

●

●●●

●●

●

●●

●

●

●

●

●

●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●●

●

●●●●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●

●●

●

●

●

●

●●

●

●

●

●●

●●

●

●●

●●●

●

●

●

●

●

●

●

●

●

●●●●●

●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●●

●

3050 2069 3653 3858 4611 3622 39215

52.23 56.65 55 54.08 52.57 51.75 53.1

occupation

20

40

60

80

100

0 1 2 3 4 8 NACategories

NN

Ras

ch


●

●●

●

●

●●●

●

●●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●●

●

●

●●

●

●●

●

●●

●

●

●

●

●

●

●●

●●

●

●●

●

●

●

●●●

●

●

●

●

●●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●●

●

●●

●

●

●●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●●●●

●

●●

●

●●

●●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●●

●

●

●

●

●●●●

●

●

●●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●●

●

●

●●●

●

●

●

●

●

●

●

●

●●●●

●

●

●

●

●●●

●

●

●

●●●

●●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●●

●

●

●●

●

●●

●

●

●

●

●

●●

●

●●

●

●●●

●

●

●

●●●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●●●

●●

●●

●

●

●

●●

●

●

●

●

●

●●●

3224 961 3332 5784 7585 38983

51.1 48.44 50.17 51.66 53.76 51.7

school_edu

20

40

60

80

100

0 1 2 3 4 NACategories

LL

Ras

ch

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●●

●

●●

●

●

●●●

●●●

●

●

●

●●

●

●●●

●

●●●

●

●

●

●

●

●●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●●●●●●●●●

●

●

●

●

●

●

●●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●●

●

●

●●

●●

●

●●

●

●

●

●

●

●

●●

●●●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●●

●●

●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●●

●

●

●●

●

●

●

●●●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●●

●

●●●●

●

●

●

●●

●

●

●

●

●

●

●

●

●●●●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●●

●

●●

●

●

●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●●●

●●

●●●●●

●

●●

●●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●●

●●

●

●

●

●

●

●

●

●●

●●

●

●●

●

●

●

●

●

●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●●

●

●●●●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●

●●

●

●

●

●

●●

●

●

●

●●

●●

●

●●

●●●

●

●

●

●

●

●

●

●

●

●●●●●

●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●●

●

3236 985 3300 5761 7569 39227

52.55 50.15 51.48 53.23 55.32 53.11

school_edu

20

40

60

80

100


NN

Ras

ch


●●●●

●

●●●

●●

●

●

●

●●

●●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●●●

●

●

●●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●●

●

●●

●

●●

●●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●●

●

●

●

●

●●●●

●

●

●●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●●

●

●

●●●

●

●

●

●

●

●

●

●●

●●●●

●

●

●

●

●●●

●

●

●

●●●

●●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●●

●

●

●●

●

●●

●

●

●

●

●

●●

●

●●

●

●●●

●

●

●

●●●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●●●

●●

●●

●

●

●

●●

●

●

●

●

●

●●●

3875 4387 2095 2417 7695 39400

50.97 51.94 53.77 55.51 50.9 51.69

non_school

20

40

60

80

100


LL

Ras

ch

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●●

●

●

●

●●●

●●

●

●

●●

●

●

●●●

●

●

●

●●●●●●

●

●●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●●●●●

●

●

●

●●

●

●

●●

●

●

●

●

●

●●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●●

●

●

●

●●

●●●

●

●

●

●●

●●

●

●

●●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●●●

●

●

●

●

●

●●●

●

●

●●

●●

●●

●

●

●●

●●●●●●●●

●

●●

●

●●

●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●●

●

●

●●

●

●

●

●●●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●●

●

●●●●

●

●

●

●●

●

●

●

●

●

●

●

●

●●●●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●●

●

●●

●

●

●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●●

●●

●●●●●

●

●●

●●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●●

●●

●

●

●

●

●

●

●

●●

●●

●

●●

●

●

●

●

●

●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●●

●

●●●●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●

●●

●

●

●

●

●●

●

●

●

●●

●●

●

●●

●●●

●

●

●

●

●

●

●

●

●

●●●●●

●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●●

●

3883 4405 2077 2391 7687 39635

52.33 53.32 55.58 57.17 52.43 53.11

non_school

20

40

60

80

100


NN

Ras

ch


●

●

●

●

●●

●

●

●●●

●

●

●

●

●

●

●

●●

●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●●●

●

●

●

●

●●

●

●●

●

●

●●

●●●●

●

●

●●

●

●

●●

●

●

●

●●●

●

●

●

●

●

●

●

●●●

●●

●

●

●

●●●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●●

●●

●

●

●●

●●●

●

●

●●

●●

●

●

●

●

●

●●

●●●●●

●

●

●

●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

● ●

●●

●

●●●●●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●●●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●●

●

●

●

●●●

●

●

●

●●●●

●

●●●

●

●

●●●

●●

●

●

●

●●

●●●

●

●●

●

●

●●

●

●●

●

●

●

●●●●

●

●

●●

●

●●

●

●●●

●

●

●

●

●●

33096 6534 20239

51.65 52.01 51.94

p_g_gender

20

40

60

80

100

F M NACategories

LL

Ras

ch

●

●

●●

●

●

●

●

●●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●●●

●●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●●●

●●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●

●●●●●●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●●

●●

●●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●●

●●

●

●

●

●

●●

●

●

●

●●●●

●

●

●

●●

●

●

●

●●●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●●●

●●●

●

●

●

●

●

●

●●

●●

●●

●

●

●

●

●

●

●●

●●●

●

●

●

●

●

●●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●●

●

●

●

●●

●

●

●

●●

●

●

●

●●

●●●●

●●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●●●

●

●

●

●

●

●

●

●

●●

●●

●●

●

●●●●

●●●●

●

●

●●

●

●

●

●

●

●

●●

●

●

●●●●●

●

●●●

●●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●

●

●

●

●

●●

●●

●

●

●

●●

●●●

●

●

●●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●●

●

●●●●●●

●

●●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

33163 6533 20382

53.16 53.62 53.22

p_g_gender

20

40

60

80

100

F M NACategories

NN

Ras

ch


●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●●

●

●

●●

●●●●

●

●●

●●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●●

●●

●

●●

●

●

●●●●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●●

●

●●

●

●●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●

●

●

●●

●●●●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●●

●

●

●●●●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●●●

●

●

●●●

●●●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●●

●

●●●●●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●●●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●●

●

●

●

●●●

●

●

●

●●●●

●

●●●

●

●

●●●

●●

●

●

●

●●

●●●

●

●●

●

●

●●

●

●●

●

●

●

●●●●

●

●

●●

●

●●

●

●●●

●

●

●

●

●●

36377 3258 20234

51.69 51.94 51.94

p_g_nesb

20

40

60

80

100

N Y NACategories

LL

Ras

ch

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●●

●

●●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●●

●●

●

●

●●●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●●●

●

●

●

●●

●

●

●

●●

●

●

●●●

●

●●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●●

●●●●

●

●

●

●●●●●

●●

●●

●

●

●

●

●

●

●

●●

●

●

●

●●●

●●

●●

●●

●

●

●

●

●●

●

●

●

●

●

●

●●●

●

●

●

●

●

●●

●●

●

●●●

●

●

●

●

●

●●●

●

●

●

●

●●

●●●

●

●●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●●

●●

●

●

●

●

●●●

●●

●

●

●●

●

●

●

●

●●

●

●●●

●●

●

●

●

●

●●

●

●●

●

●

●

●●

●

●

●

●●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●●

●

●●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●

●

●

●

●

●●

●●

●

●

●

●●

●●●

●

●

●●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●●

●

●●●●●●

●

●●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

36394 3306 20378

53.22 53.46 53.22

p_g_nesb

20

40

60

80

100

N Y NACategories

NN

Ras

ch


●●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●●●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●●●●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●●●

●●

●

●

●

●

●

●

●

●●●

●

●

●●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●●●

●

●

●

●

●

●●

●

●

●

●

●●

●●

●

●●

●

●●

●

●

●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●●●●

●●

●

●

●

●

●

●

●

●

●

●

●

●●●●

●

●

●

●

●

●

●

●●●●

●●

●

●●

●

●●●●

●

●

●●

●

●

●

●●

●

●●

●●●

●

●●

●

●

●●●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●●●

●

●

●

●

●●●

●

●

●●●

●●

●

●

●

●

●

●

●●●

●

●

●●●●

●●

●

●

●

●●

●●●●●

●

●

●

●●

●

●

●●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

643 723 2588 1747 53 54115

43.75 52.87 52.18 52.24 52.05 51.83

nesb_code

20

40

60

80

100

A P1 P2 P3 TR NACategories

LL

Ras

ch

●

●●●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●●

●

●●

●●●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●

●●●●

●●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●●

●

●

●

●●●●●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●●

●

●

●

●

●

●●

●

●

●

●●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●

●●●●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●●

●●●

●●

●●●●●

●

●

●

●

●

●

●

●●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●

●●

●

●

●

●

●

●

●

●●

●●

●

●

●●

●

●

●

●

●

●

●●

●●

●

●●

●

●

●

●

●

●

●●●

●●

●

●

●

●●

●●

●

●●

●

●

●

●

●●●

●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●●

●

●

●●

●

●●

●

●

●

●

●●

●

●

●

●●●

●

●

●

●

●

●

●

●

●●●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●●

●

●

●●●

●●

●

●

●

●●

●

●

●

●●

●

●

●

●

●●

●●

●●

●

●

●

●

●

●

●●●

●●

●

●

●

●

●●

●

●

●●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●●●●●●●

●

●

●●

●●

●

●

●

●●

●

●

●

●

●

●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●●

●

●

●●

●

657 732 2622 1759 52 54256

44.07 54.68 53.55 53.51 55.04 53.3

nesb_code

20

40

60

80

100

A P1 P2 P3 TR NACategories

NN

Ras

ch


●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●●

●

●

●●

●●●●

●

●●

●

●●

●

●

●

●

●

●

●

●

●

●●●●●

●

●

●

●

●

●

●

●

●

●●

●●

●

●●

●●

●

●●●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●●

●●

●

●

●

●

●●

●

●●

●●●

●

●

●●●

●

●

●

●

●●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●●●

●

●

●

●

●

●●●

●

●●

●●

●

●

●●

●

●

●●●●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●●

●

●

●

●

●

●●

●

●

●

●●●

●

●

●●

●

●

●

●

●

●

●●

●

●●●

●

●

●

●

●●

●

●

●●

●

●●●●●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●●●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●●

●

●

●

●●●

●

●

●

●●●●

●

●●●

●

●

●●●

●●

●

●

●

●●

●●●

●

●●

●

●

●●

●

●●

●

●

●

●●●●

●

●

●●

●

●●

●

●●●

●

●

●

●

●●

36216 3420 20233

51.82 50.54 51.94

home_langu

20

40

60

80

100

N Y NACategories

LL

Ras

ch

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●●

●●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●

●●●●

●

●●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●●

●

●●

●●●●

●

●

●

●●●●●●

●●●

●

●

●

●

●

●

●●

●

●

●

●

●

●●●

●

●●●

●

●●

●

●

●

●●●

●

●

●

●

●

●

●●●

●

●

●

●

●

●●

●

●

●●●

●

●

●

●

●

●●●

●

●

●

●

●●

●●

●

●●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●

●

●

●

●

●●

●●

●

●

●

●●

●

●●

●

●●●●

●●●●

●

●

●

●

●●

●●

●

●

●

●●●

●

●

●

●

●●●●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●●

●

●

●

●

●●

●

●

●

●●

●

●●

●

●

●

●●●●

●●●●

●●

●

●

●

●

●

●

●

●

●

●●●●

●

●

●

●●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●

●

●

●

●

●●

●●

●

●

●

●●

●●●

●

●

●●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●

●●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●

●

●●●

●

●●●●●●

●

●●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

36227 3475 20376

53.38 51.74 53.22

home_langu

20

40

60

80

100

N Y NACategories

NN

Ras

ch



●● ●●● ● ●● ●●●●●● ●●● ●● ●●● ●●●● ● ●●● ●● ●●● ●●● ●● ●●● ●● ●●●● ●● ●●●● ●● ● ● ●●●● ●●●● ●●●● ● ●● ●● ●● ●●●●● ●●●●●●●●● ●●●●● ●●●●● ●●●● ●●●●●●●● ●●●●●●●●● ●● ●●●●● ●●● ●● ●

●●● ●●● ●● ●●●● ● ●●● ● ●● ●●● ● ●●● ●●● ●●● ●●●● ●● ●● ●● ●●●● ●● ●● ●●● ●●●● ●● ●● ●●●● ●● ● ●●● ● ●● ●●●● ●● ● ●● ●● ● ●●● ● ●●● ● ●●●●● ●● ●● ● ●●●● ●●● ● ● ●● ● ●●●●● ● ● ●●● ●●●● ● ●●● ●●● ●●●● ●●● ● ●● ●●●● ●● ●● ●● ●● ● ●●●● ●● ●●● ●●● ●●● ●●●●●● ● ● ●●● ●● ●●● ●●● ●●●●●● ●●●● ●●● ●●●● ●

●● ●● ●● ●● ●● ● ●● ● ●● ●● ●●● ● ●● ●●● ●●●●●● ●●●●●● ●●● ●●● ●●● ●● ●●●● ●●● ● ● ●●● ●● ●● ●● ●● ●●● ● ●●● ●● ●●●●●● ● ●●●● ● ●● ●● ●●● ●● ● ●● ●●● ●● ●●● ●● ●● ●●● ●● ● ●● ●●● ●●●●● ●● ● ●● ●●● ● ●● ●●●●●● ●● ●●●●● ●●●● ● ●● ●●● ● ● ●●●●●●● ●●● ●● ●●● ●●

● ● ●●● ●●●● ●●● ●● ● ●● ●●● ●● ●● ● ●● ● ●●●● ● ●●●● ● ● ●●● ●●●● ● ●● ●●●● ●●● ● ●●●● ●●● ● ● ●●●●●●●● ●●●● ●●● ●●●●●

●● ●●●● ●● ●●●●●● ●● ●●● ●●●●●●●● ● ●●● ●●●

56

5583

3682

0986

8390

8626

196

43.7

645

.85

42.1

148

.87

49.3

51.1

449

.47

49.0

552

.54

● ●●

● ●● ● ●●●●●●● ● ●● ●● ● ●● ●● ●● ●● ●●●●●●● ●●●● ●● ●● ● ●●● ●●● ●● ●● ●●● ●● ● ●●● ●● ●●●●●● ●●● ● ●●● ● ● ●●●●● ●●●●● ● ● ●●●● ●● ●● ●●

●● ●● ●● ●● ●●● ●● ●● ●● ●●●● ●●●● ●● ●●● ●●●●●● ●●●● ●● ● ● ● ●● ●● ● ●●●● ●● ● ●● ● ●●●● ● ●● ●● ●● ●● ●● ●●● ●● ●●●●● ●●●● ●●●●●●●● ● ●● ●●● ●●●●● ●● ● ●●● ●●● ●●● ●● ●●● ●● ●●●●● ●●● ● ●●●●● ●●● ●●● ●●●● ●

11

410

4989

2088

3226

712

65.4

51.7

49.9

853

.82

51.5

857

.05

59.1

958

.63

59.3

5

●

●

● ●●●● ●● ●●●● ●● ●● ●●●● ●●● ●●●●●●●

37

1725

542

55.9

964

.24

62.5

365

.48

75.1

4

35

7

20406080100

1997

1998

1999

2000

2001

2002

2003

2004

2005

1997

1998

1999

2000

2001

2002

2003

2004

2005

1997

1998

1999

2000

2001

2002

2003

2004

2005

Cat

egor

ies

NN Rasch

Figure

B.1.1:Boxplotof

LLRasch

against

procyearsplitinto

grades.

228 B.2. Grade 3 and Grade 5 Tests

B.2 Grade 3 and Grade 5 Tests

Before we investigate the progress in NN Rasch scores between grades, univariate

summary statistics of this data set are given in the form of plots and counts for each

of the explanatory variables (Figures B.2.1, B.2.2 and B.2.3).

We can see from these plots that there is no large imbalance between the groups of

the categorical variables, and the proportions of data in each group of a category

are reasonable and what we would expect based on what the groups represent in

real life.


●●●●●●●●●●●●

●●

●

●●

●

●●●●●●

●

●●●●

●●●●

●●

●●

●●

●●●●

●●●●

●●●●●●●●●

●●

●

●●

●

●●●●

●●●

●

●

●

●

●●

●●●●

●

●

●●

●

●

●

●

●

●●

●

●

●●

●

●●●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●●

●

●●●

●●

●

●●●

●●

●●

●

●

●

●

●●

●

●●●

●

●●

●

●●●

●●

●

●●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●●

●

●

●●

●●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●

●

●

●●

●

●

●●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

0

50

100

100 200 300 400 500 600School Number

Num

ber

of S

tude

nts

0

1000

2000

3000

0 100 200 300 400 500 600 700 800 900gpokm

Cou

nt

●●●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●●●●

●

●

●

●

●

●

●

●●

●

●

●

●●●

●

●●

●

●

●

●

●

●●

●●

●

●

●●●

●

●

●

●

●

●

●●●●●

●

●

●●

●

●

●●

●

●●

●

●●●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●

●

●

●●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●●

●

●●●

●

●

●

●

●

●

●

●

●●

●●

●

5210 600 170 119 144 537 209 498 152 231 17 7 13

−20

0

20

40

1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6 7 NAisolation

Diff

eren

ce in

Gra

de 3

and

5 s

core

s

●●●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

●●●●

●

●

●

●

●

●

●

●●

●

●

●

●●●

●

●●

●

●

●

●

●

●●

●●

●

●

●●●

●

●

●

●

●

●

●●●●●

●

●

●●

●

●

●●

●

●●

●

●●●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●●

●

●●

●

● ●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●●●●

●

●●

●

●

●

●●

●

●

●

●

●

●

●●●

●

●

●

●

5183 1006 1226 416 76

−20

0

20

40

1.1 2.2.1 2.2.2 3.1 3.2spatial_ar

Diff

eren

ce in

Gra

de 3

and

5 s

core

s

●●●

●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●●

●

●●●●

●

●

●

●

●

●

●●

●●

●

●

●

●●●

●

●●

●

●

●

●

●

●

●●

●●

●

●

●●●

●

●

●

●

●

●

●

●

●●●●●

●

●

●●

●

●

●

●●

●

●

●●

●

●●●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●●

●

●

●

●●●●●

●

●●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

5817 2090

−20

0

20

40

M Cstaff_metr

Diff

eren

ce in

Gra

de 3

and

5 s

core

s

●

●

●

●

●●●

●

●●

●●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●●

●●

●

●

●●

●

●●●●

●

●●

●

●

●

●

●

●●●

●

●

●●

●

●

●

●●

●

●

●

●●●

●

●

●

●

●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●●●●●●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

7675 155 77

−20

0

20

40

2 1 Inconsistentatsi

Diff

eren

ce in

Gra

de 3

and

5 s

core

s

Figure B.2.1: Plots of the number of students in each school, the distribution ofgpokm values and boxplots for the distribution of the di�erence in Grade 3 andGrade 5 scores in each category of isolation, spatial_ar, staff_metr and atsi.

230 B.2. Grade 3 and Grade 5 Tests

●

●

●

●

●●●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●

●●●

●●

●

●

●

●●●

●

●

●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●●

●

●

●

●

●

●

●●

●

●

●

●●

●

●●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●●●

●

●●

●

●

●

●

●

6662 197 1048

−20

0

20

40

2 1 Inconsistentlbote

Diff

eren

ce in

Gra

de 3

and

5 s

core

s

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●●●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●●●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●●●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●●●●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

3847 4060

−20

0

20

40

F Mgender

Diff

eren

ce in

Gra

de 3

and

5 s

core

s

●

●

●

●

●●●

●

●●

●●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●●

●●

●

●

●●

●

●●●●

●

●●

●

●

●

●

●

●●●

●

●

●●

●

●

●

●●

●

●

●

●●●

●

●●

●

●

●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●●●●●●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

7729 178

−20

0

20

40

N Yaboriginal

Diff

eren

ce in

Gra

de 3

and

5 s

core

s

●

●

●

●

●●●

●

●●

●

●●

●

●●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●●

●

●●●●

●

●●

●

●

●

●

●

●●

●

●

●●●

●

●

●●●

●

●●

●

●●

●

●

●

●

●

●

●

●

●●

●●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●●●●●●●●●

●

●

●●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

7497 410

−20

0

20

40

N Ydisability

Diff

eren

ce in

Gra

de 3

and

5 s

core

s

●

●

●

●

●

●●●

●

●●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●●

●●

●

●

●●

●

●●●●

●

●●

●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●

●

●●●

●

●●

●

●

●

●

●

●●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●●●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●

●

●●●

●

●

●

●

●

●

●●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

7776 131

−20

0

20

40

N Yschool_car

Diff

eren

ce in

Gra

de 3

and

5 s

core

s

●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●

●

●

●●●

●●

●

●

●

●●

●

●

●

●

●

●

● ●

●●●

●

●●

●●

●

●

●

●●●

●

●

●●●

●

●

●

●

●

●●●

●●

●

●

●●

●●

●

●

●

●●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●●●●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●●●●●●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

488 390 736 775 922 659 3937

−20

0

20

40

0 1 2 3 4 8 NAoccupation

Diff

eren

ce in

Gra

de 3

and

5 s

core

s

Figure B.2.2: Boxplots of the distribution of the di�erence in Grade 3 and Grade5 scores in each category of lbote, gender, aboriginal, disability, school_carand occupation.


●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●●●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●

●●

●●

●

●

●

●●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●●●●

●

●

●●

●

●

●

●●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●●●●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

506 198 643 1167 1442 3951

−20

0

20

40

0 1 2 3 4 NAschool_edu

Diff

eren

ce in

Gra

de 3

and

5 s

core

s

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

● ●

●

●

●●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●●●

●

●

●●

●●

●

●

●

●●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●●●●

●

●

●

●

●

●

●

●

●●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●●●●

●

●

●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

621 855 379 468 1551 4033

−20

0

20

40

0 5 6 7 8 NAnon_school

Diff

eren

ce in

Gra

de 3

and

5 s

core

s

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●●

●●

●

●

●

●●●●

●

●●

●

●

●

●

●●●

●

●

●

●

●

●

●●

●

●

●

●●

●

●●

●

●

●●

●

●

●●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●●●●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●●

●

●●

●

●●●●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●

●

●●

●

●

●●

●

●

●

●

●

6631 1 1275

−20

0

20

40

F Inconsistent Mp_g_gender

Diff

eren

ce in

Gra

de 3

and

5 s

core

s

●

●

●

●

●●●

●

●●

●●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●●

●

●

●

●●

●

●●●●

●●

●

●

●

●

●

●●●

●

●

●●

●

●

●

●●

●

●

●●●

●

●●

●

●

●

●

●●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●●●●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●

●

●

●●●●●●●

●●

●

●

●

●

●

7229 677 1

−20

0

20

40

N Y NAp_g_nesb

Diff

eren

ce in

Gra

de 3

and

5 s

core

s

●

●

●

●

●●●

●

●●

●●

●

●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●●

●

●●●

●●

●

●

●

●

●

●●●

●

●

●●

●

●

●

●●

●

●●●

●

●●

●●

●

●●

●

●

●

●●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●●●

●

●

●●

●

●

●

●

●●

●

●

●

●

●●

●

●

●

●

●

●

●

●●●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

7257 650

−20

0

20

40

N Yhome_langu

Diff

eren

ce in

Gra

de 3

and

5 s

core

s

0

200

400

600

−25 0 25Difference in Grade 3 and 5 scores

coun

t

Figure B.2.3: Boxplots of the distribution of the di�erence in Grade 3 and Grade5 scores in each category of school_edu, non_school, p_g_gender, p_g_nesb,home_langu and a histogram of the di�erences.

Appendix C

Output

C.1 Chapter 3: Initial Model Selection

C.1.1 Full model with Main E�ects

School-Number Model

Table C.1.1: Linear regression output of school-number

model

Estimate Std. Error t-value P -valueIntercept 47.7486 5.6792 8.41 0.0000schoolno28 0.1571 4.5314 0.03 0.9724schoolno29 4.5469 5.6263 0.81 0.4190schoolno30 3.8556 5.6407 0.68 0.4943schoolno31 3.7444 3.9395 0.95 0.3419schoolno33 4.9522 3.5406 1.40 0.1619schoolno35 5.9197 4.5155 1.31 0.1899schoolno37 1.4145 3.8369 0.37 0.7124schoolno39 5.4732 3.6301 1.51 0.1316schoolno41 1.5968 3.7544 0.43 0.6706schoolno42 4.0787 4.0731 1.00 0.3167schoolno43 0.2424 3.6884 0.07 0.9476schoolno46 -1.0136 7.3728 -0.14 0.8906schoolno47 12.1347 7.3682 1.65 0.0996schoolno48 6.2669 3.6287 1.73 0.0842schoolno50 1.2494 3.5813 0.35 0.7272

Continued

233

234 C.1. Chapter 3: Initial Model Selection

Estimate Std. Error t-value P -valueschoolno51 0.5469 3.8338 0.14 0.8866schoolno52 -4.0872 7.3676 -0.55 0.5791schoolno54 8.8168 3.9379 2.24 0.0252schoolno55 5.1139 3.8348 1.33 0.1824schoolno57 13.3749 5.6262 2.38 0.0175schoolno58 3.4265 3.9379 0.87 0.3842schoolno61 3.4908 3.3335 1.05 0.2950schoolno62 1.6237 5.6275 0.29 0.7729schoolno63 -3.7926 3.7516 -1.01 0.3121schoolno65 9.1776 3.5815 2.56 0.0104schoolno67 -3.5299 3.3798 -1.04 0.2963schoolno68 6.2562 4.0871 1.53 0.1259schoolno69 6.0980 3.8337 1.59 0.1117schoolno70 6.5003 3.6864 1.76 0.0779schoolno72 8.2400 3.7550 2.19 0.0282schoolno73 2.0625 3.4007 0.61 0.5442schoolno75 0.1222 3.5394 0.03 0.9725schoolno76 -1.6420 3.6864 -0.45 0.6560schoolno77 -0.9603 3.5050 -0.27 0.7841schoolno78 -3.4845 4.2555 -0.82 0.4129schoolno79 0.8353 3.5394 0.24 0.8134schoolno82 -7.3652 5.6290 -1.31 0.1907schoolno83 3.3035 3.3995 0.97 0.3312schoolno84 0.9581 4.2538 0.23 0.8218schoolno85 1.0542 3.7516 0.28 0.7787schoolno87 -0.5206 3.6865 -0.14 0.8877schoolno89 6.9128 3.2100 2.15 0.0313schoolno90 -0.9232 3.7504 -0.25 0.8056schoolno91 6.0872 3.5416 1.72 0.0857schoolno94 1.4965 3.6833 0.41 0.6845schoolno95 4.7751 3.4469 1.39 0.1660schoolno96 7.8580 3.4465 2.28 0.0226schoolno98 5.5181 3.6279 1.52 0.1283schoolno99 8.6240 3.4225 2.52 0.0117schoolno100 0.8888 3.5417 0.25 0.8018schoolno101 10.5028 7.3660 1.43 0.1539schoolno102 0.5603 3.5405 0.16 0.8743schoolno103 8.0802 5.6484 1.43 0.1526schoolno105 5.1498 3.3474 1.54 0.1240schoolno106 2.9847 3.3635 0.89 0.3749schoolno107 2.2185 3.5802 0.62 0.5355schoolno108 5.9548 3.6844 1.62 0.1061

Continued

Appendix C. Output 235

Estimate Std. Error t-value P -valueschoolno109 5.1012 3.5799 1.42 0.1542schoolno111 4.7210 3.5047 1.35 0.1780schoolno112 7.9641 3.4216 2.33 0.0199schoolno114 6.7200 4.0735 1.65 0.0990schoolno115 -5.7713 3.4333 -1.68 0.0928schoolno116 2.9168 4.0724 0.72 0.4738schoolno117 2.6615 3.8340 0.69 0.4876schoolno118 3.9719 3.5792 1.11 0.2671schoolno120 1.1958 3.2566 0.37 0.7135schoolno123 5.0767 3.2870 1.54 0.1225schoolno124 6.2976 3.5409 1.78 0.0753schoolno125 3.9511 3.2491 1.22 0.2240schoolno126 0.0335 3.4733 0.01 0.9923schoolno127 4.4203 3.4471 1.28 0.1997schoolno128 3.6990 3.3458 1.11 0.2689schoolno129 4.9709 3.3466 1.49 0.1375schoolno130 2.4257 3.4234 0.71 0.4786schoolno134 7.0893 3.2580 2.18 0.0296schoolno135 3.9610 3.2409 1.22 0.2216schoolno138 -5.0144 4.0715 -1.23 0.2181schoolno139 1.4080 4.9215 0.29 0.7748schoolno140 6.9575 3.9388 1.77 0.0773schoolno141 10.7094 3.5815 2.99 0.0028schoolno142 11.9450 3.2842 3.64 0.0003schoolno143 -5.3684 3.9388 -1.36 0.1729schoolno146 1.7381 3.2767 0.53 0.5958schoolno147 4.7648 3.2156 1.48 0.1384schoolno148 5.5869 3.3825 1.65 0.0986schoolno149 5.0012 3.2953 1.52 0.1291schoolno150 0.2350 3.4224 0.07 0.9452schoolno151 5.6582 4.5129 1.25 0.2099schoolno152 -0.4698 3.2063 -0.15 0.8835schoolno153 7.2582 3.2649 2.22 0.0262schoolno154 -8.4475 4.5135 -1.87 0.0613schoolno155 5.1327 4.9110 1.05 0.2960schoolno157 2.2857 3.4000 0.67 0.5014schoolno158 -0.4009 4.9421 -0.08 0.9354schoolno159 -5.9718 3.7055 -1.61 0.1071schoolno160 7.6924 3.2655 2.36 0.0185schoolno161 6.6706 3.1782 2.10 0.0358schoolno162 1.7818 3.1960 0.56 0.5772schoolno164 5.7277 3.2045 1.79 0.0739

Continued


Estimate Std. Error t-value P -valueschoolno166 -0.8678 3.4733 -0.25 0.8027schoolno167 4.2271 3.5822 1.18 0.2380schoolno168 7.1711 3.4221 2.10 0.0361schoolno169 0.6549 3.2952 0.20 0.8425schoolno170 -29.5639 7.4356 -3.98 0.0001schoolno171 4.7492 3.3637 1.41 0.1580schoolno172 1.8342 3.7509 0.49 0.6248schoolno173 0.4599 3.6852 0.12 0.9007schoolno175 -4.0080 4.9128 -0.82 0.4146schoolno182 9.0645 3.3067 2.74 0.0061schoolno183 3.2030 3.5804 0.89 0.3710schoolno184 6.1008 3.3059 1.85 0.0650schoolno185 1.8111 3.2004 0.57 0.5715schoolno186 -0.1246 3.2492 -0.04 0.9694schoolno187 2.7968 3.2057 0.87 0.3830schoolno188 -1.3095 3.2216 -0.41 0.6844schoolno189 0.6575 3.1614 0.21 0.8353schoolno190 2.5114 3.2222 0.78 0.4358schoolno191 0.5747 3.2102 0.18 0.8579schoolno192 0.9603 3.2418 0.30 0.7671schoolno193 -2.0599 3.6314 -0.57 0.5705schoolno194 1.4172 3.2957 0.43 0.6672schoolno197 3.0334 3.8340 0.79 0.4288schoolno199 4.5873 3.2412 1.42 0.1570schoolno200 0.2154 3.2888 0.07 0.9478schoolno201 3.8909 3.2000 1.22 0.2240schoolno205 2.4210 3.2109 0.75 0.4509schoolno206 -1.9494 5.6268 -0.35 0.7290schoolno207 3.7387 3.4003 1.10 0.2716schoolno213 4.9947 3.1745 1.57 0.1157schoolno214 3.1983 3.2111 1.00 0.3193schoolno219 2.2386 3.1499 0.71 0.4773schoolno220 3.8820 4.9137 0.79 0.4295schoolno221 0.1310 3.2434 0.04 0.9678schoolno222 -3.0544 3.3832 -0.90 0.3666schoolno223 -1.7803 3.3084 -0.54 0.5905schoolno228 1.3550 3.1706 0.43 0.6691schoolno229 2.6077 3.5859 0.73 0.4671schoolno230 4.8285 3.1958 1.51 0.1308schoolno232 6.7412 3.6835 1.83 0.0672schoolno234 -1.4607 3.2485 -0.45 0.6530schoolno236 1.9408 3.1100 0.62 0.5326

Continued


Estimate Std. Error t-value P -valueschoolno237 5.8013 3.1872 1.82 0.0687schoolno239 6.7499 4.2542 1.59 0.1126schoolno244 5.4242 3.4474 1.57 0.1156schoolno245 6.7909 3.6874 1.84 0.0655schoolno246 -1.1589 3.1270 -0.37 0.7109schoolno247 4.4078 3.1496 1.40 0.1617schoolno248 2.8513 3.4236 0.83 0.4049schoolno249 2.7977 3.1270 0.89 0.3710schoolno251 4.3967 3.5076 1.25 0.2101schoolno252 3.1747 3.1399 1.01 0.3120schoolno253 2.8194 3.2559 0.87 0.3865schoolno256 0.5576 3.5554 0.16 0.8754schoolno257 1.6125 3.2570 0.50 0.6206schoolno259 3.6128 3.1053 1.16 0.2447schoolno261 0.1234 3.3064 0.04 0.9702schoolno262 1.7145 3.1902 0.54 0.5910schoolno263 3.1970 3.3659 0.95 0.3422schoolno267 0.8550 3.1282 0.27 0.7846schoolno268 4.6606 3.1125 1.50 0.1343schoolno270 4.2796 3.2748 1.31 0.1913schoolno271 5.9590 3.3658 1.77 0.0767schoolno273 1.8308 3.2050 0.57 0.5678schoolno274 -0.2437 3.9405 -0.06 0.9507schoolno277 5.4636 3.1103 1.76 0.0790schoolno278 -6.5504 3.2663 -2.01 0.0449schoolno279 1.3768 3.1819 0.43 0.6652schoolno282 2.3303 3.1962 0.73 0.4660schoolno283 0.8498 3.1271 0.27 0.7858schoolno284 2.7437 3.2424 0.85 0.3975schoolno287 -2.6342 3.1702 -0.83 0.4060schoolno294 6.7520 3.1577 2.14 0.0325schoolno295 4.3675 3.1267 1.40 0.1625schoolno296 3.7361 3.3829 1.10 0.2694schoolno297 4.1978 3.0987 1.35 0.1755schoolno298 0.5533 3.1710 0.17 0.8615schoolno299 3.4217 3.1079 1.10 0.2709schoolno300 0.6895 3.1206 0.22 0.8251schoolno304 3.3616 3.3197 1.01 0.3113schoolno305 4.6618 3.2570 1.43 0.1524schoolno308 1.1713 3.0792 0.38 0.7036schoolno310 5.1310 3.9403 1.30 0.1929schoolno311 1.7612 3.1286 0.56 0.5735

Continued


Estimate Std. Error t-value P -valueschoolno312 -2.2225 3.2222 -0.69 0.4904schoolno313 -0.8512 3.1449 -0.27 0.7867schoolno315 1.7648 3.4508 0.51 0.6091schoolno316 1.4131 3.5077 0.40 0.6871schoolno318 3.2811 3.1608 1.04 0.2993schoolno319 -4.0144 3.2011 -1.25 0.2098schoolno320 3.9318 3.2666 1.20 0.2287schoolno322 -3.0411 3.1750 -0.96 0.3382schoolno323 5.6739 3.2216 1.76 0.0782schoolno324 7.8083 3.2426 2.41 0.0160schoolno325 3.8914 3.1586 1.23 0.2180schoolno326 3.0823 3.1396 0.98 0.3262schoolno330 5.1860 3.1517 1.65 0.0999schoolno333 3.1121 3.8358 0.81 0.4172schoolno334 1.4260 3.1345 0.45 0.6492schoolno335 -1.1864 3.2219 -0.37 0.7127schoolno336 3.5505 3.2780 1.08 0.2788schoolno337 2.4189 3.0970 0.78 0.4348schoolno338 2.1043 3.6846 0.57 0.5679schoolno339 3.2929 3.6272 0.91 0.3640schoolno343 5.0027 3.1580 1.58 0.1132schoolno344 5.9441 3.1442 1.89 0.0587schoolno350 3.2758 3.0729 1.07 0.2864schoolno351 2.7616 3.1724 0.87 0.3840schoolno353 2.7985 3.0802 0.91 0.3636schoolno354 3.5926 3.1871 1.13 0.2597schoolno355 -2.5078 3.1791 -0.79 0.4302schoolno356 4.8283 3.1648 1.53 0.1271schoolno357 3.8749 3.1490 1.23 0.2185schoolno358 2.9551 3.0746 0.96 0.3365schoolno359 3.1912 3.0581 1.04 0.2967schoolno361 -1.0107 3.1713 -0.32 0.7500schoolno362 -0.3901 3.3191 -0.12 0.9064schoolno363 5.1625 3.7527 1.38 0.1689schoolno364 0.5717 3.6858 0.16 0.8767schoolno365 1.7795 3.0878 0.58 0.5644schoolno366 0.9063 3.1363 0.29 0.7726schoolno369 1.6739 3.2103 0.52 0.6021schoolno371 1.3169 3.2652 0.40 0.6867schoolno372 7.5559 3.3324 2.27 0.0234schoolno373 6.2729 3.1047 2.02 0.0433schoolno374 1.0516 3.0659 0.34 0.7316

Continued


Estimate Std. Error t-value P -valueschoolno376 2.5931 3.1555 0.82 0.4112schoolno377 1.6561 3.1029 0.53 0.5935schoolno378 0.1973 3.1724 0.06 0.9504schoolno380 4.2199 3.1170 1.35 0.1758schoolno381 2.4499 3.1298 0.78 0.4338schoolno382 1.3117 3.3630 0.39 0.6965schoolno385 1.3966 3.0686 0.46 0.6490schoolno386 -1.5732 3.2130 -0.49 0.6244schoolno387 4.6276 3.1126 1.49 0.1371schoolno388 -0.0468 3.0824 -0.02 0.9879schoolno389 2.6390 3.1723 0.83 0.4055schoolno390 0.3540 3.1322 0.11 0.9100schoolno391 0.2360 3.1166 0.08 0.9396schoolno392 5.3734 3.1324 1.72 0.0863schoolno393 3.0122 3.0984 0.97 0.3310schoolno395 1.2291 3.1223 0.39 0.6938schoolno396 1.6342 3.1779 0.51 0.6071schoolno398 1.5470 3.1946 0.48 0.6282schoolno399 3.9328 3.1778 1.24 0.2159schoolno400 0.7742 3.0856 0.25 0.8019schoolno401 4.2432 3.0766 1.38 0.1678schoolno403 14.3568 3.9423 3.64 0.0003schoolno405 1.3110 3.1006 0.42 0.6724schoolno406 1.3256 3.1742 0.42 0.6762schoolno407 6.0747 3.1059 1.96 0.0505schoolno410 -0.5258 3.2520 -0.16 0.8715schoolno413 5.4367 3.1088 1.75 0.0803schoolno414 -4.3770 3.1899 -1.37 0.1700schoolno415 2.3525 3.1951 0.74 0.4616schoolno416 3.4705 3.0982 1.12 0.2627schoolno417 2.2290 3.1199 0.71 0.4750schoolno418 1.0450 3.0641 0.34 0.7331schoolno419 0.6430 3.0879 0.21 0.8350schoolno420 0.2528 3.1272 0.08 0.9356schoolno421 3.2875 3.1649 1.04 0.2989schoolno422 4.7373 3.2757 1.45 0.1481schoolno423 -1.0888 3.0946 -0.35 0.7250schoolno425 5.5177 3.1182 1.77 0.0768schoolno426 -3.3561 3.1369 -1.07 0.2847schoolno427 0.3736 3.1896 0.12 0.9068schoolno428 3.7068 3.0642 1.21 0.2264schoolno430 1.5252 3.1098 0.49 0.6238

Continued


Estimate Std. Error t-value P -valueschoolno431 1.9002 3.1456 0.60 0.5458schoolno432 4.7849 3.0830 1.55 0.1207schoolno434 6.0440 4.9166 1.23 0.2190schoolno435 -0.5413 3.0796 -0.18 0.8605schoolno437 2.1655 4.5132 0.48 0.6314schoolno439 1.4502 3.2408 0.45 0.6545schoolno441 5.0041 3.0773 1.63 0.1039schoolno442 -0.0845 3.6273 -0.02 0.9814schoolno444 -1.6722 3.0577 -0.55 0.5845schoolno445 3.2054 3.2025 1.00 0.3169schoolno446 4.8288 3.3384 1.45 0.1481schoolno447 3.0954 3.0810 1.00 0.3151schoolno448 0.9600 3.2172 0.30 0.7654schoolno450 6.0598 3.9443 1.54 0.1245schoolno451 5.3828 3.3093 1.63 0.1038schoolno453 5.1076 3.0642 1.67 0.0956schoolno454 4.6462 3.1824 1.46 0.1443schoolno455 -0.0602 3.1216 -0.02 0.9846schoolno456 3.4684 3.1310 1.11 0.2680schoolno457 3.1505 3.0614 1.03 0.3034schoolno458 2.6376 3.1329 0.84 0.3998schoolno459 2.3616 3.6288 0.65 0.5152schoolno460 4.9692 3.0581 1.62 0.1042schoolno461 -1.1094 3.1442 -0.35 0.7242schoolno465 2.0878 3.2158 0.65 0.5162schoolno466 3.7381 3.0915 1.21 0.2266schoolno467 2.3920 3.4751 0.69 0.4913schoolno468 2.5430 3.0494 0.83 0.4043schoolno469 1.6296 3.1190 0.52 0.6013schoolno470 2.6142 3.1150 0.84 0.4014schoolno471 -1.5927 3.1080 -0.51 0.6083schoolno472 5.5645 3.0910 1.80 0.0718schoolno473 4.1886 3.2360 1.29 0.1956schoolno474 -1.1872 3.1203 -0.38 0.7036schoolno475 -0.2483 3.1750 -0.08 0.9377schoolno476 0.9584 3.1779 0.30 0.7630schoolno477 -0.8998 3.0810 -0.29 0.7702schoolno478 5.1094 3.0992 1.65 0.0992schoolno479 3.2154 3.1036 1.04 0.3002schoolno480 3.0265 3.0780 0.98 0.3255schoolno482 2.6452 3.1417 0.84 0.3998schoolno483 5.2972 3.1056 1.71 0.0881

Continued


Estimate Std. Error t-value P -valueschoolno484 -1.2949 3.0573 -0.42 0.6719schoolno485 5.3582 3.0621 1.75 0.0802schoolno486 5.1688 3.0448 1.70 0.0896schoolno489 5.8512 3.0896 1.89 0.0583schoolno490 2.3584 3.1490 0.75 0.4539schoolno491 3.1782 3.0670 1.04 0.3001schoolno492 2.4884 3.0975 0.80 0.4218schoolno493 4.4993 3.1250 1.44 0.1500schoolno494 -0.0883 3.0853 -0.03 0.9772schoolno495 4.8883 3.0632 1.60 0.1106schoolno496 3.8424 3.2235 1.19 0.2333schoolno497 2.7840 3.2769 0.85 0.3956schoolno499 -4.0485 4.5115 -0.90 0.3695schoolno500 2.5901 3.2378 0.80 0.4237schoolno502 1.9598 3.1033 0.63 0.5277schoolno503 0.4524 3.4034 0.13 0.8942schoolno504 3.3198 3.3103 1.00 0.3159schoolno505 4.1517 3.1551 1.32 0.1882schoolno508 3.6686 3.0655 1.20 0.2314schoolno509 4.0238 3.0875 1.30 0.1925schoolno510 5.5255 3.1609 1.75 0.0805schoolno512 -1.3319 3.0446 -0.44 0.6618schoolno515 5.5230 3.1286 1.77 0.0775schoolno516 1.5992 3.1055 0.51 0.6066schoolno517 1.3896 3.0803 0.45 0.6519schoolno519 -1.6443 3.0904 -0.53 0.5947schoolno521 5.1379 3.0881 1.66 0.0962schoolno522 3.0168 3.0541 0.99 0.3233schoolno523 4.7379 3.1181 1.52 0.1287schoolno524 0.8445 3.0710 0.27 0.7833schoolno525 0.4174 3.2852 0.13 0.8989schoolno526 3.7348 3.0453 1.23 0.2201schoolno527 7.9240 3.0575 2.59 0.0096schoolno528 7.7967 3.1289 2.49 0.0127schoolno529 3.8457 3.1242 1.23 0.2184schoolno531 2.7117 3.0615 0.89 0.3758schoolno532 2.9516 3.1353 0.94 0.3465schoolno533 2.4911 3.0663 0.81 0.4166schoolno534 3.7111 3.3227 1.12 0.2641schoolno536 -0.4293 3.1811 -0.13 0.8926schoolno537 -1.9230 7.3784 -0.26 0.7944schoolno538 3.3116 3.1305 1.06 0.2901

Continued


Estimate Std. Error t-value P -valueschoolno539 7.1896 3.0825 2.33 0.0197schoolno540 5.6380 3.1829 1.77 0.0765schoolno541 4.0501 3.9392 1.03 0.3039schoolno542 2.2298 3.0681 0.73 0.4674schoolno543 1.6655 3.0881 0.54 0.5897schoolno545 7.6255 3.0832 2.47 0.0134schoolno546 7.5842 3.0694 2.47 0.0135schoolno547 2.8910 3.0486 0.95 0.3430schoolno548 2.6776 3.0409 0.88 0.3786schoolno549 1.0646 4.0805 0.26 0.7942schoolno550 3.6627 3.2884 1.11 0.2654schoolno551 2.4242 3.2961 0.74 0.4621schoolno552 6.3693 3.0542 2.09 0.0370schoolno553 5.0309 3.1524 1.60 0.1105schoolno554 -1.7526 3.1632 -0.55 0.5796schoolno555 -1.2211 3.0681 -0.40 0.6906schoolno556 6.7398 3.0587 2.20 0.0276schoolno557 3.8511 3.0671 1.26 0.2093schoolno558 4.3927 3.0603 1.44 0.1512schoolno559 0.3754 3.1071 0.12 0.9038schoolno561 -0.8744 3.0870 -0.28 0.7770schoolno564 1.3889 3.1969 0.43 0.6640schoolno566 6.4768 3.0596 2.12 0.0343schoolno567 2.6849 3.0493 0.88 0.3786schoolno568 4.9512 3.1105 1.59 0.1115schoolno569 3.0122 3.0932 0.97 0.3302schoolno571 5.2225 3.0512 1.71 0.0870schoolno573 5.6567 3.0832 1.83 0.0666schoolno574 4.4645 3.0659 1.46 0.1454schoolno575 3.0786 3.1864 0.97 0.3340schoolno576 4.5801 3.0528 1.50 0.1336schoolno578 0.2534 3.0747 0.08 0.9343schoolno581 4.9604 3.0531 1.62 0.1042schoolno584 4.4964 3.0811 1.46 0.1445schoolno593 7.1204 3.0890 2.31 0.0212schoolno595 1.7282 3.1294 0.55 0.5808schoolno596 1.5750 3.0952 0.51 0.6109schoolno597 1.7715 3.1233 0.57 0.5706schoolno599 1.8692 3.3204 0.56 0.5735schoolno600 0.9382 3.1206 0.30 0.7637schoolno608 1.8778 3.1328 0.60 0.5489schoolno614 3.9615 3.2064 1.24 0.2167

Continued


Estimate Std. Error t-value P -valueschoolno623 8.1360 3.2739 2.49 0.0130schoolno624 3.7130 3.0691 1.21 0.2264schoolno628 4.3240 4.9277 0.88 0.3802schoolno637 1.3941 7.3773 0.19 0.8501schoolno639 0.6531 3.1128 0.21 0.8338schoolno649 3.3143 3.0412 1.09 0.2758schoolno650 2.8320 3.0410 0.93 0.3517procyear1999 -7.4803 6.2054 -1.21 0.2280procyear2000 -1.1543 4.8209 -0.24 0.8108procyear2001 -1.0363 4.8162 -0.22 0.8296procyear2002 -0.0091 4.8153 -0.00 0.9985procyear2003 -0.6325 4.8150 -0.13 0.8955procyear2004 -1.2816 4.8161 -0.27 0.7902procyear2005 4.8264 5.6888 0.85 0.3962gradedyear5 8.8438 0.1211 73.03 0.0000gradedyear7 16.4989 0.4210 39.19 0.0000atsi1 -2.0826 0.9092 -2.29 0.0220atsiInconsistent -1.9754 0.7824 -2.52 0.0116lbote1 -0.7619 0.2969 -2.57 0.0103lboteInconsistent -0.4326 0.2114 -2.05 0.0408genderM 1.0748 0.0965 11.14 0.0000aboriginalY -1.3835 0.8713 -1.59 0.1123disabilityY -8.1355 0.2147 -37.89 0.0000school_carY -0.7971 0.1900 -4.20 0.0000occupation1 0.7338 0.2961 2.48 0.0132occupation2 0.6044 0.2495 2.42 0.0154occupation3 0.6260 0.2426 2.58 0.0099occupation4 0.1599 0.2398 0.67 0.5049occupation8 0.1782 0.2466 0.72 0.4699school_edu1 -1.6512 0.3769 -4.38 0.0000school_edu2 -1.3228 0.3263 -4.05 0.0001school_edu3 -0.4219 0.3151 -1.34 0.1805school_edu4 0.2808 0.3172 0.89 0.3760non_school5 0.6838 0.2784 2.46 0.0140non_school6 1.5297 0.3052 5.01 0.0000non_school7 2.2191 0.3262 6.80 0.0000non_school8 0.6472 0.2688 2.41 0.0161p_g_genderM 0.0331 0.1415 0.23 0.8148p_g_nesbY -0.2325 0.2241 -1.04 0.2995



Table C.1.2: Linear regression output of school-covariates

model

Estimate Std. Error t-value P -valueIntercept 52.0977 4.9951 10.43 0.0000procyear1999 -9.6148 6.4424 -1.49 0.1356procyear2000 -2.8535 4.9975 -0.57 0.5680procyear2001 -2.5657 4.9929 -0.51 0.6073procyear2002 -1.4896 4.9919 -0.30 0.7654procyear2003 -1.9823 4.9916 -0.40 0.6913procyear2004 -2.5689 4.9925 -0.51 0.6069procyear2005 2.6786 5.9040 0.45 0.6501gradedyear5 8.6832 0.1235 70.33 0.0000gradedyear7 16.1488 0.4296 37.59 0.0000gpokm -0.0095 0.0020 -4.70 0.0000isolation1.5 0.0667 0.7499 0.09 0.9292isolation2 -2.3020 1.1960 -1.92 0.0543isolation2.5 1.8365 1.1942 1.54 0.1241isolation3 4.0460 1.2784 3.16 0.0016isolation3.5 1.8044 1.2401 1.46 0.1457isolation4 2.1361 1.2956 1.65 0.0992isolation4.5 3.8550 1.3818 2.79 0.0053isolation5 3.2281 1.6227 1.99 0.0467isolation5.5 2.8284 1.6350 1.73 0.0837isolation6 1.5005 1.9891 0.75 0.4507isolation6.5 12.9792 5.3562 2.42 0.0154isolation7 3.4680 3.0958 1.12 0.2626spatial_ar2.2.1 0.2797 0.7323 0.38 0.7025spatial_ar2.2.2 -0.2527 0.8225 -0.31 0.7586spatial_ar3.1 1.5225 0.9615 1.58 0.1133spatial_ar3.2 3.5129 1.1601 3.03 0.0025staff_metrC -0.5350 0.8650 -0.62 0.5363atsi1 -2.5740 0.9317 -2.76 0.0057atsiInconsistent -2.7961 0.8021 -3.49 0.0005lbote1 -0.9101 0.3009 -3.03 0.0025lboteInconsistent -0.4461 0.2177 -2.05 0.0405genderM 1.1482 0.0997 11.51 0.0000aboriginalY -1.7597 0.8980 -1.96 0.0501disabilityY -8.5044 0.2194 -38.76 0.0000school_carY -1.0799 0.1941 -5.56 0.0000occupation1 1.0991 0.2971 3.70 0.0002

Continued


Estimate Std. Error t-value P -valueoccupation2 1.1309 0.2469 4.58 0.0000occupation3 0.9514 0.2386 3.99 0.0001occupation4 0.1673 0.2356 0.71 0.4777occupation8 -0.2730 0.2412 -1.13 0.2578school_edu1 -2.5957 0.3774 -6.88 0.0000school_edu2 -1.9995 0.3248 -6.16 0.0000school_edu3 -0.6940 0.3122 -2.22 0.0262school_edu4 0.1199 0.3154 0.38 0.7038non_school5 0.6393 0.2828 2.26 0.0238non_school6 1.9020 0.3111 6.11 0.0000non_school7 2.9241 0.3313 8.83 0.0000non_school8 0.6556 0.2725 2.41 0.0161p_g_genderM 0.1324 0.1378 0.96 0.3368p_g_nesbY -0.0612 0.2256 -0.27 0.7863school size 0.0027 0.0006 4.50 0.0000

C.1.2 Simplest Main E�ects Model by stepAIC

School-Number Model

Table C.1.3: Linear regression output of simplest-

stepAIC school-number model

Estimate Std. Error t-value P -valueIntercept 47.6956 5.6789 8.40 0.0000schoolno28 0.1298 4.5312 0.03 0.9771schoolno29 4.5563 5.6261 0.81 0.4180schoolno30 3.8292 5.6405 0.68 0.4972schoolno31 3.7557 3.9389 0.95 0.3404schoolno33 4.9393 3.5403 1.40 0.1630schoolno35 5.9071 4.5153 1.31 0.1908schoolno37 1.3998 3.8366 0.36 0.7152schoolno39 5.4714 3.6300 1.51 0.1318schoolno41 1.6201 3.7529 0.43 0.6660schoolno42 4.0759 4.0730 1.00 0.3170schoolno43 0.2474 3.6882 0.07 0.9465schoolno46 -1.0366 7.3725 -0.14 0.8882

Continued


Estimate Std. Error t-value P -valueschoolno47 12.1278 7.3679 1.65 0.0998schoolno48 6.2516 3.6286 1.72 0.0849schoolno50 1.2425 3.5811 0.35 0.7286schoolno51 0.5476 3.8336 0.14 0.8864schoolno52 -4.0675 7.3666 -0.55 0.5808schoolno54 8.8114 3.9378 2.24 0.0253schoolno55 5.1184 3.8347 1.33 0.1820schoolno57 13.3651 5.6259 2.38 0.0175schoolno58 3.4324 3.9376 0.87 0.3834schoolno61 3.4959 3.3334 1.05 0.2943schoolno62 1.6111 5.6273 0.29 0.7746schoolno63 -3.7869 3.7513 -1.01 0.3128schoolno65 9.1532 3.5814 2.56 0.0106schoolno67 -3.5359 3.3797 -1.05 0.2955schoolno68 6.2356 4.0870 1.53 0.1271schoolno69 6.0993 3.8336 1.59 0.1116schoolno70 6.4951 3.6863 1.76 0.0781schoolno72 8.2562 3.7539 2.20 0.0279schoolno73 2.0540 3.4005 0.60 0.5458schoolno75 0.1120 3.5391 0.03 0.9748schoolno76 -1.6383 3.6861 -0.44 0.6567schoolno77 -0.9693 3.5049 -0.28 0.7821schoolno78 -3.4953 4.2553 -0.82 0.4114schoolno79 0.8270 3.5392 0.23 0.8152schoolno82 -7.3763 5.6288 -1.31 0.1901schoolno83 3.2914 3.3992 0.97 0.3329schoolno84 0.9529 4.2537 0.22 0.8227schoolno85 1.0530 3.7515 0.28 0.7789schoolno87 -0.5449 3.6863 -0.15 0.8825schoolno89 6.9095 3.2099 2.15 0.0314schoolno90 -0.9300 3.7503 -0.25 0.8042schoolno91 6.0765 3.5414 1.72 0.0862schoolno94 1.4872 3.6831 0.40 0.6864schoolno95 4.7626 3.4467 1.38 0.1670schoolno96 7.8500 3.4463 2.28 0.0228schoolno98 5.5130 3.6278 1.52 0.1286schoolno99 8.6220 3.4224 2.52 0.0118schoolno100 0.8799 3.5416 0.25 0.8038schoolno101 10.5297 7.3650 1.43 0.1528schoolno102 0.5488 3.5403 0.16 0.8768schoolno103 7.9413 5.6467 1.41 0.1596schoolno105 5.1454 3.3473 1.54 0.1243

Continued


Estimate Std. Error t-value P -valueschoolno106 2.9852 3.3634 0.89 0.3748schoolno107 2.2188 3.5801 0.62 0.5354schoolno108 5.9581 3.6842 1.62 0.1059schoolno109 5.0911 3.5797 1.42 0.1550schoolno111 4.7218 3.5047 1.35 0.1779schoolno112 7.9655 3.4214 2.33 0.0199schoolno114 6.7180 4.0734 1.65 0.0991schoolno115 -5.9170 3.4304 -1.72 0.0846schoolno116 2.9028 4.0721 0.71 0.4760schoolno117 2.6718 3.8338 0.70 0.4859schoolno118 3.9554 3.5790 1.11 0.2691schoolno120 1.1938 3.2565 0.37 0.7139schoolno123 5.0875 3.2868 1.55 0.1217schoolno124 6.2868 3.5407 1.78 0.0758schoolno125 3.9472 3.2490 1.21 0.2244schoolno126 0.0260 3.4731 0.01 0.9940schoolno127 4.4244 3.4471 1.28 0.1993schoolno128 3.6925 3.3456 1.10 0.2698schoolno129 4.9571 3.3463 1.48 0.1385schoolno130 2.4341 3.4231 0.71 0.4770schoolno134 7.1105 3.2571 2.18 0.0290schoolno135 3.9581 3.2408 1.22 0.2220schoolno138 -5.0161 4.0714 -1.23 0.2180schoolno139 1.3848 4.9212 0.28 0.7784schoolno140 6.9682 3.9384 1.77 0.0769schoolno141 10.7102 3.5807 2.99 0.0028schoolno142 11.9442 3.2841 3.64 0.0003schoolno143 -5.3560 3.9384 -1.36 0.1739schoolno146 1.7535 3.2756 0.54 0.5924schoolno147 4.7589 3.2155 1.48 0.1389schoolno148 5.5767 3.3822 1.65 0.0992schoolno149 4.9914 3.2951 1.51 0.1298schoolno150 0.2331 3.4221 0.07 0.9457schoolno151 5.6633 4.5126 1.25 0.2095schoolno152 -0.4730 3.2062 -0.15 0.8827schoolno153 7.2519 3.2648 2.22 0.0263schoolno154 -8.4579 4.5132 -1.87 0.0609schoolno155 5.1231 4.9108 1.04 0.2969schoolno157 2.2855 3.4000 0.67 0.5015schoolno158 -0.5163 4.9404 -0.10 0.9168schoolno159 -5.9615 3.7051 -1.61 0.1076schoolno160 7.6868 3.2655 2.35 0.0186

Continued


Estimate Std. Error t-value P -valueschoolno161 6.6646 3.1781 2.10 0.0360schoolno162 1.7743 3.1959 0.56 0.5788schoolno164 5.7251 3.2043 1.79 0.0740schoolno166 -0.8863 3.4731 -0.26 0.7986schoolno167 4.2336 3.5820 1.18 0.2373schoolno168 7.1609 3.4219 2.09 0.0364schoolno169 0.6536 3.2951 0.20 0.8428schoolno170 -29.7337 7.4337 -4.00 0.0001schoolno171 4.7435 3.3635 1.41 0.1585schoolno172 1.8272 3.7507 0.49 0.6261schoolno173 0.4634 3.6850 0.13 0.8999schoolno175 -4.1648 4.9103 -0.85 0.3963schoolno182 9.0594 3.3065 2.74 0.0062schoolno183 3.1964 3.5799 0.89 0.3719schoolno184 6.0953 3.3058 1.84 0.0652schoolno185 1.8039 3.2003 0.56 0.5730schoolno186 -0.1367 3.2489 -0.04 0.9664schoolno187 2.7856 3.2055 0.87 0.3849schoolno188 -1.3074 3.2215 -0.41 0.6849schoolno189 0.6503 3.1613 0.21 0.8370schoolno190 2.5044 3.2221 0.78 0.4370schoolno191 0.5663 3.2100 0.18 0.8600schoolno192 0.9753 3.2413 0.30 0.7635schoolno193 -2.0791 3.6312 -0.57 0.5669schoolno194 1.3729 3.2953 0.42 0.6770schoolno197 3.0255 3.8338 0.79 0.4300schoolno199 4.5792 3.2411 1.41 0.1577schoolno200 0.2107 3.2886 0.06 0.9489schoolno201 3.8907 3.1999 1.22 0.2240schoolno205 2.4312 3.2108 0.76 0.4489schoolno206 -1.9548 5.6266 -0.35 0.7283schoolno207 3.6986 3.4000 1.09 0.2767schoolno213 4.9901 3.1743 1.57 0.1160schoolno214 3.1633 3.2108 0.99 0.3245schoolno219 2.2419 3.1498 0.71 0.4766schoolno220 3.8708 4.9135 0.79 0.4308schoolno221 0.0739 3.2428 0.02 0.9818schoolno222 -3.0539 3.3831 -0.90 0.3667schoolno223 -1.8056 3.3082 -0.55 0.5852schoolno228 1.3483 3.1705 0.43 0.6707schoolno229 2.6178 3.5858 0.73 0.4654schoolno230 4.8136 3.1957 1.51 0.1320

Continued


Estimate Std. Error t-value P -valueschoolno232 6.7392 3.6834 1.83 0.0673schoolno234 -1.4697 3.2483 -0.45 0.6509schoolno236 1.9394 3.1098 0.62 0.5329schoolno237 5.7902 3.1869 1.82 0.0693schoolno239 6.7441 4.2540 1.59 0.1129schoolno244 5.4167 3.4473 1.57 0.1161schoolno245 6.7847 3.6872 1.84 0.0658schoolno246 -1.1548 3.1269 -0.37 0.7119schoolno247 4.3991 3.1495 1.40 0.1625schoolno248 2.8150 3.4233 0.82 0.4109schoolno249 2.7923 3.1269 0.89 0.3719schoolno251 4.3642 3.5074 1.24 0.2134schoolno252 3.1714 3.1399 1.01 0.3125schoolno253 2.7887 3.2557 0.86 0.3917schoolno256 0.6151 3.5545 0.17 0.8626schoolno257 1.6155 3.2569 0.50 0.6199schoolno259 3.6149 3.1052 1.16 0.2444schoolno261 0.1222 3.3063 0.04 0.9705schoolno262 1.7179 3.1901 0.54 0.5902schoolno263 3.2034 3.3657 0.95 0.3412schoolno267 0.8557 3.1281 0.27 0.7844schoolno268 4.6739 3.1115 1.50 0.1331schoolno270 4.2378 3.2744 1.29 0.1956schoolno271 5.9480 3.3655 1.77 0.0772schoolno273 1.8192 3.2048 0.57 0.5703schoolno274 -0.2258 3.9403 -0.06 0.9543schoolno277 5.4525 3.1101 1.75 0.0796schoolno278 -6.5546 3.2662 -2.01 0.0448schoolno279 1.3446 3.1817 0.42 0.6726schoolno280 -0.4351 3.4000 -0.13 0.8982schoolno282 2.3352 3.1961 0.73 0.4650schoolno283 0.8504 3.1270 0.27 0.7857schoolno284 2.7336 3.2423 0.84 0.3992schoolno287 -2.6446 3.1701 -0.83 0.4041schoolno294 6.7458 3.1576 2.14 0.0327schoolno295 4.3402 3.1265 1.39 0.1651schoolno296 3.7182 3.3827 1.10 0.2717schoolno297 4.1823 3.0985 1.35 0.1771schoolno298 0.5467 3.1709 0.17 0.8631schoolno299 3.4152 3.1078 1.10 0.2718schoolno300 0.6570 3.1203 0.21 0.8332schoolno304 3.3648 3.3196 1.01 0.3108

Continued


Estimate Std. Error t-value P -valueschoolno305 4.6654 3.2569 1.43 0.1520schoolno308 1.1465 3.0790 0.37 0.7096schoolno310 5.1124 3.9402 1.30 0.1945schoolno311 1.7520 3.1285 0.56 0.5755schoolno312 -2.2264 3.2221 -0.69 0.4896schoolno313 -0.8578 3.1448 -0.27 0.7850schoolno315 1.7199 3.4505 0.50 0.6182schoolno316 1.4264 3.5075 0.41 0.6842schoolno318 3.2718 3.1608 1.04 0.3006schoolno319 -4.0323 3.2009 -1.26 0.2078schoolno320 3.9354 3.2665 1.20 0.2283schoolno322 -3.0682 3.1748 -0.97 0.3338schoolno323 5.6675 3.2214 1.76 0.0785schoolno324 7.7782 3.2424 2.40 0.0165schoolno325 3.8617 3.1584 1.22 0.2215schoolno326 3.0736 3.1395 0.98 0.3276schoolno330 5.1585 3.1515 1.64 0.1017schoolno333 3.0346 3.8350 0.79 0.4288schoolno334 1.4320 3.1336 0.46 0.6477schoolno335 -1.1942 3.2217 -0.37 0.7109schoolno336 3.5465 3.2777 1.08 0.2793schoolno337 2.3608 3.0964 0.76 0.4458schoolno338 2.1193 3.6843 0.58 0.5651schoolno339 3.2839 3.6271 0.91 0.3653schoolno343 4.9970 3.1578 1.58 0.1136schoolno344 5.9199 3.1439 1.88 0.0597schoolno350 3.2475 3.0726 1.06 0.2906schoolno351 2.7575 3.1722 0.87 0.3847schoolno353 2.7936 3.0801 0.91 0.3644schoolno354 3.5830 3.1870 1.12 0.2609schoolno355 -2.5098 3.1790 -0.79 0.4298schoolno356 4.8066 3.1646 1.52 0.1288schoolno357 3.8552 3.1489 1.22 0.2209schoolno358 2.9451 3.0744 0.96 0.3381schoolno359 3.1284 3.0574 1.02 0.3062schoolno361 -1.0123 3.1710 -0.32 0.7495schoolno362 -0.4200 3.3189 -0.13 0.8993schoolno363 5.1733 3.7526 1.38 0.1680schoolno364 0.5278 3.6854 0.14 0.8861schoolno365 1.7708 3.0877 0.57 0.5663schoolno366 0.8973 3.1361 0.29 0.7748schoolno369 1.6698 3.2101 0.52 0.6030

Continued


Estimate Std. Error t-value P -valueschoolno371 1.3240 3.2651 0.41 0.6851schoolno372 7.5513 3.3323 2.27 0.0235schoolno373 6.2829 3.1040 2.02 0.0430schoolno374 1.0364 3.0658 0.34 0.7353schoolno376 2.5771 3.1553 0.82 0.4141schoolno377 1.6481 3.1027 0.53 0.5953schoolno378 0.1893 3.1722 0.06 0.9524schoolno380 4.2161 3.1169 1.35 0.1762schoolno381 2.4430 3.1297 0.78 0.4351schoolno382 1.2892 3.3628 0.38 0.7014schoolno385 1.3968 3.0685 0.46 0.6490schoolno386 -1.6821 3.2113 -0.52 0.6004schoolno387 4.6132 3.1125 1.48 0.1383schoolno388 -0.0479 3.0823 -0.02 0.9876schoolno389 2.6012 3.1720 0.82 0.4122schoolno390 0.3592 3.1321 0.11 0.9087schoolno391 0.2273 3.1164 0.07 0.9419schoolno392 5.3213 3.1320 1.70 0.0893schoolno393 3.0063 3.0983 0.97 0.3319schoolno395 1.2243 3.1222 0.39 0.6950schoolno396 1.6182 3.1778 0.51 0.6106schoolno398 1.5466 3.1945 0.48 0.6283schoolno399 3.9162 3.1777 1.23 0.2178schoolno400 0.7737 3.0854 0.25 0.8020schoolno401 4.2432 3.0765 1.38 0.1678schoolno403 14.3479 3.9421 3.64 0.0003schoolno405 1.2966 3.1004 0.42 0.6758schoolno406 1.3192 3.1740 0.42 0.6777schoolno407 6.0722 3.1056 1.96 0.0506schoolno410 -0.5611 3.2517 -0.17 0.8630schoolno413 5.4084 3.1085 1.74 0.0819schoolno414 -4.3832 3.1898 -1.37 0.1694schoolno415 2.3385 3.1949 0.73 0.4642schoolno416 3.4502 3.0981 1.11 0.2654schoolno417 2.2172 3.1197 0.71 0.4773schoolno418 1.0364 3.0639 0.34 0.7352schoolno419 0.6477 3.0878 0.21 0.8339schoolno420 0.2428 3.1270 0.08 0.9381schoolno421 3.2805 3.1648 1.04 0.2999schoolno422 4.6411 3.2743 1.42 0.1564schoolno423 -1.1250 3.0943 -0.36 0.7162schoolno425 5.5126 3.1181 1.77 0.0771

Continued


Estimate Std. Error t-value P -valueschoolno426 -3.3656 3.1368 -1.07 0.2833schoolno427 0.3725 3.1895 0.12 0.9070schoolno428 3.7065 3.0641 1.21 0.2264schoolno430 1.5350 3.1096 0.49 0.6216schoolno431 1.8948 3.1455 0.60 0.5469schoolno432 4.7913 3.0827 1.55 0.1201schoolno434 6.0664 4.9164 1.23 0.2172schoolno435 -0.5344 3.0791 -0.17 0.8622schoolno437 2.1535 4.5130 0.48 0.6332schoolno439 1.4394 3.2405 0.44 0.6569schoolno441 5.0018 3.0772 1.63 0.1041schoolno442 -0.0914 3.6271 -0.03 0.9799schoolno444 -1.6984 3.0576 -0.56 0.5786schoolno445 3.1921 3.2023 1.00 0.3189schoolno446 4.7216 3.3368 1.41 0.1571schoolno447 3.1140 3.0802 1.01 0.3121schoolno448 0.9052 3.2165 0.28 0.7784schoolno450 6.0471 3.9442 1.53 0.1253schoolno451 5.3899 3.3092 1.63 0.1034schoolno453 5.1086 3.0642 1.67 0.0955schoolno454 4.6219 3.1822 1.45 0.1464schoolno455 -0.0900 3.1213 -0.03 0.9770schoolno456 3.4586 3.1308 1.10 0.2693schoolno457 3.1273 3.0612 1.02 0.3070schoolno458 2.6263 3.1327 0.84 0.4018schoolno459 2.3046 3.6283 0.64 0.5253schoolno460 4.9541 3.0580 1.62 0.1052schoolno461 -1.1123 3.1441 -0.35 0.7235schoolno465 2.0958 3.2157 0.65 0.5146schoolno466 3.7297 3.0913 1.21 0.2276schoolno467 2.3839 3.4748 0.69 0.4927schoolno468 2.5353 3.0493 0.83 0.4057schoolno469 1.6219 3.1188 0.52 0.6030schoolno470 2.5993 3.1149 0.83 0.4040schoolno471 -1.5854 3.1079 -0.51 0.6100schoolno472 5.5347 3.0907 1.79 0.0734schoolno473 4.1856 3.2358 1.29 0.1959schoolno474 -1.1859 3.1203 -0.38 0.7039schoolno475 -0.2782 3.1748 -0.09 0.9302schoolno476 0.9492 3.1777 0.30 0.7652schoolno477 -0.9017 3.0809 -0.29 0.7698schoolno478 5.1010 3.0992 1.65 0.0998

Continued


Estimate Std. Error t-value P -valueschoolno479 3.2097 3.1034 1.03 0.3010schoolno480 3.0172 3.0778 0.98 0.3269schoolno482 2.6385 3.1416 0.84 0.4010schoolno483 5.2869 3.1055 1.70 0.0887schoolno484 -1.2960 3.0572 -0.42 0.6716schoolno485 5.3252 3.0618 1.74 0.0820schoolno486 5.1551 3.0446 1.69 0.0904schoolno489 5.8045 3.0891 1.88 0.0603schoolno490 2.3393 3.1488 0.74 0.4575schoolno491 3.1687 3.0668 1.03 0.3015schoolno492 2.4389 3.0971 0.79 0.4310schoolno493 4.4446 3.1245 1.42 0.1549schoolno494 -0.0913 3.0852 -0.03 0.9764schoolno495 4.8883 3.0632 1.60 0.1105schoolno496 3.8203 3.2233 1.19 0.2359schoolno497 2.7242 3.2764 0.83 0.4057schoolno499 -4.0607 4.5113 -0.90 0.3681schoolno500 2.5866 3.2377 0.80 0.4244schoolno502 1.9420 3.1031 0.63 0.5314schoolno503 0.4335 3.4033 0.13 0.8986schoolno504 3.3446 3.3101 1.01 0.3123schoolno505 4.1188 3.1548 1.31 0.1917schoolno508 3.6405 3.0654 1.19 0.2350schoolno509 4.0209 3.0874 1.30 0.1928schoolno510 5.4792 3.1605 1.73 0.0830schoolno512 -1.3521 3.0445 -0.44 0.6570schoolno515 5.4852 3.1283 1.75 0.0795schoolno516 1.5929 3.1053 0.51 0.6080schoolno517 1.3724 3.0801 0.45 0.6559schoolno519 -1.6646 3.0902 -0.54 0.5901schoolno521 5.1300 3.0879 1.66 0.0967schoolno522 2.9797 3.0538 0.98 0.3292schoolno523 4.7345 3.1179 1.52 0.1289schoolno524 0.8219 3.0709 0.27 0.7890schoolno525 0.4222 3.2850 0.13 0.8977schoolno526 3.6854 3.0449 1.21 0.2261schoolno527 7.8937 3.0573 2.58 0.0098schoolno528 7.7792 3.1287 2.49 0.0129schoolno529 3.7919 3.1237 1.21 0.2248schoolno531 2.7061 3.0614 0.88 0.3767schoolno532 2.9375 3.1351 0.94 0.3488schoolno533 2.5026 3.0656 0.82 0.4143

Continued


Estimate Std. Error t-value P -valueschoolno534 3.7029 3.3226 1.11 0.2651schoolno536 -0.4485 3.1809 -0.14 0.8879schoolno537 -1.9351 7.3781 -0.26 0.7931schoolno538 3.3089 3.1304 1.06 0.2905schoolno539 7.1565 3.0823 2.32 0.0203schoolno540 5.6395 3.1827 1.77 0.0764schoolno541 4.0435 3.9390 1.03 0.3047schoolno542 2.1985 3.0678 0.72 0.4736schoolno543 1.6588 3.0880 0.54 0.5912schoolno545 7.6262 3.0831 2.47 0.0134schoolno546 7.5591 3.0692 2.46 0.0138schoolno547 2.8631 3.0484 0.94 0.3476schoolno548 2.6712 3.0408 0.88 0.3797schoolno549 1.1338 4.0794 0.28 0.7811schoolno550 3.6286 3.2882 1.10 0.2698schoolno551 2.4262 3.2960 0.74 0.4617schoolno552 6.3619 3.0539 2.08 0.0372schoolno553 5.0251 3.1523 1.59 0.1109schoolno554 -1.7544 3.1631 -0.55 0.5791schoolno555 -1.2070 3.0676 -0.39 0.6940schoolno556 6.7217 3.0586 2.20 0.0280schoolno557 3.7858 3.0664 1.23 0.2170schoolno558 4.3892 3.0601 1.43 0.1515schoolno559 0.3728 3.1069 0.12 0.9045schoolno561 -0.9191 3.0866 -0.30 0.7659schoolno564 1.3350 3.1963 0.42 0.6762schoolno566 6.4576 3.0594 2.11 0.0348schoolno567 2.6820 3.0492 0.88 0.3791schoolno568 4.9458 3.1104 1.59 0.1118schoolno569 3.0043 3.0930 0.97 0.3314schoolno571 5.1969 3.0510 1.70 0.0885schoolno573 5.6541 3.0831 1.83 0.0667schoolno574 4.4532 3.0657 1.45 0.1464schoolno575 3.0355 3.1860 0.95 0.3407schoolno576 4.5824 3.0526 1.50 0.1333schoolno578 0.2445 3.0745 0.08 0.9366schoolno581 4.9465 3.0530 1.62 0.1052schoolno584 4.4659 3.0809 1.45 0.1472schoolno593 7.0630 3.0885 2.29 0.0222schoolno595 1.7208 3.1293 0.55 0.5824schoolno596 1.5451 3.0950 0.50 0.6176schoolno597 1.7531 3.1232 0.56 0.5746

Continued


Estimate Std. Error t-value P -valueschoolno599 1.8717 3.3203 0.56 0.5730schoolno600 0.9324 3.1205 0.30 0.7651schoolno608 1.8767 3.1328 0.60 0.5491schoolno614 3.9622 3.2063 1.24 0.2166schoolno623 8.1361 3.2738 2.49 0.0130schoolno624 3.7002 3.0689 1.21 0.2279schoolno628 4.3604 4.9275 0.88 0.3762schoolno637 1.3828 7.3771 0.19 0.8513schoolno639 0.6321 3.1127 0.20 0.8391schoolno649 3.3126 3.0411 1.09 0.2760schoolno650 2.8238 3.0409 0.93 0.3531procyear1999 -7.3968 6.2040 -1.19 0.2332procyear2000 -1.0910 4.8204 -0.23 0.8209procyear2001 -0.9709 4.8156 -0.20 0.8402procyear2002 0.0545 4.8147 0.01 0.9910procyear2003 -0.5724 4.8144 -0.12 0.9054procyear2004 -1.2166 4.8155 -0.25 0.8005procyear2005 4.8278 5.6882 0.85 0.3960gradedyear5 8.8460 0.1211 73.06 0.0000gradedyear7 16.5014 0.4210 39.19 0.0000atsi1 -2.0708 0.9090 -2.28 0.0227atsiInconsistent -1.9754 0.7823 -2.52 0.0116lbote1 -0.8875 0.2688 -3.30 0.0010lboteInconsistent -0.5128 0.1956 -2.62 0.0088genderM 1.0740 0.0965 11.13 0.0000aboriginalY -1.3763 0.8711 -1.58 0.1141disabilityY -8.1348 0.2147 -37.89 0.0000school_carY -0.7994 0.1900 -4.21 0.0000occupation1 0.7467 0.2955 2.53 0.0115occupation2 0.6149 0.2491 2.47 0.0136occupation3 0.6320 0.2425 2.61 0.0092occupation4 0.1628 0.2397 0.68 0.4970occupation8 0.1776 0.2465 0.72 0.4713school_edu1 -1.6505 0.3765 -4.38 0.0000school_edu2 -1.3199 0.3263 -4.05 0.0001school_edu3 -0.4206 0.3151 -1.33 0.1819school_edu4 0.2754 0.3172 0.87 0.3853non_school5 0.6797 0.2783 2.44 0.0146non_school6 1.5241 0.3051 5.00 0.0000non_school7 2.2135 0.3262 6.79 0.0000non_school8 0.6443 0.2687 2.40 0.0165



Table C.1.4: Linear regression output of simplest-

stepAIC school-covariates model

Estimate Std. Error t-value P -valueIntercept 52.0345 4.9936 10.42 0.0000procyear1999 -9.4744 6.4401 -1.47 0.1413procyear2000 -2.7752 4.9959 -0.56 0.5786procyear2001 -2.4876 4.9914 -0.50 0.6182procyear2002 -1.4163 4.9904 -0.28 0.7766procyear2003 -1.9102 4.9902 -0.38 0.7019procyear2004 -2.4984 4.9910 -0.50 0.6167procyear2005 2.7650 5.9031 0.47 0.6395gradedyear5 8.6885 0.1233 70.45 0.0000gradedyear7 16.1636 0.4294 37.65 0.0000gpokm -0.0095 0.0020 -4.73 0.0000isolation1.5 0.0513 0.7490 0.07 0.9454isolation2 -2.8263 0.8268 -3.42 0.0006isolation2.5 1.3203 0.8363 1.58 0.1144isolation3 3.5421 0.9538 3.71 0.0002isolation3.5 1.2990 0.9061 1.43 0.1517isolation4 1.6305 0.9882 1.65 0.0990isolation4.5 3.3442 1.1102 3.01 0.0026isolation5 2.7441 1.4021 1.96 0.0503isolation5.5 2.3524 1.4190 1.66 0.0974isolation6 0.9823 1.8149 0.54 0.5883isolation6.5 12.4128 5.2948 2.34 0.0191isolation7 2.9338 2.9986 0.98 0.3279spatial_ar2.2.1 0.2784 0.7322 0.38 0.7038spatial_ar2.2.2 -0.2642 0.8223 -0.32 0.7480spatial_ar3.1 1.4888 0.9604 1.55 0.1211spatial_ar3.2 3.5158 1.1594 3.03 0.0024atsi1 -2.5664 0.9315 -2.76 0.0059atsiInconsistent -2.8092 0.8019 -3.50 0.0005lbote1 -0.9289 0.2663 -3.49 0.0005lboteInconsistent -0.4597 0.1994 -2.31 0.0212genderM 1.1475 0.0997 11.50 0.0000aboriginalY -1.7650 0.8978 -1.97 0.0493disabilityY -8.5013 0.2193 -38.76 0.0000

Continued


Estimate Std. Error t-value P -valueschool_carY -1.0801 0.1940 -5.57 0.0000occupation1 1.1197 0.2963 3.78 0.0002occupation2 1.1458 0.2462 4.65 0.0000occupation3 0.9535 0.2385 4.00 0.0001occupation4 0.1710 0.2355 0.73 0.4679occupation8 -0.2811 0.2411 -1.17 0.2437school_edu1 -2.5863 0.3771 -6.86 0.0000school_edu2 -1.9975 0.3247 -6.15 0.0000school_edu3 -0.7029 0.3121 -2.25 0.0243school_edu4 0.1094 0.3153 0.35 0.7287non_school5 0.6498 0.2826 2.30 0.0215non_school6 1.8989 0.3110 6.11 0.0000non_school7 2.9271 0.3312 8.84 0.0000non_school8 0.6571 0.2723 2.41 0.0158school size 0.0027 0.0006 4.54 0.0000

258 C.2. Chapter 7. Initial Longitudinal Analysis

C.2 Chapter 7. Initial Longitudinal Analysis

C.2.1 Grade 3 and Grade 5 Tests

School-Number Model

Table C.2.1: Linear regression output of school-number

model

Estimate Std. Error t-value P -valueIntercept 27.5375 2.2463 12.26 0.0000Grade 3 Rasch -0.4165 0.0116 -35.98 0.0000schoolno41 -3.3498 3.4570 -0.97 0.3326schoolno61 2.6747 2.7700 0.97 0.3343schoolno67 -3.8928 3.1762 -1.23 0.2204schoolno73 2.4323 2.8618 0.85 0.3954schoolno75 0.4934 2.9885 0.17 0.8689schoolno77 2.3795 2.8656 0.83 0.4064schoolno79 1.1626 3.4486 0.34 0.7361schoolno84 -1.0347 5.1748 -0.20 0.8415schoolno85 3.1634 3.4496 0.92 0.3592schoolno89 4.2497 2.6961 1.58 0.1151schoolno91 0.0796 3.1811 0.03 0.9800schoolno94 -1.4702 3.4528 -0.43 0.6703schoolno95 3.4377 2.9869 1.15 0.2498schoolno96 5.2491 2.9926 1.75 0.0795schoolno102 -0.2318 3.4515 -0.07 0.9464schoolno105 3.0337 2.9895 1.01 0.3103schoolno106 0.5589 2.9884 0.19 0.8517schoolno114 3.9561 5.1851 0.76 0.4455schoolno120 1.5995 2.5894 0.62 0.5368schoolno123 2.9313 3.1716 0.92 0.3554schoolno125 1.5546 2.8678 0.54 0.5878schoolno127 2.9639 3.1671 0.94 0.3494schoolno128 0.2100 3.1683 0.07 0.9472schoolno130 1.4211 3.4478 0.41 0.6802schoolno134 -3.2760 2.9953 -1.09 0.2741schoolno135 5.0405 3.1729 1.59 0.1122schoolno142 2.5581 2.7671 0.92 0.3553schoolno146 -0.6618 2.8725 -0.23 0.8178schoolno147 2.2232 2.6935 0.83 0.4092schoolno148 -3.2607 2.9913 -1.09 0.2758

Continued


Estimate Std. Error t-value P -valueschoolno149 2.7406 2.9859 0.92 0.3588schoolno152 0.5063 2.5522 0.20 0.8428schoolno153 5.2729 2.6935 1.96 0.0504schoolno155 2.1262 5.1698 0.41 0.6809schoolno157 -0.2197 3.4549 -0.06 0.9493schoolno160 2.1037 2.6958 0.78 0.4352schoolno161 3.2662 2.5153 1.30 0.1942schoolno162 0.5663 2.5190 0.22 0.8221schoolno164 3.3550 2.5883 1.30 0.1950schoolno167 -1.8634 3.4544 -0.54 0.5896schoolno168 1.2154 3.4479 0.35 0.7245schoolno169 0.0965 3.0079 0.03 0.9744schoolno171 -0.7345 3.1678 -0.23 0.8167schoolno173 -6.1143 5.2076 -1.17 0.2404schoolno182 5.9586 2.7650 2.15 0.0312schoolno183 -2.5980 3.9579 -0.66 0.5116schoolno184 4.5057 2.9870 1.51 0.1315schoolno185 -1.0891 3.1678 -0.34 0.7310schoolno186 -4.4783 2.6943 -1.66 0.0966schoolno187 2.7415 2.5546 1.07 0.2833schoolno188 -0.3549 2.9960 -0.12 0.9057schoolno189 -0.2038 2.4876 -0.08 0.9347schoolno190 1.9854 2.6605 0.75 0.4556schoolno191 3.0232 2.6940 1.12 0.2618schoolno192 5.1594 3.9585 1.30 0.1925schoolno194 2.6734 2.8638 0.93 0.3506schoolno199 0.0507 2.6910 0.02 0.9850schoolno200 2.6400 2.9954 0.88 0.3782schoolno201 2.3024 2.9943 0.77 0.4420schoolno205 1.0756 2.8699 0.37 0.7078schoolno207 4.8783 3.4521 1.41 0.1577schoolno213 0.3102 2.5525 0.12 0.9033schoolno214 2.1754 2.5949 0.84 0.4019schoolno219 0.8520 2.5551 0.33 0.7388schoolno220 -3.9159 5.1839 -0.76 0.4501schoolno222 -5.9487 3.1750 -1.87 0.0611schoolno223 -2.8115 2.7843 -1.01 0.3127schoolno228 -0.4804 2.6968 -0.18 0.8586schoolno229 3.2888 3.1914 1.03 0.3028schoolno230 0.4597 2.7739 0.17 0.8684schoolno234 -1.7129 3.4496 -0.50 0.6195schoolno236 1.2898 2.3547 0.55 0.5839

Continued


Estimate Std. Error t-value P -valueschoolno237 2.9109 2.5915 1.12 0.2614schoolno239 5.2102 3.9528 1.32 0.1876schoolno245 6.8807 3.9601 1.74 0.0824schoolno246 -2.9716 2.4639 -1.21 0.2279schoolno247 2.4799 2.5172 0.99 0.3246schoolno248 5.6010 3.9723 1.41 0.1586schoolno249 -0.4386 2.4422 -0.18 0.8575schoolno252 -0.4268 2.4887 -0.17 0.8638schoolno253 0.1520 2.9913 0.05 0.9595schoolno257 3.3020 2.9887 1.10 0.2693schoolno259 3.0165 2.4241 1.24 0.2135schoolno261 -1.9133 2.9898 -0.64 0.5222schoolno262 -0.9195 2.6395 -0.35 0.7276schoolno263 6.3809 5.1852 1.23 0.2186schoolno267 1.2429 2.3954 0.52 0.6039schoolno268 2.0100 2.4337 0.83 0.4089schoolno271 1.3086 2.7859 0.47 0.6386schoolno273 1.0984 2.6370 0.42 0.6770schoolno274 0.0431 5.1819 0.01 0.9934schoolno277 3.7277 2.4450 1.52 0.1274schoolno278 -0.9703 2.8728 -0.34 0.7356schoolno279 2.5167 2.8620 0.88 0.3793schoolno282 2.9430 2.7001 1.09 0.2758schoolno283 -2.8740 2.5883 -1.11 0.2669schoolno287 -1.2622 3.2250 -0.39 0.6955schoolno294 -1.5972 2.6954 -0.59 0.5535schoolno295 0.6431 2.4867 0.26 0.7960schoolno296 0.5594 3.1758 0.18 0.8602schoolno297 2.9898 2.3317 1.28 0.1998schoolno298 1.3354 2.5925 0.52 0.6065schoolno299 3.1302 2.3754 1.32 0.1877schoolno300 -0.6422 2.5183 -0.26 0.7987schoolno304 0.1323 3.1747 0.04 0.9668schoolno305 3.0189 3.4476 0.88 0.3813schoolno308 -1.1533 2.3099 -0.50 0.6176schoolno310 0.0723 3.9662 0.02 0.9855schoolno311 1.3373 2.8626 0.47 0.6404schoolno312 -1.3073 2.6436 -0.49 0.6210schoolno313 -2.5934 2.5188 -1.03 0.3033schoolno315 2.8474 3.5101 0.81 0.4173schoolno316 -2.0683 3.4634 -0.60 0.5504schoolno318 -1.5782 2.9950 -0.53 0.5983

Continued


Estimate Std. Error t-value P -valueschoolno319 -0.7724 2.7046 -0.29 0.7752schoolno322 3.4180 2.8717 1.19 0.2340schoolno323 -0.8214 2.7672 -0.30 0.7666schoolno324 6.5210 2.9911 2.18 0.0293schoolno325 -2.3735 2.5545 -0.93 0.3529schoolno326 -0.7416 2.5484 -0.29 0.7711schoolno330 -0.7797 2.6508 -0.29 0.7687schoolno333 1.2772 5.1932 0.25 0.8057schoolno334 0.5099 2.4767 0.21 0.8369schoolno335 1.4205 2.8746 0.49 0.6212schoolno336 -1.0440 2.7784 -0.38 0.7071schoolno337 3.2699 2.3945 1.37 0.1722schoolno338 -0.5275 3.9514 -0.13 0.8938schoolno339 4.4279 3.9513 1.12 0.2625schoolno343 1.0283 2.5917 0.40 0.6916schoolno344 2.6537 2.5873 1.03 0.3051schoolno350 3.6331 2.3257 1.56 0.1184schoolno351 -0.0157 2.4789 -0.01 0.9950schoolno353 1.8000 2.3266 0.77 0.4392schoolno354 3.1660 2.6401 1.20 0.2305schoolno355 -2.5254 2.4740 -1.02 0.3074schoolno356 1.8654 2.5896 0.72 0.4713schoolno357 2.1963 2.5525 0.86 0.3896schoolno358 0.5802 2.3163 0.25 0.8022schoolno359 0.7079 2.2723 0.31 0.7554schoolno361 -0.5219 2.7725 -0.19 0.8507schoolno363 3.7967 5.1777 0.73 0.4634schoolno364 -5.2662 5.2125 -1.01 0.3124schoolno365 -0.8284 2.3531 -0.35 0.7248schoolno366 -0.6231 2.5229 -0.25 0.8049schoolno369 5.5249 2.7754 1.99 0.0466schoolno371 4.1493 2.9882 1.39 0.1651schoolno373 3.3298 2.3808 1.40 0.1620schoolno374 0.9295 2.3059 0.40 0.6869schoolno376 1.3170 2.5555 0.52 0.6063schoolno377 0.9459 2.4059 0.39 0.6942schoolno378 -3.1593 2.7022 -1.17 0.2424schoolno380 -2.0147 2.4383 -0.83 0.4087schoolno381 -4.0718 2.5180 -1.62 0.1060schoolno382 -2.8964 3.1733 -0.91 0.3614schoolno385 2.5849 2.2933 1.13 0.2598schoolno386 -0.6000 2.8133 -0.21 0.8311

Continued


Estimate Std. Error t-value P -valueschoolno387 2.0395 2.4212 0.84 0.3997schoolno388 -1.2362 2.3246 -0.53 0.5949schoolno389 1.3567 2.7101 0.50 0.6167schoolno390 1.0721 2.5202 0.43 0.6706schoolno391 0.1244 2.4700 0.05 0.9598schoolno392 1.3712 2.6414 0.52 0.6037schoolno393 -0.2929 2.3794 -0.12 0.9021schoolno395 -0.3901 2.4434 -0.16 0.8732schoolno396 1.3171 2.6400 0.50 0.6179schoolno399 3.0204 3.4577 0.87 0.3824schoolno400 -0.0092 2.3482 -0.00 0.9969schoolno401 1.7118 2.3497 0.73 0.4663schoolno403 11.1963 5.1872 2.16 0.0310schoolno405 1.8706 2.3773 0.79 0.4314schoolno406 -1.6182 2.9883 -0.54 0.5882schoolno407 2.8655 2.4994 1.15 0.2517schoolno410 1.9125 2.8792 0.66 0.5066schoolno413 4.4833 2.4644 1.82 0.0690schoolno414 -3.0873 2.7708 -1.11 0.2653schoolno415 2.9989 2.6346 1.14 0.2551schoolno416 1.4984 2.4439 0.61 0.5398schoolno417 2.0393 2.5890 0.79 0.4309schoolno418 1.8476 2.3114 0.80 0.4242schoolno419 2.1880 2.3452 0.93 0.3509schoolno420 0.6673 2.4957 0.27 0.7892schoolno421 -1.6063 2.9905 -0.54 0.5912schoolno422 3.9975 3.1802 1.26 0.2088schoolno423 0.4103 2.5295 0.16 0.8711schoolno425 -1.3802 2.4729 -0.56 0.5768schoolno426 -2.5182 2.5257 -1.00 0.3188schoolno427 1.9077 2.6571 0.72 0.4728schoolno428 -1.2431 2.3091 -0.54 0.5904schoolno430 2.0290 2.4648 0.82 0.4105schoolno432 0.5115 2.3942 0.21 0.8309schoolno435 0.6297 2.3330 0.27 0.7873schoolno439 -3.9880 3.0016 -1.33 0.1841schoolno441 3.7547 2.3543 1.59 0.1108schoolno444 -0.6981 2.2630 -0.31 0.7578schoolno445 3.3367 2.8725 1.16 0.2455schoolno446 -1.7607 3.9803 -0.44 0.6583schoolno447 3.8297 2.2866 1.67 0.0941schoolno448 1.3689 3.9607 0.35 0.7297

Continued


Estimate Std. Error t-value P -valueschoolno450 0.2587 3.9688 0.07 0.9480schoolno451 2.5859 2.8803 0.90 0.3694schoolno453 0.5193 2.3181 0.22 0.8227schoolno455 -0.9567 2.4078 -0.40 0.6911schoolno456 3.0562 2.9868 1.02 0.3063schoolno457 1.4699 2.3020 0.64 0.5232schoolno458 1.7954 2.5159 0.71 0.4755schoolno459 3.2305 3.9736 0.81 0.4163schoolno460 -1.2273 2.2630 -0.54 0.5876schoolno461 2.7391 2.9901 0.92 0.3597schoolno465 4.5362 3.1771 1.43 0.1534schoolno466 -1.2428 2.4195 -0.51 0.6075schoolno467 6.2157 3.4610 1.80 0.0726schoolno468 2.1862 2.2339 0.98 0.3278schoolno469 -0.7715 2.6948 -0.29 0.7747schoolno470 1.5107 2.5872 0.58 0.5593schoolno471 1.2497 2.4465 0.51 0.6095schoolno472 5.0792 2.3568 2.16 0.0312schoolno473 4.2033 3.9582 1.06 0.2883schoolno474 1.3590 2.4452 0.56 0.5784schoolno475 -1.9910 2.6445 -0.75 0.4516schoolno476 -0.2802 2.9872 -0.09 0.9253schoolno477 -0.5544 2.3725 -0.23 0.8153schoolno478 0.2334 2.4042 0.10 0.9227schoolno479 0.6391 2.3653 0.27 0.7870schoolno480 1.7883 2.3432 0.76 0.4454schoolno482 -3.3103 2.5145 -1.32 0.1881schoolno483 2.0194 2.3899 0.84 0.3982schoolno484 -2.2105 2.2557 -0.98 0.3272schoolno485 0.1771 2.3019 0.08 0.9387schoolno486 2.1944 2.2211 0.99 0.3232schoolno489 2.6977 2.3603 1.14 0.2531schoolno490 1.7925 2.5306 0.71 0.4788schoolno491 2.0123 2.3416 0.86 0.3902schoolno492 2.1482 2.4500 0.88 0.3807schoolno493 3.8626 2.4530 1.57 0.1154schoolno494 0.8286 2.4065 0.34 0.7306schoolno495 1.9613 2.2566 0.87 0.3848schoolno496 1.1156 2.9965 0.37 0.7097schoolno497 2.9363 3.4626 0.85 0.3965schoolno500 1.5351 3.1809 0.48 0.6294schoolno502 1.8110 2.4928 0.73 0.4676

Continued


Estimate Std. Error t-value P -valueschoolno503 1.9001 5.1764 0.37 0.7136schoolno504 0.0095 3.4646 0.00 0.9978schoolno505 2.0652 2.9932 0.69 0.4903schoolno508 2.4774 2.3541 1.05 0.2927schoolno509 0.1011 2.4388 0.04 0.9669schoolno510 2.1893 2.7677 0.79 0.4290schoolno512 0.3353 2.2485 0.15 0.8815schoolno515 1.6483 2.5526 0.65 0.5185schoolno516 -0.3873 2.4647 -0.16 0.8752schoolno517 -0.1081 2.2958 -0.05 0.9625schoolno519 -0.1811 2.3597 -0.08 0.9388schoolno521 0.1620 2.3868 0.07 0.9459schoolno522 1.2776 2.2444 0.57 0.5692schoolno523 0.8915 2.9871 0.30 0.7654schoolno524 1.0831 2.3433 0.46 0.6440schoolno525 3.7363 3.1716 1.18 0.2389schoolno526 3.4716 2.2250 1.56 0.1188schoolno527 2.8924 2.2925 1.26 0.2071schoolno528 1.8903 2.5143 0.75 0.4522schoolno529 1.8843 2.4664 0.76 0.4449schoolno531 1.7228 2.2890 0.75 0.4517schoolno533 2.7973 2.2838 1.22 0.2207schoolno534 -2.7546 3.1829 -0.87 0.3868schoolno536 -1.3990 3.1725 -0.44 0.6592schoolno538 1.7310 2.7643 0.63 0.5312schoolno539 -0.0128 2.3321 -0.01 0.9956schoolno540 4.2372 2.9908 1.42 0.1566schoolno542 1.9247 2.2912 0.84 0.4009schoolno543 2.8559 2.3706 1.20 0.2284schoolno545 4.2953 2.3414 1.83 0.0667schoolno546 3.1228 2.2892 1.36 0.1726schoolno547 3.4903 2.2415 1.56 0.1195schoolno548 1.8169 2.2229 0.82 0.4138schoolno549 2.7925 5.2275 0.53 0.5932schoolno550 1.7735 3.0053 0.59 0.5551schoolno551 0.9170 3.1687 0.29 0.7723schoolno552 1.8953 2.2626 0.84 0.4023schoolno553 1.8434 2.5320 0.73 0.4666schoolno555 -0.6048 2.3312 -0.26 0.7953schoolno556 6.7524 2.2922 2.95 0.0032schoolno557 -0.6380 2.6431 -0.24 0.8093schoolno558 3.9261 2.2595 1.74 0.0824

Continued


Estimate Std. Error t-value P -valueschoolno559 1.2927 2.5494 0.51 0.6121schoolno561 -0.7649 2.4159 -0.32 0.7515schoolno564 -1.0407 2.7765 -0.37 0.7078schoolno566 3.9125 2.3059 1.70 0.0898schoolno567 0.9013 2.2534 0.40 0.6892schoolno569 0.7463 2.5151 0.30 0.7667schoolno571 2.0048 2.2692 0.88 0.3770schoolno573 -0.1241 2.4639 -0.05 0.9598schoolno574 2.6848 2.2846 1.18 0.2400schoolno575 -0.8015 3.1700 -0.25 0.8004schoolno576 2.1865 2.2723 0.96 0.3360schoolno578 -1.6135 2.3536 -0.69 0.4930schoolno581 1.1152 2.3111 0.48 0.6294schoolno584 0.1694 2.3860 0.07 0.9434schoolno593 3.4802 2.4229 1.44 0.1510schoolno595 -1.2036 2.5559 -0.47 0.6377schoolno596 2.2955 2.4324 0.94 0.3454schoolno597 -1.3452 3.4554 -0.39 0.6971schoolno599 3.3456 3.4953 0.96 0.3385schoolno600 0.2587 2.5362 0.10 0.9188schoolno608 3.8193 2.5669 1.49 0.1369schoolno614 0.9885 2.8780 0.34 0.7313schoolno624 0.9655 2.6356 0.37 0.7142schoolno639 -0.0909 2.5622 -0.04 0.9717schoolno649 2.0530 2.2254 0.92 0.3563schoolno650 -1.6855 2.2292 -0.76 0.4496proc2001 1.8125 0.2553 7.10 0.0000proc2002 -0.1265 0.2926 -0.43 0.6656atsi1 0.5884 1.6028 0.37 0.7136atsiInconsistent -0.2467 0.9642 -0.26 0.7981lbote1 0.0167 0.6735 0.02 0.9802lboteInconsistent -0.4385 0.3045 -1.44 0.1499genderM 0.3420 0.1600 2.14 0.0326aboriginalY -2.5945 1.4686 -1.77 0.0774disabilityY -1.9197 0.3893 -4.93 0.0000school_carY -0.8841 0.6374 -1.39 0.1655occupation1 0.0024 0.5094 0.00 0.9963occupation2 0.1996 0.4318 0.46 0.6438occupation3 0.0196 0.4223 0.05 0.9630occupation4 -0.2211 0.4194 -0.53 0.5980occupation8 -0.2960 0.4304 -0.69 0.4916school_edu1 0.5570 0.6535 0.85 0.3940

Continued


Estimate Std. Error t-value P -valueschool_edu2 -0.0149 0.5843 -0.03 0.9796school_edu3 0.3174 0.5594 0.57 0.5705school_edu4 0.5122 0.5645 0.91 0.3643non_school5 0.3458 0.4822 0.72 0.4733non_school6 0.4142 0.5253 0.79 0.4304non_school7 0.9254 0.5568 1.66 0.0966non_school8 0.0999 0.4647 0.22 0.8297p_g_genderM -0.2987 0.2373 -1.26 0.2082p_g_nesbY -0.1481 0.4038 -0.37 0.7138home_languN -0.5579 0.4374 -1.28 0.2023

Bibliography

[1] Australian Curriculum Assessment & Reporting Authority (ACARA).

http://www.acara.edu.au/default.asp.

[2] Hirotugu Akaike. Information theory and an extension of the maximum likeli-

hood principle. Second International Symposium on Information Theory, pages

267�281, 1973.

[3] Sivakumar Alagumalai, David D. Curtis, and Njora Hungi, editors. Applied

Rasch Measurement: A Book of Exemplars. Springer, Netherlands, 2005.

[4] Douglas Bates. P-values and all that Lmer. https://stat.ethz.ch/pipermail/r-

help/2006-May/094765.html, 2006.

[5] Stan: A C++ Library for Probability and Sampling. http://mc-stan.org/, 2012.

[6] Sheila M. Bird, David Cox, Vern T. Farewell, Harvey Goldstein, Tim Holt, and

Peter C. Smith. Performance indicators: good, bad, and ugly. Journal of the

Royal, 168(1):1�27, January 2005.

[7] Trever G. Bond and Christine M. Fox. Applying the Rasch Model. Lawrence

Erlbaum Associates Publishers, New Jersey, 2001.

[8] Geo� Brindley. Outcomes-based assessment in practice: some examples and

emerging insights. Language Testing, 18(4):393�407, October 2001.

[9] Vincent Calcagno. Glmulti: Model selection and multimodel inference made

easy, http://cran.r-project.org/package=glmulti, 2012.

267

268 Bibliography

[10] Vincent Calcagno and Claire Mazancourt. Glmulti : An R Package for Easy

Automated Model Selection with ( Generalized ) Linear Models. Journal of

Statistical Software, 34(12), 2010.

[11] George Casella and Roger L. Berger. Statistical Inference. Duxbury Press,

2002.

[12] Karl Bang Christensen. Latent Covariates in Generalized Linear Models: A

Rasch Model Approach. In Advances in Statistical Methods for the Health Sci-

ences2, pages 95�108. Birkhauser Boston, 2007.

[13] Trevor Cobbold. Big Increase in Students Withdrawn from NA-

PLAN Tests, http://www.saveourschools.com.au/league-tables/big-increase-

in-students-withdrawn-from-naplan-tests, 2012.

[14] Trevor Cobbold. Many Schools Have High Withdrawal Rates from NA-

PLAN, http://www.saveourschools.com.au/league-tables/many-schools-have-

high-withdrawal-rates-from-naplan, 2012.

[15] Bert P. M. Creemers and Gerry J. Reezigt. School E�ectiveness and School

Improvement: Sustaining Links. School E�ectiveness and School Improvement,

8(4):396�429, 1997.

[16] Igusti Darmawan and John P Keeves. Accountability of teachers and schools:

A value-added approach. International Education Journal, 7(2):174�188, 2006.

[17] Andrew Gelman, John B. Carlin, Hal S. Stern, and Donald B. Rubin. Bayesian

Data Analysis. Chapman and Hall/ CRC Texts in Statistical Science, second

edition, 2004.

[18] Andrew Gelman and Jennifer Hill. Data Analysis Using Regression and Multi-

level/Hierarchical Models. Cambridge University Press, 2006.

[19] Andrew Gelman and Donald B. Rubin. Inference from iterative simulation

using multiple sequences. Statistical science, 7(4):457�472, 1992.

[20] Harvey Goldstein. League tables and their limitations: statistical issues in com-

parisons of institutional performance. Journal of the Royal Statistical Society.

Series A (Statistics in Society), 159(3):385�443, 1996.

Bibliography 269

[21] Harvey Goldstein. Using examination results as indicators of school and college

performance. Journal of the Royal Statistical Society. Series A, 159(1):149�163,

1996.

[22] Harvey Goldstein. Methods in school e�ectiveness research. School e�ectiveness

and school improvement, 8(4):369�395, 1997.

[23] Harvey Goldstein and George Leckie. School league tables: what can they really

tell us? Signi�cance, 5(2):67�69, June 2008.

[24] David A. Harville. Extension of the Gauss-Markov Theorem to Include the

Estimation of Random E�ects. The Annals of Statistics, 4(2):384�395, 1976.

[25] David A. Harville. Maximum Likelihood Approaches to Variance Component

Estimation and to Related Problems. Journal of the Americal Statistical Asso-

ciation, 72(358):320�338, 1977.

[26] Matthew D. Ho�man and Andrew Gelman. The No-U-Turn Sampler : Adap-

tively Setting Path Lengths in Hamiltonian Monte Carlo (pre-print). (2011):1�

30.

[27] Njora Hungi. Measuring School E�ects Across Grades. Shannon Research Press,

South Australia, 2003.

[28] Ben Jenson. Measuring What Matters: Student Progress. Grattan Institute,

January 2010.

[29] James G. Ladwig. What NAPLAN Doesn't Address (But Could and Should).

Professional Voice : The NAPLAN Debate, 8(1):35�40, May 2010.

[30] Nan M. Laird and James H. Ware. Random-E�ects Models for Longitudinal

Data. Biometrics, 38(4):963�974, December 1982.

[31] Christopher Moore. Linear mixed-e�ects regression P-

values in R: A likelihood ratio test Function.

http://blog.lib.umn.edu/moor0554/canoemoore/2010/09/lmer_p-

values_lrt.html, 2010.

270 Bibliography

[32] Douglas Bates. lme vs Lmer. https://stat.ethz.ch/pipermail/r-sig-mixed-

models/2009q3/002912.html, 2009.

[33] Geo� N. Masters. A Rasch Model for Partial Credit Scoring. Psychometrika,

47(2):149�174, February 1982.

[34] Robert H. Meyer. Value-Added Indicators of School Performance: A Primer.

Economics of Education Review, 16(3):283�301, 1997.

[35] National Assessment Program (NAP). http://www.nap.edu.au/.

[36] OpenBUGS. http://www.openbugs.info/w/.

[37] Jose C. Pinheiro and Douglas M. Bates. Mixed-E�ects Models in S and S-plus.

Springer, 2000.

[38] Stephen W. Raudenbush. What are value-added models estimating and what

does this imply for statistical practice? Journal of Educational and Behavioral

Statistics, 29(1):121 �129, 2004.

[39] Stephen W. Raudenbush and J. Douglas Willms. The Estimation of School

E�ects. Journal of Educational and Behavioral Statistics, 20(4):307�335, 1995.

[40] Brian D. Ripley. Model Choice. Technical report.

[41] My School. http://www.myschool.edu.au/.

[42] James Sick. Rasch Analysis Software Programs,

http://jalt.org/test/PDF/Sick4.pdf. Shiken: JALT Testing & Evaluation

SIG Newsletter, 13(November):13�16, 2009.

[43] Tom Snijders. Analysis of longitudinal data using the hierarchical linear model.

Quality and Quantity, 30(4):405�426, November 1996.

[44] John D. Storey. A direct approach to false discovery rates. Journal of the Royal

Statistical Society: Series B (Statistical Methodology), 64(3):479�498, August

2002.

[45] R Core Team. R: A Language and Environment for Statistical Computing,

2012.

Bibliography 271

[46] William Venables and Brian Ripley. Modern Applied Statistics with S. Springer,

2002.

[47] J. Douglas Willms and Stephen W. Raudenbush. A Longitudinal Hierarchical

Linear Model for Estimating School E�ects and Their Stability. Journal of

Educational Measurement, 26(3):209�232, 1989.

[48] M. Wu and R. Adams. Applying the Rasch Model to Psycho-social Measure-

ment: A practical approach. Educational Measurement Solutions, Melbourne,

2007.

mathematical modelling and statistical analysis of school ...€¦ · mathematical modelling and...

Documents