Statistical Analysis of 2005-2007 KS4 Results

The use of data to inform planning and management is acknowledged as a strength at Bettws High School -- recognised in our recent Estyn inspection and as part of the School Improvement Plan. Reference is made to collated data to inform planning and to scrutinise current performance. Received wisdom from Estyn, WAG and within the school is that “Boys perform less well than Girls” in both KS3 and KS4 - with this differential increasing as you progress through the school. Indeed school data substantiates this with end of KS4 average points of 24.6 and 23.5 for Female / Male respectively. Consideration in setting and whole school initiatives have been implemented to ad-dress this performance differential. Published and whole school data exemplifying this differential is based on the mean value of the cohort. No consideration has been given to the spread, range or deviations of the data set. Using statistical analysis of variance it is possible to determine if the perceived difference in means is a true difference or merely a consequence of the spread/range of the data. To assess if the sex linked differential is a sound conclusion, the terminal data for the 2005-2007 cohort, was assessed statistically. The following baseline and achievement data was included in the study: Student attendance, reading age, KS2 English, Maths and Science results, KS3 Eng-lish, Maths and Science results, CATs scores, Teacher attendance (where possible) and Sex. Figure 1 shows the data set for Average Point score segmented by Sex. The chart shows the spread of the data and the shaded box the interquartile range. The two data sets being linked by a line joining their means. What can be clearly seen from this chart, is that whilst the means are dif-ferent, the data sets completely overlap. Analysis of variance supports this with no statistical differ-ence being reported between Male and Female. Consequentially some other factor must be re-sponsible for the spread of data. Analysis of the factors listed demonstrates that the most significant factor in determining the 2005-2007 average points score is KS3 English results. As can be seen from Figure 2, whilst there is still spread of the data, the interquartile ranges for the level 3/4, 5 and 6/7 show clear differences. Indeed the lower quartile of the level 6/7 does not overlap the upper quartile of the 3/4. This differ-ence is significant to 0.001 - indicating that 99.9% of the variation in the data can be explained by using the KS3 English results. Other factors determined as significant are: KS3 Science results, student attendance and teacher attendance. Further, where the data exists (Science) Teacher At-tendance is the single most significant factor in determining performance at KS4 (Figure 3). Interestingly, CATs, KS2 results and Reading Age were found to be not significant in determining average points score. The potential implications of Figures 1-2 and the subsequent analysis could be profound. For ex-ample, money and effort spent on initiatives into addressing the sex differential could be better spent on raising the performance of those students achieving Levels 3-4 at KS3 English. Figure 3, provides tantalising evidence that the most effective way to increase Science performance would be to investigate and increase staff attendance. Of course, with all data obtained from real sources, there can be latent effects that it is not possible to analyse for. For example, it could transpire that the most effective teachers have the highest at-tendance, thus are we analysing teacher attendance or performance? A broader, longitudinal study, covering other school would be needed to determine if these effects are “general” or specific to the school / cohort in question.

Sex

Sco

re

MaleFemale

50

40

30

20

10

0

Average Point Score on Leaving

Figure 1. Average points score on leaving segmented by sex of student

Recommendations / Further Study

To substantiate the effects seen in this analysis, two additional studies are required. In the first instance, corroboration by analysing for the same factors across several schools. Ideally as many schools as practical would make the analysis more statistically significant. Sensitivities need to be considered as it would be possible to analyse the data to show “which school performed better”. For this reason, it is envisaged that LEA support will be required to obtain the necessary data and to fully explain the reason for the study. Data required A data package for the 2005-2007 cohort, consisting of; student attendance, reading age, KS2 English, Maths and Science results, KS3 English, Maths and Science results, CATs scores and sex. The impracticality of cross referencing teacher against students will make the assessment of teacher attendance impossible. Secondly, a more long term study at Bettws High is necessary to assess both local trends and the effects of any proactive/remedial actions. Funding Practically speaking, the data analysis is not difficult - it is the preparation, cleaning and entry that is the most time-consuming. Funding from sources such as LEA, GTCW and NFER is being inves-tigated to allow this study to continue and expand.

Year 11 Teacher Attendance

Sco

re

95%+90-95%-90%

55

50

45

40

35

30

25

20

15

Final Science Points Score

Figure 3. Science points score on leaving, segmented by Teacher Attendance

Figure 2. Average points score on leaving segmented by KS3 English Results

KS3 English Levels

Sco

re

6/753/4

50

40

30

20

10

0

Average Point Score on Leaving