Achievement in Relation to Background Factors

Background Data GatheringDuring every cycle of assessment, background data are gathered by questionnaire and analysis of fieldworkers reports.

Questionnaires are given to students (including a section for parents), head teachers and teachers.

Conditions, facilities etc. in schools are observed and recorded by test administrators.

Background data are correlated with learning achievement to identify needs for improvement and factors associated with achievement.

Background Data Gathering FrameworkBackground data are classified into five areas following the Craig-Heneveld Framework:

The StudentsSupporting Inputs from Outside the SchoolTeaching-Learning ProcessEnabling Conditions School Climate

Results are presented in the next section

The StudentsData are gathered from students about:

Socio-economic background (e.g., family size, fathers/mothers education and occupation)

Factors in the home environment related to achievement (e.g., help w/ homework, space for reading, need to work outside)

Attitudes towards education

For Example:Do you work outside the home to earn money? Conclusion: about 23% of Grade 4 students reported having to work outside the home. Their achievement is significantly lower than others.

Variable %Scaled ScoresMathsUrduScienceSocial StudiesYes23 No77383403370389455473481503DifferenceSignificantSignificantSignificantSignificant

Supporting Inputs from OutsideInformation is gathered from different sources about:

Parent and community support (e.g., frequent school-parent meetings, financial and other support, parental involvement in homework)

Effective support from the education system (e.g., supervision, availability of teaching materials and supplementary materials)

Adequate material support (e.g., facilities, furniture in school, school budget for supplies)

For Example (1):How many of the following six activities in the school does the head-teacher invite parents to participate in?

Attend special events (e.g., functions, sports events)Raise funds for the schoolParticipate in development planningVolunteer for school projects, programs, and tripsEnsure that their child completes his/her homeworkServe on school committees (e.g. personnel, finance)

For Example (2):Number of activities:Conclusion: most head teachers reported inviting parents to participate in a significant number of school-related activities. This participation is positively associated with student achievement.

Teaching - Learning ProcessInformation is gathered from different sources about:

Learning time (no. of teaching days, length of period, time on task)

Variety of teaching strategies used in class (e.g., questioning techniques, investigation, practical work)

Homework (amount given, how it is assessed and what feedback is given)

Student assessment and what feedback on their work students receive

For Example:How often does your teacher check your Maths homework? Conclusion: the great majority (95%) of students reported that they are taught by teachers who check their mathematics homework at least twice weekly. These students achieve significantly better than students who report their homework is checked less frequently.

Enabling ConditionsInformation is gathered from different sources about:

Leadership (e.g., whether the head-teacher is primarily an administrative or an instructional leader in the school)

Capable teaching force (e.g., teacher education, qualifications, experience, style)

Flexibility and autonomy (e.g., decisions and discretion of head teacher, teacher)

High level of time in school (starting/finishing time of periods, presence/absence of teachers)

For Example (1):How many of the following six indicators of academic leadership does the head teacher have explicit policy for? Corporal punishmentHomeworkRecording and reporting student progressExplaining students mistakesDealing with weak studentsParental involvement

For Example (2):Conclusion: about two-thirds (63%) of all students are in schools with no explicit policy for academic matters (i.e., most head teachers see themselves as administrative rather than instructional leaders). Policy for academic matters is associated with student achievement.

School ClimateInformation is gathered from different sources about:High expectations for students (school expects and rewards success)

Teacher Attitudes (attitudes to students and school goals, perception of school climate)

Order and Discipline (e.g., punishment policy and methods, feeling safe at school)

Organized Curriculum available to all

Classroom Climate (e.g., student participation, teacher-student interaction)

For Example (1):Students Perception of School Climate

I like being in school

I think that students in my school try to do their best

I think that teachers in my school care about the students

I think that teachers in my school want students to do their best

For Example (2):Conclusion: an overwhelming majority (92%) of students responded yes to all four statements. This was associated with higher achievement in both language and mathematics

Use of Blackboard and AchievementTeachers use of black board significantly increased students achievement in all the four subjects Ninety-seven percent teachers reported they daily use black boards

Use of Textbooks and Additional ResourcesMajority of the subject teachers used textbooks as major source of knowledge.

Around 10% teachers reported consulting additional resources for teaching

Science teachers were more likely to consult additional resources than other subject teachersBack to Slide 15

Multi-grade Teaching and AchievementLower frequency of Multi-grade Teaching is linked with increased performance for both students and teachersTwenty- one percent teachers reported they have to teach more than on classes in one period

Rewards and Punishment in SchoolStudent who Never got punished by their teachers performed significantly better Fifty-eight percent students reported they never got punished. Only .03 % students reported frequent punishmentRewarding students performance increased achievement all the subjects

PTA/ SMC in School: Head Teachers Report Around 80% head teachers reported presence of PTA/ SMCs in SchoolPresence, need, funding, or role of PTA/SMC did not increase students performance in any of the NEAS tested schoolOnly 39% head teachers reported getting funds from Govt.Ninety-seven percent head teachers reported need for SMCs in Schools

Teaching Resources and AchievementOnly 46% Head teachers reported they have the curriculum document in their schoolOnly 37% teachers reported using the curriculum document; use of curriculum document and other teaching resources did not significantly increase students achievement

Teachers General Education in the NEAS SampleMale teachers were better qualified than female teachers in both rural and urban areasThere were more matriculate female than male teachers In the NEAS selected schools there were no teachers with higher than masters qualification

Head Teachers and Teachers Professional EducationAbout 57% teachers were PTC qualified

There were more M. Ed head teachers than teachers

There were less than 1% B. Ed Teachers and head teachers

Teachers Professional TrainingLess than 50% subject teachers secured any kind of training

Subject curriculum and problem solving were highest reported areas of training obtained

Teachers Scaled ScoresTeachers scaled scores were significantly higher than students in all four subjects Teachers scores were well above the set mean of 500 (SD 100)

Assessment of Students Performance in School

Parents Educational Profiles in the NEAS SampleUrban fathers were relatively more educated than rural fathers

Only .02% urban fathers were graduates

Seventy-two percent rural and 53 % urban mothers were illiterate

There were no graduate or postgraduate mothers in NEAS sample

Fathers Education and Student AchievementThe table shows a trend that children of fathers with post-primary education performed better on different subjects

Fathers Occupation in the NEAS SampleMajority of rural fathers were agriculture landowners

Majority of urban fathers were small business owners and skilled workers

Fifteen percent urban and twelve percent rural fathers in govt. jobs

Fathers Occupation and Students AchievementFathers occupation did not significantly affect students achievement on Maths and Science testsChildren of Agricultural wage earner however, scored significantly lower as compared to govt. servants on language test Children of private employed parents scored significantly lower on social studies testAll other mean differences were non-significant

Strengthening Links between NEAS and the Educational Planning ProcessNEAS can support educational planning by:providing data about the actual conditions in schools; andindicating the possible impact of policy actions on student achievement.

Educational planners can make NEAS more effective by:proposing topics to be added to background data gathering; andsuggesting relationships to be explored.

Conclusion and Next Steps

The presentation in summary:NEAS has assessed the achievement of Class IV students in 4 subjects: Urdu, Mathematics, Science and Social studies.

The assessment reveals large differences in average performance among provinces and between students from differing backgrounds.

These differences in average performance can be traced back to differences in student and family characteristics, school processes and inputs.

As illustrated in the next slide the factors that influence performance can be divided into four types:

Additional work:The lack of performance standards limits the interpretability of the assessment results.

Additional analysis is needed to identify causal relationships and hence where policy might yield improvement.

The next slides illustrate how multi-level multivariate analyses can be used to reflect on the impact of different policy interventions.

Expected changes in achievement over time:All other things being equal one would not expect to see large changes in achievement levels from year to year. Evidence suggests that education reforms take at least 5 to 10 years to bear fruit.

For Pakistan one would actually expect test scores to fall over time as rising enrollment rates draw more children from relatively disadvantaged backgrounds into the system.

Future Vision of National Assessment in Pakistan

Proposed Recommendations by National Select Committee:

NEAS future planning should be till 2020 at Grade 4, 5, 8, 10 and 12 levels

Private schools should be included in NEAS sample on Pilot basis in 2008 National Assessment

Listening and speaking competencies (skills) of student assessment would be part of NEAS future activities

Pakistan can explore the possibility to participate in the next TIMSS in 2012

Thank You