world vision solomon islands makira early childhood ...the world vision ece program is located in...

World Vision Solomon Islands

Makira Early Childhood Development Program

Impact Report

November 2012

for

World Vision Australia

Dr Linda Kelly

Dr Libby-Lee

Hammond

Dr Andrew McConney

Praxis Consultants

Makira Early Childhood Development Program Impact report

November 2012

2

Executive Summary

The Solomon Island Makira Early Childhood Development Program operated from

2005 to 2011 through a partnership between World Vision Solomon Islands

(WVSI), and Makira Provincial Government. Working through two phases the

program focused on improving literacy and primary school attendance while also

including a more holistic approach to child development. Previous research has

identified positive outcomes for the program. This study sought to identify the

program impact.

A quasi-experimental approach was taken to investigation of impact, looking to

compare the performance of children from Makira province who had graduated

to school, with the performance of children from a comparable province. This was

complemented by additional qualitative research.

Overall the findings provide statistical verification of higher competencies in

numeracy and literacy for children who attended the early childhood education

program compared to those children who had not had the opportunity to attend

the program. Attendance rates were also significantly higher.

The research was limited by sample size and other factors but provides good

quality indicative results of the value of the Solomon Island Makira Early

Childhood Development Program in improvement in educational results for

children in remote areas of the Solomon Islands.


November 2012

3

Introduction

The Solomon Island Makira Early Childhood Development Program operated from 2005 to

2011 through a partnership between World Vision Solomon Islands (WVSI), Makira Provincial

Government and the Solomon Islands Ministry of Education and Human Resource

Development (MEHRD). The program operated over two phases and aimed to provide early

childhood education (ECE) to 2000 children in target communities in Makira province. This

paper reports on an impact assessment of the program outcomes undertaken 12 months

since program completion.

Project overview

The early childhood education program started in Makira (the largest island in the province),

working with some of the most remote communities in the east of the island. Originally titled

the ‘Girl Child Reading and Rescue Project’, the original intention was to include girls in the

education system alongside boys so that they could both have basic life skills appropriate for

rural life and for integration with the formal school system’.

The first phase of the program focused on improving literacy and primary school attendance

while also including a more holistic approach to child development. The first phase of the

program was implemented from July 2005 until June 2008. The second phase of the program

built upon this first phase with the additional intention of increasing access to higher

education opportunities for selected community members (ECE trainers who had completed

basic training through phase I of the Project) to become qualified early childhood education

teachers. The second phase of the program ran from 2008 to 2011.

The program was undertaken in partnership with communities and with the Provincial

Education Authority. In the second phase of the program a partnership was also developed

with the Solomon Islands College of Higher Education (SICHE). World Vision also worked with

the National government through the Ministry of Education and Human Resource

Development (MEHRD), and with the Makira Provincial Government. Health promotion,

financial planning and cultural promotion activities were also integrated into the project in

some communities with participation from both men and women. The project also promoted

community participation and sought to increase community ownership and social cohesion.

The formal program objectives focused upon improving basic literacy skills for young children

and increasing the number of children moving into primary school. According to program

evaluations, outcomes of the program over six years met these objectives as well as leading to

several additional outcomes.

The program has attracted attention nationally and is now being trialled in other provinces by

the MEHRD.

Previous research

The program has been identified as having many successful outcomes. Independent

evaluations of the program undertaken in September 20091 and again in August 20112

1 Donnelly, J. (2009), ‘Girl Child Reading and Rescue Project Phase Two Review.’ September 2 Chinnery, S. & Orr, C. (2011), ‘Education Projects in the Pacific, World Vision’, final draft report,

August.


November 2012

4

reported that it has contributed towards the mental, physical and language development of

students and has potentially improved the overall literacy rate of young children in the

Province. In contrast to a national net primary school enrolment rate of 56%3, at least 90% of

children graduating from the program have been mainstreamed into primary school in Makira

province.

In addition, other outcomes which were identified in the independent evaluations include

increased capacity of government staff and other stakeholders; broader capacity

development within communities to address issues and work together: health promotion to

children at local level; improvements in gender relations and equality within communities;

and the provision of higher education opportunities for local teachers in the province.

Current research

The various project reports and commentary, including the independent evaluations, point to

several factors that have enabled successful outcomes in this program. In order to provide

further learning about those outcomes and to test the long-term impact of the program,

World Vision commissioned a program review. The review had two major objectives:

1. To investigate the enabling factors that contributed to the success of this program

and also the barriers experienced and the actions taken to overcome these.

2. To measure the impact of the projects on the learning and development of students

who participated, with a view of confirming and describing the positive effect ECE has

on child learning and development, and more generally on family members and

communities .

The first objective was explored through research undertaken in June 20124. The findings

pointed to several enabling factors for the successful outcomes to date, including including

strong local ownership and use of local resources and materials. The Research concluded that

the successful outcomes of the program were due to a good quality programming approach

and good quality program management coupled with an emphasis upon building ownership

and partnerships with all stakeholders at all levels. Effective partnerships with communities,

the provincial government and SICHE have been developed through considerable attention to

communication and to systems of accountability to those stakeholders. The study also found

that the program clearly benefited from having dedicated and appropriately qualified staff

that had been able to establish effective relationships with all varieties of stakeholders.

The commitment and capacity of all partners was critical to the outcomes. That research

provided a picture of the way World Vision and other stakeholders, such as community

members, provincial government staff and personnel from SICHE, worked together to achieve

program outcomes.

3 Reported in 1999 census.

4 This research was undertaken by Praxis Consultants in June 2012. A report on the research

undertaken for Objective one is available from Praxis Consultants, World Vision Solomon Islands or

World Vision Australia.


November 2012

5

Some challenges were identified, in particular how sustainable the program outcomes are

likely to be for families and children and how well the program is able to ensure girls and

women and children with disability are also benefiting from the program outcomes.

The second objective of the study is the focus of this report. Beyond the outcomes of the

program, World Vision, through support from AusAID, was interested to understand what

impact the program has had on children as they move into school in the Solomon Islands. In

particular the purpose of understanding impact is to ascertain the difference for children

having had the benefit of ECE prior to commencing schooling, compared to children in

Solomon Islands who have not had this opportunity.

This report outlines the methodology and approach for the impact study, summarises the

findings from the study and the conclusions which can be drawn about program impact. It

complements the earlier report around program enabling factors and should be read together

with that report.

Methodology

Approach

In order to understand the approach taken to impact assessment for this program it is

necessary to have some perspective on the challenges around education in the Solomon

Islands, in particular the province of Makira-Ulawa , and the expected impact of ECE in

response to these challenges.

According to the 1999 census, 50.1% of children in the Solomon Islands between the ages of 5

to 9 years do not attend school. The total primary enrolment (% net) in the Solomon Islands

was 82% in 20075 and was reportedly at 94% in 20106. However the primary completion rate

has declined from 83% to 76% between 2006 and 2008.7 Other points of concern are literacy

and numeracy rates which still remain low: only one out of every two learners met the literacy

standards for year 6 and more than half are underachievers in numeracy8.

Commonly in the Solomon Islands, children in rural areas do not start school until they are

already eight or nine years of age. The majority of these children have had no exposure to any

early education like preschool. By the age of nine, when children in rural areas are old enough

to walk long distances to school, it appears that those who have not yet learned to read or

write are very likely to remain outside the formal education system.

The Government of Solomon Islands has attempted to address poor literacy levels and school

retention rates in the early years of primary school by developing a culturally relevant

curriculum. As part of the new curriculum and wider redeveloped system, the Ministry of

Education has also developed an early childhood education policy aimed at 3 to 5 year olds.

5 World Bank – Data: http://data.worldbank.org/indicator/SE.PRM.NENR/countries 6 AusAID support to the Solomon Islands Education Sector (2011-12) – Design and Implementation

Document (DSID), September 2010 7 Performance Assessment Framework (PAF) 2006-2008, July 2009. Ministry of Education and Human

Resources Department 8 Solomon Islands – Australia Implementation Strategy Priority Outcome One- Service Delivery,

Education, Nov 2011

http://data.worldbank.org/indicator/SE.PRM.NENR/countries


November 2012

6

This policy requires children to have three years of early childhood education or ‘kindy’,

before enrolling at primary school. While provinces seek to enforce this policy, the

responsibility for implementing it lies with local communities.

The World Vision ECE program is located in the province of Makira-Ulawa . According to local

education officials, before the program commenced the literacy rates of children from

Makira-Ulawa province were one of the lowest in Solomon Islands. Girls performed better

than boys however there are fewer girls at school than boys. For rural communities (85% of

the population) in particular it is difficult for children to access schools. Most people in rural

areas live in remote communities which can be several hours walk or canoe ride from health

and education services.

The impact sought by the program in this situation was to improve the likelihood of children

from this area attending school and also improve their performance at school. There was a

particular emphasis in the program on improving the situation for girls.

The approach taken to assessing impact therefore was quasi-experimental. The impact study

sought to compare outcomes for children from the ECD program with those outcomes for

children who had not attended ECE.

Sampling

A selection of children who have graduated from both phases of the ECE program and who

are now at school were tested for their ability in literacy and numeracy. The children’s results

were compared with a comparable group of children who had not attended ECE. The research

sought to identify if there was a statistically significant difference in children’s competency in

literacy and numeracy. In addition the impact study sought to compare and identify

differences in attendance rates.

Children were tested in schools in Makira province. It was not possible to undertake a random

sample of the children across the whole province due to time and resources. Therefore the

research sought to get a range of data from different locations that would at least represent

major differences in the project area. Therefore schools were chosen in Makira intentionally

from Phase I and Phase II communities. One school was located in the provincial capital and

then others from the more remote rural communities. Testing was undertaken across six

schools in the province. 81 children were tested, 40 girls and 41 boys.

The contrast or control province was chosen on the basis of its similarities with Makira

province. Central province , similar to Makira, has had less investment by the National

Government and consequently both are characterised by low levels of development and poor

availability of basic services. Both provinces are matrilineal societies. In Central province,

similar to Makira, the majority of communities must be reached by boat once you leave the

provincial capital and surrounds. Both provinces have very poor communication systems. The

schools tested in this Province included one close to the Provincial capital and then the others

a significant boat trip away to mirror the Makira schools as much as possible. In this province

63 children were tested from four schools, 34 girls and 29 boys.


November 2012

7

In both provinces children at the school on the day of the testing were randomly selected for

the testing (provided they could identify their birth date). In Makira province children who

were tested and them subsequently identified as not attending ECE (a small number) were

then exclude from the final data. In Central province it was clarified that none of the children

tested had attended the World Vision program.


November 2012

8

As noted above, quantitative data was collected from a total of 144 children, as outlined in

Table 1.

Table 1. Total number of children tested by province and school

Province School Girls Boys

Makira FMC ( Phase 1) 9 3 Toroa (Phase 1) 2 6 Naharahau (Phase 1) 5 4 Namuga (Phase 1) 10 10 Gupuna(Phase 2) 10 10 Aorigi (Phase 2) 4 8 Subtotal 40 41 Central Bonarogu 10 10 Soso 8 3 Vura 10 8 Nautuha 6 8 Subtotal 34 29 Totals

74

70

All children were in class I or class 2 in primary school. Children ranged in age from 6 to 10.

The average age of all children tested in Makira province was 9.59 years. The average age of

children tested in Central province was 8.56 years.

Consent was sought from the schools and the Education Department for the testing of the

children.

In both provinces focus groups were undertaken with teachers and where possible in

communities with families and community leaders.

Data collection

Considerable attention was given to appropriate data collection for this study. Review was

made of existing data on student performance in the Solomon Islands and based upon advice

from education advisors, within AusAID and Solomon Islands, a decision was made not to use

this existing information. Concerns were raised around accuracy and comprehensiveness of

existing data as well as the difficulty of accessing timely and relevant data.

In light of this it was decided to undertake independent testing of children. Early childhood

experts from Murdoch University in Australia9 agreed to work with the study and identified a

series of appropriate and feasible testing instruments which could be applied to give

indicative information around children’s literacy and numeracy competency in grades one and

two of primary school. These tools were chosen on the basis of their long-term usage for such

testing in cross cultural settings (indicating their overall reliability), their ability to be easily

9 Dr Libby-Lee Hammond, Associate Prof, Early Childhood Education, Murdoch University and Dr

Andrew McConney, Senior Lecturer, Assessment, Evaluation and Research Methods, School of Education, Murdoch University.


November 2012

9

and feasibly applied in the Solomon Islands context and the fact that they gave good, although

not complete, information around literacy and numeracy competencies for young children. It

was recognised that the instruments would not cover all elements of literacy and numeracy

competency but the tests chosen were expected to provide good indicative information

around children’s ability. (Annex one provides full details of the tests utilised for this study).

Once the tests were chosen the early childhood education advisers from Murdoch University

supervised piloting of the instruments within the Solomon Islands school context. This pilot

exercise was undertaken over a 10 day period and involved extensive testing and refining of

the instruments within different schools and with different aged children. The piloting was not

sufficient to standardise the test for all Solomon Island conditions but did serve to ensure that

it was able to be utilised as a comparative tool between children in the Solomon Islands. In

other words the test was both feasible and comprehensible within the context, and regularly

and reliably supplied indicative results about children’s literacy and numeracy competencies.

During the piloting period, local researchers were trained in the use of the tests. The

education adviser from Murdoch University supervised them in repeated application of the

tests in different contexts in order to ensure that they were able to reliably administer the

range of instruments. In addition a scoring system was developed to ensure accurate and

timely recording of results. The system was a ‘partnered’ system to ensure that results

recorded were checked by more than one person at the time of entry. In addition to this, the

raw data was subject to further checking by the education adviser following completion of the

whole research exercise.

In addition to the testing, qualitative information was sought from teachers, parents and

community leaders to provide some further verification for any observed differences between

children’s performance. This qualitative information was captured through focus groups. As

above, the focus group questions were piloted across schools and communities before being

utilised for the final study. (The focus group question outlines are attached at Annex Two).

Following the piloting of both testing instruments and focus group questions the full study

was undertaken across a two week period in two provinces of Solomon Islands.

Data analysis

As described above, data from the testing was entered at the end of each day into a scoring

sheet (see Annex Three). This resulted in two spreadsheets representing the data from both

provinces. Also as noted above, this data was entered by two people to check its validity. It

was further checked through review of raw data sheets by the education advisers in Australia.

Alongside the testing data, focus group discussions were conducted by two people, with one

person recording comments from all participants. This information was translated into

English.

Both sets of quantitative and qualitative data were sent to Australia. An independent

evaluation consultant10 examined the quantitative data and subjected it to statistical testing

10

Dr Linda Kelly, Praxis Consultants.


November 2012

10

to analyse the differences between the two provinces overall and between different subsets

of the children across both provinces11.

The independent consultant used the results of the analysis together with the information

from the focus groups to produce this report. Further advice and insight was sought from the

early education advisers from Murdoch University prior to finalisation of the report.

Limitations

The study is focused examination of the different outcomes for children in early years of

education comparing those that have had the opportunity for ECE education through the

World Vision program in partnership with the Makira-Ulawa Provincial Education Authority,

and those who have not. It seeks to address identified challenges in the Solomon Islands

context of children either not attending school or struggling in literacy and numeracy

competencies particularly in the early years of school. It focuses on giving clear and evidence-

based assessment of any impact the Makira ECE program may have had on these two issues.

It was clear from the beginning of the study that given limited resources and time the

research would only seek to give good indicative information about the impact of the Makira

ECD program on children’s educational performance. Given that it was not possible to

undertake a fully random sample of children from Makira province, nor was it possible to

undertake a random sample of a control group with exact matching characteristics to the

treatment group, the study could never be considered to have proven impact in the purely

scientific sense.

In addition to this, the two populations of children were of different sizes and therefore data

could not be compared on paired matching basis which somewhat lessened the strength of

the analysis able to be undertaken (although the choice of the T-test was made to allow for

this).

Finally, and perhaps most importantly, the study is a snapshot in time. While it provides

information about children who were in the early phase of the program as well as children in

the more recent phase it was not able to look beyond grades one and two of primary school

to consider the implications for children in higher grades or as children progressed from

primary to secondary school. Clearly this information about longer term impact would be of

enormous benefit for the program and donors such as AusAID, as well as stakeholders such as

the Education Department in the Solomon Islands. These and other areas remain to be

explored.

While the study is limited in what it can claim to have proven, given these limitations, it does

serve as a useful indicative study of program impact utilising both quantitative and qualitative

data as evidence of that impact.

11

The statistical test utilised was the ‘Welch unpaired T test’. Typically t-tests compare the means of two groups to analyse whether there is statistical difference between these two numbers. The variation of the t-test used in this situation allowed for the fact that the two sets of data contained different size populations with potentially different standard deviations in each population. The Welch T Test allows for this variation.


November 2012

11

Findings

Comparison between the two sets of data indicated that on all scores for numeracy and

literacy, the children from Central province scored, on average, lower that the children from

Makira province. This does not prove program impact by itself. The analysis of data was

undertaken to identify if the difference was significant from a statistical perspective. In other

words was the difference large enough to indicate a different level of performance, or was the

difference in fact due likely to be within a normal range for this data12?

Numeracy results

Children were tested across six elements for indication of their competencies in numeracy. (As

noted details of each of these tests are contained in Annex One.) The tests investigated

children’s abilities in numbers, their understanding of quantity and their ability to apply early

arithmetic strategies. The final numeracy test looked at children’s conceptual ability in simple

multiplication and division strategies. Comparative results for children between the two

provinces are contained in the following table.

Table 2. Results for numeracy tests by province

Test Average for children from Makira province

Average for children from Central province

Statistical comparison of results

Numeral Identification

2.89 2.16 Difference is extremely significant

Forward number sequencing


Backward number sequencing


Subitising 1.59 0.90 Difference is extremely significant.

Early arithmetic strategies

2.63 1.84 Difference is very significant

% who were able to complete this test from Makira province

% who were able to complete this test from Central province

Multiplication and division concepts13

65.85% 39.7%

As the results indicate, children who had attended ECE in Makira province were able to

outperform children who had not attended ECE on all tests to a statistically significant degree.

12 The T-Test applied looked for difference between the groups, identifying if the difference was

extremely statistically significant ( p<0.0101, or there was a 1% probability that the results are due to

chance) , or very statistically significant (p<0.001, or there was a 1% probability that the results are due

to chance). In a small number of cases there was identification of differences which were not quite

statistically significant ( p<0.1, or there was a 10% probability that the results are due to chance).

13 This test was not able to be subject to the same statistical analysis given the results of the test were

that children were either able or unable to complete the text. The raw results however show considerable difference between the two test groups.


November 2012

12

Comparing the children by gender produces similar results, as outlined in Table 3. the only

variation is for the test for early arithmetic strategies, where children are required to

demonstrate their ability to manipulate materials to solve arithmetic tests. Results for this

test indicated there was no statistical difference between girls who had attended ECE in

Makira province and those in Central province who had not attended ECE.


November 2012

13

Table 3. Results for numeracy tests by province and gender

Test Average for girls from Makira province

Average for girls from Central province


Average for boys from Makira province

Average for boys from Central province











Subitising 1.63 0.97 Difference is very significant



2.40 1.85 Difference is not significant


% of girls who were able to complete this test Makira

% of girls who were able to complete this test Central

% of boys who were able to complete this test Makira

% of boys who were able to complete this test Central

Multiplication and division concepts

70% 38.2% 65.85% 41.37%

Finally attention was given to the differences in performance of children who had been

through phase 1 and phase 2 of the ECE program. As the following results indicate there is not

a statistical difference between children’s numeracy ability whether they attended phase 1 or

phase 2 of the program. However it should also be noted that on average children in phase 2

performed slightly better than children from phase 1 of the program. Further testing of this

difference might be of some value.


November 2012

14

Table 4. Results for numeracy tests by program phase

Test Average for children who attended phase 1 of the ECE program in Makira province

Average for children who attended phase 2 of the ECE program in Makira province







3.88 4.34 Difference is not quite significant

Subitising 1.55 1.66 Difference is not significant


2.41 2.97 Difference is not quite significant

% who were able to complete this test who attended phase 1

% who were able to complete this test to attended phase 2 of

Multiplication and division concepts

65.3% 71.87%

Literacy

Children were tested across five areas to give an indication of their competency in literacy.

This included testing for letter identification, word reading, writing vocabulary, hearing and

recording sounds in words and finally reading accuracy. The results were similar to those for

numeracy, with children who had attended ECE in Makira province outperforming children

from the Central province schools who had not attended ECE. Table 5 below provides the

overall findings.

Table 5. Literacy results for children between provinces

Test Average for children from Makira province

Average for children from Central province

Statistical analysis of results

Letter Identification 50.23 36.86 Difference is extremely significant

Word reading 10.26 2.19 Difference is extremely significant

Writing vocabulary 15.86 5.05 Difference is extremely significant

Hearing and recording sounds


Reading accuracy 79% 2% This difference was so extreme a statistical test was not applied


November 2012

15

Looking at the difference across genders (Table 6), it is clear that for literacy both boys and

girls who were part of the ECE program outperformed boys and girls from Central province.

Table 6. Literacy results for girls and boys between provinces

Test Average for girls from Makira province

Average for girls from Central province


Average for boys from Makira province

Average for boys from Central province


Letter Identification



Word reading



Writing vocabulary






Reading accuracy

83% 3% This difference was so extreme a statistical test was not applied

75% 0% This difference was so extreme a statistical test was not applied

Finally attention was given to the difference for children in phase 1 and phase 2 of the ECE

program. As the results in Table 7 show, while children generally performed at a similar level,

there were some differences on one test (hearing and recording sounds) that suggests

children from Phase two were able to perform better on this test.

Table 7. Literacy results for children by program phase

Test Average for children from phase 1

Average for children from phase 2


Letter Identification 49.47 51.41 Difference is not significant

Word reading 9.39 11.59 Difference is not quite significant

Writing vocabulary 16.28 15.25 Difference is not significant



Reading accuracy 65.05 68.39 Difference is not


November 2012

16

significant

Further examination of this difference between the two groups indicates that there is some

difference between the genders for the test around hearing and recording sounds. The scores

for boys in phase 1 and 2 for this test around indicates no statistically significant difference

between the two groups. However, girls from phase two score better than girls from phase

one on this test to a very statistically significant degree. This needs to be interpreted

carefully, because the numbers become very small when comparisons are made between the

phases, however it does suggest some difference in experience between the genders in phase

1 and phase 2 that might be worth further research.

In light of this some further detailed comparison was made around reading accuracy between

girls and boys in phase 1 and 2. In this case no statistical significant difference was found

between girls in phase 1 and 2 or between boys in phase 1 and 2 for reading accuracy.

Attendance

Attention was also given to attendance by children. The results of this section need to be

treated with some caution because attendance was not tested through an independent tool

but was recorded from interviews with teachers. Teachers were asked to identify the

percentage attendance for each child prior to them being tested.

A summary of the overall attendance is provided in the following figure.


November 2012

17

Once again statistical analysis of the figures was supplied to the attendance data. Tables 8 &

9 summarise that analysis.

Table 8. Results for school attendance by gender and province

Average attendance girls in Makira

Average attendance girls in Central


Average attendance boys in Makira

Average attendance in Central


92.4% 83.82% Difference is extremely significant


Table 9. Results for school attendance by gender and phase

Average attendance girls from phase 1 of the

Average attendance girls from phase 2


Average attendance boys from phase 1

Average attendance rate for boys from



90.22% 96.94% Difference is not quite statistically significant

Overall the data shows that statistically significant higher number of girls and boys attend

school if they have attended the ECE program. However one of the interesting variations is

that girls from phase 1 of the program have a statistical significant lower attendance rate than

girls from phase 2. This is not so for boys.

Summary of findings

Overall the findings provide statistical verification of different competencies in numeracy and

literacy between children who attended the ECE program in the Makira province and those

children in Central province who had not had the opportunity to attend an ECE program. This

is in line with the qualitative information gathered from teachers who in their responses

indicated clearly observable difference between children who had attended ECE and those

who had not.

Yes students who have attended ECE have found it very easy to be able to cope with

the level of teaching in class 1 and 2. Those who do not attend ECE still problems even

in coordination in writing and not even been able to recognize number and simple

letters. (Focus group discussion with teachers from Gupuna primary school, Makira,

phase II)

With the current ones ( who have gone through ECE) their capacity to be able to learn

is fast, they grasp concepts faster than those who do not ( speaking from experience),

they are eager to learn as well.(Focus group discussion with teachers from Namuga

primary school Makira, phase I)

Teachers were also able to identify gender differences.


November 2012

18

Girls seem to be very confident as well as they come to class, they can be able to

interact well with others and not too shy, this is because they see themselves capable

to read and write just like any other kids. During those days when there was not ECE,

girls without ECE usually have low esteem and tend to leave school early (focus group

discussion with teachers from Aorigi primary school, Makira, Phase II).

Teachers working in Central province were aware of ECE programs and the difference it might

make for their children.

There is a major difference , I have been teaching in Boromole and they have an ECE

there, I do not have any problem in teaching them, they grasp ideas so quickly and

make sense of what is taught as well. ……Our kids here find it hard to be able to read

and write and even some do not understand how to count as well, this should be done

in the ECE level however class 1 teacher is burdened to teach them how to read and

write and at the same time have for complete the class syllabus as well. (Focus group

discussion with teachers from Bonaroqu primary school)

There are 2 kids in this school, who are kids of teachers here that have gone through

ECE in other schools before coming here……The 2 kids have done exceptionally well

compared to the others. The two kids also show interest in their school compared to

most of the ones who do not attend, those not attending ECE are less motivated to

learn as they know that they will not able to get this correct, they have low self

esteem. (Focus group discussion with teachers and community leaders from Soso

primary school).

In a similar way, and in line with previous research, parents are also able to identify the

differences in their children as a result of the ECE program.

Yes my kids get to be able to read when they are still very small, in the past reading is

a level we reach when we are older ( slower readers, even today some of us do not

even know how to read as yet)………Kids can be able to reason things but at that age

in our time we are not able to reason like that. (Focus group discussion with parents in

Gupuna, Makira, Phase II)

The findings suggest, but do not necessarily prove, a difference in some areas between

children who were in phase 1 and phase 2 of the program. Significantly there may be a

difference in attendance especially for girls from phase 1 of the program compared to girls

who went through phase 2 of the program. Further research may be merited around this area.

Also the results possibly suggest some differences in outcomes between boys and girls who

attended the ECE program. Average results for girls are higher in literacy and lower in

numeracy than boys. However only for one test showed any statistical difference between the

two genders and therefore these results need to be treated with some caution. It may

indicate the need for some more extensive testing around the different experiences of girls

and boys.


November 2012

19

Other findings

In addition to the information gathered for the impact study, other information became

available through the observation of researchers and through the focus group discussions.

A significant observation in a number of the schools was the lack of books. It is certainly not

possible to expect that children will learn to read if there are no books present.

A further significant observation was the methods and tools available for teachers to assess

the literacy and the competencies of their pupils. It appeared in most cases that teachers

struggled to understand and apply available testing tools (supplied by the Ministry of

Education). Without teachers being confident and competent to use such tools, information

collected will neither assist them in improving the teaching nor allow them to provide

accurate information to the Provincial education authority and the National Ministry of

Education. The teachers themselves raised the issues of teacher qualification and teaching

resources. There was consistent feedback from the teachers that without books and

equipment for teaching and adequate training for the teachers, all students would be

disadvantaged within the school system. This was a strong and consistent message from all

the communities.

Finally, from those teachers working in Central province there was a very strong message

about their support for ECE and the need for this to be established in communities prior to

children attending school.

Conclusions

The results from the impact study provide a strong indication that the World Vision Makira

ECE program has had substantial impact upon boys and girls literacy and numeracy

competency in primary school. It also appears that the program has had impact upon the

attendance rates of both boys and girls. The program can therefore be said to have had an

impact in the areas identified as of significant challenge for educational improvement in

Solomon Islands.

It is important to recognise that the results do not identify which aspect of the ECE program

has specifically contributed to this observable impact. As previous research has shown, the

ECE program operated by World Vision in partnership with Makira province, has a number of

enabling factors. These include the attention given to teacher support and training and the

extensive work with parents and communities around engagement in the benefits of

education as well as broader community development activities. The World Vision approach,

in particular the staff employed and their empowering and respectful way of working with

community members, had been very significant in the positive outcomes of the program. The

commitment of Makira-Ulawa Provincial government and other significant stakeholders,

including personnel from SICHE, have all been factors in contributing to program success.

It would be a mistake therefore to conclude simply that ECE programs by themselves will

produce the same impact as observed through this program. Indeed, the findings to date

suggest it is likely that it is the combination of these factors that has made such a clearly

observable difference in the lives of girls and boys in Makira province.


November 2012

20

There are some areas for possible further research. These include the different experience

between girls and boys in the ECE program and the different experience between children

who are in phase 1 and phase 2 of the program. Further research in these areas would be

valuable to better understand enabling factors and for further design or extension of the

program.

More significantly, with due respect to the limitations of this impact study, there would be

value in expanding this research to a large-scale study, possibly incorporating other provinces

where ECE programs have been in operation and also across more cohorts of children to look

at difference for children as they proceed through more senior levels of schools (beyond

grade 1 or 2) .

This study has been important to indicate that ECE does have an impact upon children’s

educational competency and ongoing attendance. It is also been important to demonstrate

that with relatively few resources, good quality and valid impact assessment can be

undertaken. Finally the study also demonstrates the commitment of World Vision, Makira-

Ulawa Provincial government and other stakeholders to children’s education in the Solomon

Island. There is clearly considerable opportunity for AusAID and other donors, together with

the Government of Solomon Islands, to learn from and build upon this commitment.


November 2012

21

Annex One: Numeracy and Literacy Tests

Schedule for Early Number Assessment (SENA 1)

Student’s name:

Class:

Date of initial assessment

D.O.B:

Date of second assessment


November 2012

22

Numeral identification

(1) 3 (2) 6 (3) 10 (4) 2

(5) 9 (6)

8 (7)

5 (8)

0

(9) 7

(10)

4

(11)

23

(12)

15

(13) 12

(14)

43

(15)

13

(16)

20

(17)

100

(18)

66

Forward number word sequences

Start counting from … I’ll tell you when to stop.

(19) 1................32 (20) 62.................73 (21) 96.................113

What is the next number after …?

(22) 5 (23) 9 (24) 13 (25) 19

(26) 27 (27)

80 (28)

69 (29)

46

Backward number word sequences

Count backwards from … I’ll tell you when to stop.

(30) 10...............1 (31) 23...............16 (32) 103...............98

What number comes before…?


November 2012

23

(33) 5 (34) 9 (35) 16 (36) 20

(37) 47 (38) 13 (39) 70 (40) 31

Schedule for Early Number Assessment (SENA 1)

Subitising

How many dots are there?

(41) 4 (42)

6 (43) 5 (44)

3

(45) 4 + 4 (46)

4 + 5

Counting

(47) Put out 5 blue counters. How many blue counters are there?

(48) Put out a pile of red counters. Get me 8 red counters.

(49) Put out 8 red counters and 5 blue counters in two groups.

How many counters altogether?

Addition

(50) 4 + 3 Here are four counters. (Briefly display, then screen.)

Here are three more counters. (Briefly display, then screen.)

How many counters are there altogether?

(51) I have seven apples and I get another two apples.

How many apples do I have altogether?

(52) 9 + 4 Here are nine counters. (Briefly display, then screen.)

Here are four counters. (Briefly display, and then screen.)


November 2012

24

How many counters are there altogether?

Subtraction

(53) I have 7 bananas and I eat 2. How many bananas do I have left?

(54) 12 remove 3 I have 12 counters. (Briefly display, then screen.)

I’m taking away 3 counters. (Remove 3.) How many are left?

(55) 11 remove… = 7 I have 11 counters. (Briefly display, then screen.)

I’m taking away some counters and there are 7 left. (Remove 4 counters.) How

many did I take away?

Multiplication and division

(56) Present a pile of counters, more than 12, to the student. (Randomly spaced, not

in a line. Do not count them out.) Using these counters, make three groups

with four in each group. How many counters are there altogether?

Makira Early Childhood Development Program Impact Report

November 2012

25

Individual analysis sheet (SENA 1)

Student’s name:

D.O.B: Initial interview date:

Numeral identification (Tasks 1 – 18)

Emergent (Level 0) 1–10 (Level 1) 1–20 (Level 2) 1–100 (Level 3)

Forward number word sequences (Tasks 19 – 29)

Emergent (Level 0)

Initial (10) (Level 1)

Intermediate (10)

(Level 2)

Facile (10)

(Level 3)

Facile (30)

(Level 4)

Facile (100)

(Level 5)

Backward number word sequences (Tasks 30 – 40)

Emergent (Level 0)


Intermediate (10)

(Level 2)

Facile (10)

(Level 3)

Facile (30)

(Level 4)

Facile (100)

(Level 5)

Subitising (Tasks 40 – 46)

Emergent (Level 0)

Perceptual (Level 1)

Conceptual (Level 2)

Early arithmetical strategies (Tasks 47 – 55)

Emergent (Stage 0)

Perceptual (Stage 1)

Figurative

(Stage 2)

Counting-on and back (Stage 3)

Facile

(Stage 4)


November 2012

26

Multiplication and division (Task 56)

Unable to form

groups

Able to form

groups

Able to find the total by:

Reference guide (SENA 1)

Numeral identification (Tasks 1–18) [+ extra numeral cards to determine Level 4]

Emergent (Level 0)

1-10 (Level 1)

1-20 (Level 2)

1-100 (Level 3)

1-1000 (Level 4)

May identify some but not all numerals 1-10

Identifies numerals 1-10

Identifies numerals 1-20

Identifies numerals to 100

*Not in SENA 1 materials. Use some numeral cards from SENA 2 if needed.

Forward number word sequences FNWS (Tasks 19–29)

Emergent (Level 0)


Intermediate (10) (Level 2)

Facile (10) (Level 3)



Cannot count to 10

Can count to 10 but cannot give number after

Can count to 10 and give number after, but counts from one

Can count to 10 and give number after

As with Facile (10) but with numbers up to 30


Backward number word sequences BNWS (Tasks 30–40)

Emergent (Level 0)


Intermediate (10) (Level 2)




Cannot count backwards from 10

Can count backwards from 10 but cannot give number before

Can count backwards from 10 and give number before, but counts from one

Can count backward from 10 and give number before



Subitising (Tasks 41–46)

Emergent Perceptual Conceptual

May be able to recognise dot patterns for very small numbers, say 2. Needs to count the dot pattern by ones for larger numbers

Students can instantly recognise dice patterns in questions 41-44

Student is able to see the eight-dot & nine-dot domino pattern as both two groups and as “a whole” in questions 45-46


November 2012

28

Early arithmetic strategies EAS (Tasks 47–55)

Emergent (Stage 0)

Perceptual (Stage 1)

Figurative (Stage 2)

Counting On (Stage 3)

Facile (Stage 4)

Cannot count to 10.

Unable to coordinate

number words with items when counting

Needs to see, touch or hear items to work out answer. Counts from one. Objects remain constantly in view

Can complete concealed items tasks but counts from one

Uses larger number and counts on to find the answer

Uses known facts and other non-count- by-one strategies (e.g. doubles, partitioning) to solve problems


November 2012

29

LETTER IDENTIFICATION SCORE SHEET

Date:

Name: Age: TEST SCORE: /54

Recorder: Date of Birth: STANINE GROUP:

A S Word I.R. A S Word I.R. Confusions:

Letters Unknown:

Comment:

A a

F f

K k

P p

W w

Z z

B b

H h

O o

J j

U u

a

C c

Y y

L l

Q q

M m

D d

N n Recording:

A Alphabet response:

tick (check)

S Letter-sound response:

tick (check)

Word Record the word the

child gives

I.R. Incorrect response:

Record what the child

says

S s

X x

I i

E e

G g

R r

V v

T t


November 2012

30

g

TOTALS

TOTAL SCORE


November 2012

31

WORD READING SCORE SHEET

Use any one list of words

Date:

Name: TEST SCORE: /15

Age: Date of Birth: STANINE GROUP:

Recorder:

Record incorrect responses beside word

LIST A

LIST B

LIST C

I

and

Father

Mother

to

come

are

will

for

here

look

a

me

he

you

shouted

up

at

am

like

school

with

in

went

car

where

get

children

Mr

we

help

going

they

not

big

ready

too

go

this

meet

let

boys

away

on

please

COMMENTS


November 2012

32

WRITING VOCABULARY OBSERVATION SHEET

Date:

Name:

Recorder:

(Fold heading under before child uses sheet)

Age:

Date of Birth:

TEST SCORE:

STANINE GROUP:

COMMENTS


November 2012

33

HEARING AND RECORDING SOUNDS IN WORDS OBSERVATION SHEET

Date:

Name:

Recorder:

(Fold heading under before child uses sheet)

Age:

Date of Birth:

Form:

TEST SCORE:

STANINE GROUP:

/37

OBSERVATION SURVEY SUMMARY SHEET Name: Date: D. of B.: Age: yrs mths

School: Recorder:

Text Titles Errors Error Accuracy Self-correction

Running Words Ratio Rate Ratio

Easy 1: % 1:

Instructional 1: % 1:

Hard 1: % 1:

Directional movement

Analysis of Errors and Self-corrections

Information used or neglected [Meaning (M), Structure or Syntax (S), Visual (V)]

Easy

Instructional

Hard

Cross-checking on information (Note that this behaviour changes over time)

How the

reading

sounds

Easy

Instructional

Hard

Letter

Identi-

fication

Raw Score

Stanine

Concepts

About Print

* Sand

Stones

Shoes

Moon

Word

Reading

* List A

List B

List C

Other (Enter test name)

Writing

Vocabulary

Hearing and Recording Sounds in Words

* A

B

C

D

E


November 2012

35

Other tasks

Writing sample

Story

Spelling

* Circle whatever was used


November 2012

Annex Two: Interview schedules for Makira research

1. Teachers:

The purpose of interviewing teachers at the schools is to have their views on the preparation and

ongoing standards of the children who had attended kindy before coming to school, in comparison

with other children. This will serve to further verify the information we collect from the children.

Introduction

I am here today because I would like to talk to you about the World Vision Girl Child Reading &

Rescue projects funded by World Vision. World Vision has asked me to talk to teachers about the

Early Development Centres on Makira. We want to find out your ideas about this project, what you

think has been good and also whether you think the project is helping the children when they go to

school. We are also talking to families to find out their ideas. We will also be conducting some

assessments of children’s literacy and numeracy to see how they are progressing with their learning.

If I ask you something and you don’t understand what I mean, please ask me to explain again or ask

a different way. If I ask you something you don’t want to answer, it is fine to say: no, I don’t want to

answer that.

If you decide during the talk that you don’t want to keep going you can tell me and we will stop.

What you tell me will be kept private and after we talk I might use your ideas in a written report for

World Vision or in a research paper, but I won’t use your name.

Would you like to talk with me today? Yes/No

Is it ok for me to write down what you say so I can remember it? Yes/No

Is it ok for me to take a photo of you? Yes/No

Is it ok for me to use your ideas in a report or a research paper? Yes/No

If you have any questions or comments about this, please speak to me about them.

If you have any complaints about this research, please speak to ___________________(local World

Vision staff member) or ________________(translator) who will take your complaint to World

Vision and Murdoch University for investigation.

The Questions

a. Are you aware of the children in your class who attended kindy before they came to school?

b. Have you noticed any differences in those children? This might include differences in their

learning? Differences in their behaviour? Differences in their social development, that is the

way that they interact with their friends and other children? Anything else?

c. If there were differences between children who attended Kindy and those that did not at the

beginning of the year, have these differences continued throughout the year?

d. Finally we are interested in how kindy might be different for girls and boys. Have you

observed any differences between the boys and girls who attended kindy and now come to

school?

e. Thank you very much for your time. Is there anything else that you would like to tell me?

37

2. Parents:

The purpose of speaking to parents is to understand whether ECE has influenced their attitude or

ideas about education for the children.

Introduction

Thank you for agreeing to talk to me about your child/children. I am here today because I would like

to talk to you about the World Vision Girl Child Reading & Rescue projects funded by World Vision.

World Vision has asked me to talk to people in this village about the Early Development Centre. We

want to find out your ideas about this project, what you think has been good and maybe not so

good. We are also talking to teachers and children to find out their ideas.


a different way. If I ask you something you don’t want to answer, it is fine to say no, I don’t want to

answer that.









If you have any complaints about this research, please speak to

___________________(local World Vision staff member) or ________________(translator)

who will pass on your complaint to World Vision and Murdoch University for investigation.

The questions

a. Have your children attended kindy? Are they boys or girls?

b. Can you explain what were the good things from sending children to kindy?

c. Do your children now go to school? Why did you send them to school? Was there any

difference between the girls or boys?

d. For how many years do you want them to stay at school? Why?

e. Have you seen any differences in children who attended Kindy compared with those who did

not? (Such as older children in your family or in other families)

f. Thank you very much for answering my questions. Is there anything else that you would like

to tell me?

38

3. Community leaders

The purpose of interviewing leaders in the community is to verify and cross check what the parents

are telling us. We want to check that what they have told us about children attending school is

correct. We also want to get the opinion of the leaders about any differences in children who

attended kindy and those that did not.

Introduction

Thank you for agreeing to talk to me about the children in this community. I am here today because I

would like to talk to you about the World Vision Girl Child Reading & Rescue projects funded by

World Vision. World Vision has asked me to talk to people in this village about the Early

Development Centre. We want to find out your ideas about this project, what you think has been

good and maybe not so good. We are also talking to teachers and children to find out their ideas.


a different way. If I ask you something you don’t want to answer, it is fine to say no, I don’t want to

answer that.









If you have any complaints about this research, please speak to ___________________(local World

Vision staff member) or ________________(translator), who will pass on your complaint to World

Vision and Murdoch University for investigation.

The Questions

a. For children in this community, how many of them attend school?

b. For those children who attended kindy in the community, do they now attend school? Has

going to kindy made it more likely that girls or boys will attend school?

c. Have you noticed any other differences between children who attended kindy and those

that did not?

d. Can you please explain to me why education is important to this community? What do you

hope it will offer for children and their families?

e. Thank you very much answering the questions. Is there anything else that you would like to

tell me?

39

Annex Three

Scoring sheet for test results

School

Child

's

Nam

e

Consent

Age

Yr 1

/2

M/F

Atte

nd

%

Phase 1

ED

C

Phase 2

ED

C

No E

DC

Num

era

l ID

FN

WS

BN

WS

Subitis

i

ng

EAS

LI

Score

Word

Readin

g

Writin

g

Vocab

HRSW

Score

Readin

g

Level

Readin

g

Accura

c

y

world vision solomon islands makira early childhood ...the world vision ece program is located in...

Documents