the achievement of learners in early grade reading in leb ......leb-acoli and english was higher...

The Achievement of Learners in Early Grade Reading in Leb-Acoli

Midline Report 2017

NATIONAL ASSESSMENT OF PROGRESS IN EDUCATION (NAPE)

The Achievement of Learners in Early Grade Reading Leb-Acoli

Midline Report 2017

NATIONAL ASSESSMENT OF PROGRESS IN EDUCATION (NAPE)

PREPARED BY:

Dan Kyagaba Principal Examinations Officer

Amos Opaman Senior Examinations Officer

Jumanyol Kennedy Examinations OfficerSserunkuuma Lutalo Bbosa Examinations Officer

PUBLISHED BY: Uganda National Examinations Board

All rights reserved. Reproduction of this publication for educational or other non-commercial purposes is allowed without prior permission from the copyright holder provided the source is acknowledged.

Reproduction for re-sale or other commercial purposes is prohibited, unless prior permission, in writing, is obtained from UNEB.

FOR FURTHER INFORMATION, PLEASE REFER TO:The Executive Secretary,Uganda National Examinations BoardP.O. Box 7066,Kampala, Uganda

Telephone: 256 41 4 286 173 256 41 4 286 636

Email: [email protected]

Website: www.uneb.ac.ug

©2017 Uganda National Examinations Board

A publication of the Achievement of Learners in Early Grade Reading in Leb-Acoli Midline Report, Uganda National Examinations Board.

http://www.uneb.ac.ug/

EGRA MIDLINE REPORT 2017, LEB ACOLI i

ACKNOWLEDGEMENTSWe glorify the Almighty God for His Grace that enabled us to accomplish the UNEB-EGRA activities for the 2017 period.

Special appreciation also goes to the Ministry of Education and Sports for the UTSEP and continued support to the UNEB activities in the country.

We are sincerely grateful to the UTSEP consortium for the financial support and guidance that has helped the growth of UNEB-EGRA.

We are also greatly indebted to our brothers and sisters under the SHRP, LARA, Mango Tree and others who in different occasions provided us with technical assistance.

The chairperson, Professor Lutalo-Bosa and the members of the NAPE Advisory Committee all deserve a pat on the back for their invaluable advice which has greatly helped to guide the UNEB-EGRA exercise.

We extend our sincere appreciation to the learners, teachers, headteachers of schools that were sampled, Assessors, the respective District Education officials of UNEB-EGRA districts and the staff of UNEB for their various roles in the 2017 UNEB-EGRA.

Finally, we acknowledge the invaluable contribution of the rest of the NAPE officers: Ms. Mariam Namirimu, Mr. John Bwete, Ms. Solomy Namukwaya, Ms. Josephine Nassonko, Mrs. Barbara K. Barigye, Mrs. Dorothy K. Ssettuba, Mrs. Sarah Mashamba Okuni and Mrs. Grace Nakiganda Kamulegeya. This accomplishment would not have been possible without you.

EGRA MIDLINE REPORT 2017, LEB ACOLIii

TABLE OF CONTENTS

ACKNOWLEDGEMENT ............................................................................................................................................. I

FOREWORD ......................................................................................................................................................... 2

EXECUTIVE SUMMARY ........................................................................................................................................... 3

CHAPTER 1 ......................................................................................................................................................... 6

Introduction ................................................................................................................................................................. 61.1 Background ............................................................................................................................................... 61.2 Baseline Survey ........................................................................................................................................ 61.3 Midline Survey........................................................................................................................................... 6

CHAPTER 2 ......................................................................................................................................................... 8

Methodology ................................................................................................................................................................. 82.1 Introduction ............................................................................................................................................... 82.2 Instruments ............................................................................................................................................... 82.3 Survey Design ........................................................................................................................................... 92.4 Data Collection........................................................................................................................................ 102.5 Statistical Data Analysis .......................................................................................................................... 11

CHAPTER 3 ....................................................................................................................................................... 12

Letter Identification ...................................................................................................................................................... 123.1 Introduction ............................................................................................................................................. 123.2 The Achievement of P 1, P 2 and P 3 Learners in Letter Identification .................................................. 12

CHAPTER 4 ....................................................................................................................................................... 12

Letter Sounds ............................................................................................................................................................... 134.1 Introduction ............................................................................................................................................. 134.2 The Achievement of P 1, P 2 and P 3 Learners in Telling Letter Sounds ................................................ 134.3 Mean number of Letters sounded correctly per minute ......................................................................... 14

EGRA MIDLINE REPORT 2017, LEB ACOLI iii

CHAPTER 5 ....................................................................................................................................................... 15

Vocabulary ............................................................................................................................................................... 155.1 Introduction ............................................................................................................................................. 155.2 Vocabulary: Identification of Objects ...................................................................................................... 155.2.1 The Achievement of P 1, P 2 and P 3 Learners in Object Identification ................................................. 155.3 Vocabulary: Description of Activities in Pictures ..................................................................................... 165.3.1 The Achievement of P 1, P 2 and P 3 Learners in Description of Activities in Pictures.......................... 16

CHAPTER 6 ....................................................................................................................................................... 17

Listening Comprehension............................................................................................................................................. 176.1 Introduction ............................................................................................................................................. 176.2 The Achievement of P 1, P 2 and P 3 Learners in Listening Comprehension ........................................ 17

CHAPTER 7 ....................................................................................................................................................... 18

Oral Passage Reading And Comprehension ................................................................................................................ 187.1 Introduction ............................................................................................................................................. 187.2 The Achievement of P 1, P 2 and P 3 Learners in Fluency ..................................................................... 18 7.3 The Achievement of P 1, P 2 and P 3 Learners in Fluency ..................................................................... 197.4 The Achievement of P 1, P 2 and P 3 Learners in Reading Comprehension ......................................... 19

CHAPTER 8 ....................................................................................................................................................... 208.1 Introduction ..................................................................................................................................................... 208.2 The Level of Adherence of Teachers to the 5Ts .............................................................................................. 20

CHAPTER 9 ....................................................................................................................................................... 21

Views Of Headteachers And Teachers On Their Roles In Promoting Early Grade Reading ........................................ 219.0 Introduction ............................................................................................................................................. 219.1 Head teachers’ Role in Early Grade Reading ......................................................................................... 219.2 Teachers supervised during Reading lessons......................................................................................... 229.3 Teachers having ever been trained to teach Reading ............................................................................. 23

RECOMMENDATIONS ............................................................................................................................................ 24

EGRA MIDLINE REPORT 2017, LEB ACOLI 1

ACRONYMS AND ABBREVIATIONS

CC Coordinating Centre

CCT Coordinating Centre Tutors

DES Directorate of Education Standards

DQA Data Quality Assurance

EFA Education for All

EGRA Early Grade Reading Assessment

EPPA Education Planning and Policy Analysis

GPE Global Partnership for Education

IRR Inter-rater Reliability

LARA Literacy Achievement and Retention Activity

LCs Language Coordinators

MoES Ministry of Education and Sports

NAPE National Assessment of Progress in Education

NCDC National Curriculum Development Centre

P 1 Primary One

P 2 Primary Two

P 3 Primary Three

RTI Research Triangle Institute

SHRP School Health and Reading Programme

TAs Tool Administrators (Assessors)

TDMS Teacher Development and Management System

TIET Teacher Instructor Education and Training

TLs Team Leaders

UNEB Uganda National Examinations Board

UTSEP Uganda Teacher and School Effectiveness Project

EGRA MIDLINE REPORT 2017, LEB ACOLI2

FOREWORDThe Monitoring of learners’ achievement is an indispensable activity in the learning process, if the objectives of teaching are to be met and standards ensured. At the infant level of education, regular monitoring of learners’ achievement through continuous assessment, most especially at the class level, is considered to be a cornerstone of the Early Grade Reading (EGR) approach.

With the inception of the Uganda Teacher and School Effectiveness Project (UTSEP) in 2015, UNEB, through NAPE, was privileged to be assigned the task of assessing the P 1, P 2 and P 3 learners’ reading skills under the EGR approach. This assessment is the Early Grade Reading Assessment (EGRA). This kind of assessment was already going on in schools in some other districts by other players, notably School Health and Reading Programme (SHRP). Therefore, due to the existence of other forms of EGRA by other stakeholders, EGRA by UNEB was named UNEB-EGRA.

UNEB conducted the first UNEB-EGRA survey in October, 2016 as a baseline survey. The assessment was carried out in 11 local languages and then English for all the learners assessed. The findings from the baseline UNEB-EGRA were released and disseminated to the stakeholders. For further detailed reading, the baseline report can be accessed on the UNEB website: www.uneb.ac.ug

In October, 2017, UNEB conducted the midline UNEB-EGRA survey. The findings from the midline UNEB-EGRA survey are presented in this report. The report provides information on what learners in infant classes (P 1, P 2 and P 3) know and can do in the basic reading skills. Such information has a lot of implication on the teaching and learning process of reading in particular, and on all supporting efforts in that process especially during early grade reading.

It is my sincere hope that all stakeholders will use the report to enrich their contribution towards the teaching of reading to the infants.

Dan N. Odongo

EXECUTIVE SECRETARY


EXECUTIVE SUMMARYThe Ministry of Education and Sports in Uganda is pre-occupied with improving the quality of education in the country through improving the quality of learning obtaining in the schools. One of the aspects in the learning process that has greatly attracted the attention of key education stakeholders in this endeavour is the level of learners’ reading skills. The Ministry of Education and Sports has, of recent, adopted the Early Grade Reading (EGR) model which focuses on exposure of learners to basic reading skills. Early Grade Reading Assessment (EGRA) helps to monitor the progress of learners in the acquisition of reading skills during the early years of schooling. This midline UNEB-EGRA was conducted in 11 local languages in September-October 2017. It is a follow up of the baseline assessment in October 2016.

The same study design was implemented as at the baseline level. At baseline, a stratified two-stage cluster sampling design was used. Data from Education Management Information System (EMIS) 2015 provided the list of public primary schools and their respective pupil enrolment figures for each class. A proportional number of schools was randomly selected from each TDMS Coordinating Centre within selected districts (Table 1.01, Page 7). These same schools were visited during the midline survey. A simple random sample of 20, 20 and 10 learners present on the day of the survey, were selected from pupils of each of P 1, P 2 and P 3, respectively. Survey weights for sampled learners were accounted for in the analysis in order to obtain unbiased estimates of the desired benchmarks (proportions and means).

A total of 189 Assessors and 12 Data Quality Assurance (DQA) officers were recruited for the exercise. They underwent rigorous training to ensure that they were familiar with the administration of an oral interview, as well as the specific implementation and coding practices associated with EGRA. The assessment tools were uploaded on Tangerine software. Since EGRA is individually administered by assessors, an Inter-rater Reliability (IRR) was computed out twice to ensure consistency of assessors with the gold standard in the administration of the EGRA tool. The Piloting of EGRA instruments was also done before scale-up to full data collection. This was done to ensure that: the instruments perform as expected, the assessors gain valuable hands-on experience and trainers identify the gaps in assessors’ understanding and adaptability skills.

Each sampled learner in P 1, P 2 and P 3 classes from 405 public primary schools was orally assessed individually both in the local language that is used as a medium of instruction and in English. In addition, there were three (3) contextual instruments administered, namely: Lesson observation, Pupil interview schedule and Headteacher interview schedule. All these instruments were intended to establish the context in which learners learn to read at school and at home.

OVERALL LEVEL OF ACHIEVEMENT OF LEARNERS

Letter Identification

The proportion of learners who were able to identify at least 4 out of 5 letters in Leb-Acoli and English was higher at midline than at baseline.

At midline the proportion of P 2 learners who were able to identify at least 4 out of 5 letters was higher in Leb-Acoli than in English.

The proportion of P 2 learners at midline who were able to identify at least 4 out of 5 letters in Leb-Acoli and English was higher than that of P 2 learners at baseline.

Telling Letter Sounds

The proportion of learners who were able to sound correctly one or more letters per minute in Leb-Acoli and English was higher at midline than at baseline.


At midline, the proportion of P 2 learners who were able to sound correctly one or more letters per minute was higher in Leb-Acoli than in English.

The proportion of P 2 learners at midline who were able to sound correctly one or more letters per minute in Leb-Acoli and English was higher than that of P 2 learners at baseline.

The mean number of letters sounded correctly per minute in Leb-Acoli and English was higher at midline than at baseline.

At midline the mean number of letters sounded correctly per minute by P 2 learners was higher in Leb-Acoli than in English.

The mean number of letters sounded correctly per minute by P 2 learners at midline in Leb-Acoli and English was higher than that of P 2 learners at baseline.

Vocabulary

The proportion of learners who were able to identify at least 3 out 4 of objects in Leb-Acoli and English was higher at midline than at baseline.

At midline the proportion of P 2 learners who were able to identify at least 3 out of 4 objects was higher in Leb-Acoli than in English.

The proportion of P 2 learners at midline who were able to identify at least 3 out of 4 objects in Leb-Acoli and English was higher than that of P 2 learners at baseline.

The proportion of learners who were able to describe activities in at least 2 out of 3 pictures in Leb-Acoli and English was higher at midline than at baseline.

At midline the proportion of P 2 learners who were able to describe activities in at least 2 out of 3 pictures was higher in Leb-Acoli than in English.

Listening Comprehension

The proportion of learners who were able to answer at least 3 out of listening comprehension questions in Leb-Acoli and English was higher at midline than at baseline.

At midline the proportion of P 2 learners who were able to answer at least 3 out of 4 listening comprehension questions was higher in Leb-Acoli than English.

The proportion of P 2 learners at midline who were able to answer at least 3 out of 4 listening comprehension questions in Leb-Acoli and English was higher than that of P 2 learners at baseline.

Oral Passage Reading (Fluency)

The proportion of learners who were able to read correctly one or more words per minute in Leb-Acoli and English was higher at midline than at baseline.

At midline, the proportion of P 2 learners who were able to read correctly one or more words per minute was higher in Leb-Acoli than English.

The proportion of P 2 learners at midline who were able to read correctly one or more words per minute in Leb-Acoli and English was higher than that of P 2 learners at baseline.

The proportion of learners who were able to read correctly 20 or more words per minute in Leb-Acoli and English was higher at midline than at baseline.


At midline the proportion of P 2 learners who were able to read correctly 20 or more words per minute was higher in English than in Leb-Acoli.

The proportion of P 2 learners at midline who were able to read correctly 20 or more words per minute in Leb-Acoli was higher than that of P 2 learners at baseline.

Reading Comprehension

The proportion of learners who were able to answer correctly one or more comprehension questions in Leb-Acoli and English was higher at midline than at baseline.

At midline the proportion of P 2 learners who were able to answer correctly one or more comprehension questions was higher in Leb-Acoli than English.

The proportion of P 2 learners at midline who were able to answer correctly one or more comprehension questions in Leb-Acoli and English was higher than that of P 2 learners at baseline.

Level of Adherence of Teachers to the Early Grade Reading (EGR) model

At midline, in almost 9 in 10 lessons observed, the P 2 teachers taught reading lessons in the mother tongue, while 3 in 10 of the lessons observed had learners with textbooks/printed materials in their hands (i.e., 1:1 learner to text book ratio).

The proportion of learners who were able to read individually from printed materials or books was higher at midline than at baseline.

Views of Headteachers and Teachers on Promoting Early Grade Reading

Headteachers’ Role in Early Grade Reading Generally, the proportion of headteachers by their perceived role in promoting Early Grade Reading was higher at midline than at baseline. Most headteachers, at midline viewed supervisory, logistical support and sensitization as their main roles in promoting Early Grade Reading. This was the same trend at baseline.

Teachers supervised during Reading lessons:At midline, 9 in 10 teachers reported that they have ever been supervised during the reading lesson.

Teachers ever trained to teach Reading:The proportion of teachers who reported that they were trained to teach reading was higher at midline than at baseline.

Apart from Moes/TIET, a number of other organizations have trained teachers to teach reading. They include RTI/SHRP, Mango Tree, Aga-Khan and the District Education Office.

Many teachers (95.9%) reported that they applied what they learnt during their EGR training.


CHAPTER 1INTRODUCTION1.1 BACKGROUNDThe Ministry of Education and Sports in Uganda is pre-occupied with improving the quality of education in the country through improving the quality of learning obtaining in the schools. One of the aspects in the learning process that has greatly attracted the attention of the key education stakeholders in this endeavour is the level of learners’ reading skills. The Ministry of Education has, of recent, adopted the Early Grade Reading (EGR) model which focuses on exposure of the learners to the basic reading skills.

Through the Uganda Teacher and School Effectiveness Project (UTSEP), the Ministry of Education and Sports formally supports the EGR efforts. Such support has led to the training of teachers in the Early Grade Reading model and the provision of instructional materials for EGR to schools, among others. The support started with 27 districts where EGR is implemented (see Table 1.01). The target classes for EGR are P 1, P 2 and P 3. It is in line with that support, that Uganda National Examinations Board (UNEB) through its National Assessment of Progress in Education (NAPE) was requested to implement the EGR corresponding assessment. That is, Early Grade Reading Assessment (EGRA) among P 1, P 2 and P 3 learners. This came to be known as UNEB-EGRA. UNEB conducted the first EGRA in October 2016 as a baseline survey. This was done in selected schools in the 27 districts, in 11 local languages used as languages of instruction in primary schools in those districts (see Table 1.01). The same learners were also assessed in English.

1.2 BASELINE SURVEYThe baseline survey was intended to explore and determine what learners know and can do in terms of the key basic reading skills, as defined under the Early Grade Reading model. The baseline survey also explored the level of adherence of the teachers of reading to the EGR model. The assessment also sought the head teachers’ and teachers’ views on selected aspects of teaching reading. Among the three classes assessed, that is, P 1, P 2 and P 3, the main class of interest for the baseline UNEB-EGRA was P 1.

The baseline findings were released in the report entitled ‘The Achievement of Learners in Early Grade Reading in Selected Districts in Uganda’. There were also reports customized to each of the 11 local languages. These reports can be accessed on the UNEB website www.uneb.ac.ug

1.3 MIDLINE SURVEYFrom September to October 2017, UNEB conducted the second UNEB-EGRA. This was the midline survey. The midline survey employed the same methodology as had been applied during the baseline survey. The midline UNEB-EGRA was meant to assess what the learners know and can do departing from the baseline findings and in view of the multiplicity of EGR interventions that have been carried out in all public primary schools in the 27 districts. The assessment was conducted among P 1, P 2 and P 3 leaners, as was the case during the baseline survey. However, during the midline, the main class of interest was P 2 and that is where emphasis in reporting is more focused. The P 2 class of midline survey was the P 1 class during the baseline survey and the main class of interest then. This is the class which has been at the centre of most of the EGR interventions. So, the midline survey portrays what P 2 learners can do after being subjected to the EGR intervention compared to what they could do during the baseline before the EGR interventions.

http://www.uneb.ac.ug


The findings of the midline survey are presented in this report. The report is comprised of eight chapters. Chapter 1: Introduction, Chapter 2: Survey Procedures, Chapter 3: Letter Identification, Chapter 4: Telling Letter Sounds, Chapter 5: Vocabulary Knowledge, Chapter 6: Listening Comprehension, Chapter 7: Oral Reading Comprehension and Chapter 8: Level of Adherence to the EGR Model. Chapters 3, 4, 5, 6 and 7 present the results of the learners’ level of achievement in the specified areas of early grade reading. Each of those chapters begins with an introduction, followed by a description of how the respective skill area was assessed, and then a discussion of the findings as reflected in the figures for each of the 11 local languages and English.

Each of the figures shown in chapters 3 – 8, illustrates the achievement of the learners in 2016 and the same cohort in 2017 when they are in the next class, by local language and English.

TABLE 1.01: LOCAL LANGUAGES ASSESSED BY DISTRICT

LOCAL LANGUAGE DISTRICTAteso Amuria, Bukedea, Soroti

Leb-Acoli Agago, Amuru, Lamwo, Nwoya

Leb-Lango Alebtong, Amolatar, Dokolo, Oyam

Lhukonzo Bundibugyo

Luganda Kyankwanzi and Mubende

Lugbarati Maracha

Lumasaaba Bududa and Bulambuli

Lusoga Bugiri, Buyende, Kaliro, Luuka, Mayuge, Namayingo and Namutumba

Ngakarimojong Kotido

Runayankore-Rukiga Kamwenge

Runyoro-Rutooro Kibaale


CHAPTER 2METHODOLOGY2.1 IntroductionThis chapter presents a description of the instruments and procedures that were used in selecting the sample (schools and learners), identifying and training the Assessors, Language coordinators and Monitors. It also describes the processes of pre-testing the instruments, collection, management and analysis of data.

2.2 InstrumentsThese consisted of oral tests and contextual instruments.

2.2.1 Oral TestThe oral test was developed in English and then later translated into all the 11 local languages. The same test was administered to each individual sampled learner in P 1, P 2 and P 3 in both the local language and English. The oral test was developed following the EGRA framework. The composition of the oral test is presented in Table 2.01.

TABLE 2.01: COMPOSITION OF THE ORAL TEST

COMPONENTS OF READING COMPETENCES BENCH MARKSALPHABETIC PRINCIPLE(Relating the letter name to its symbol)

Identifies letters of the alphabet (Knowledge of letter names)

Identifies at least 4 letters out of 5, i.e. 80% or more.

PHONEMIC AWARENESS Tells the sound of the letter Sounds correctly at least 1 letter Mean number of letters sounded correctly per minute.

FLUENCY (Oral passage reading and Comprehension)

Reading Reads texts with speed and accuracy

Reads correctly at least 20 words per minute Reads correctly at least 1 word per minute

Comprehension Reads the passage fluently with comprehension

Answers correctly at least 1 question out of 4, i.e. 25% or more.

VOCABULARY (Knowledge and use of words appropriately and creatively).

Demonstrates understanding of vocabulary e.g. identifying objects

Identifies correctly at least 3 objects out of 4, i.e. 75% or more

Describes the activities in the pictures using the learnt

vocabulary.

Describes correctly at least 2 out of 3 activities shown in the pictures, i.e. 67% or more

COMPREHENSION Listens to a story and derives meaning from it

Answers correctly at least 3 questions out of 4, i.e., 75% or more


2.2.2 Contextual instrumentsThere were three contextual instruments namely: Lesson observation guide, Pupil interview schedule and Headteacher interview schedule. These contextual instruments were used to provide information on the context in which learners learn to read. At midline, P 2 reading lessons were observed in 171 schools while at baseline, P 1 reading lessons were observed in 168 schools out of the 405 randomly selected public schools.

2.2.3 Pre-testing InstrumentsThe instruments were pre-tested in all the 11 local languages. This was meant to refine the tools and to acquaint the Assessors with the administration of oral tests and the optional use of the tablet (the mobile data collection device).

2.3 Survey Design2.3.1 Survey Population The target population consisted of learners in P 1, P 2 and P 3 in all the public primary schools in the 27 districts under UTSEP.

2.3.2 Sampling DesignA stratified two-stage cluster sampling design was used. The first stage involved selecting a random sample of schools, stratified by coordinating centres within the district. Within each stratum, the number of schools selected was proportional to the number of schools in the coordinating centres. A total of 15 schools were selected from each district except in Agago (11 schools) and Namayingo (8 schools).

At the second stage, a simple random sample of 20, 20 and 10 learners present on the day of the survey, were selected from pupils of each of P 1, P 2 and P 3, respectively. Random selection of schools and learners was done to minimize selection bias.

2.3.3 Selection of SchoolsA list of public primary schools from the Education Management Information System (EMIS) 2015, showing the pupil enrolment figures for P 1, P 2 and P 3 provided the sampling frame for schools. It was prudent to select schools basing on P 3 enrolment, because the number of learners in P 3 in a school is usually less than that of P 1 and P 2. This, therefore, ensured that the number of learners in P 1 and P 2 in a school was big enough to meet the minimum sample size.

2.3.4 Selection of TesteesA simple random sample of 20 learners was obtained from each of P 1 and P 2 classes and 10 learners from P 3 class. A pre-prepared selection list (a matrix of class size and random numbers) was used to select the learners (Appendix 1).


2.3.5 Sample SizeThe distribution of the sample size for the testees by local language and class is shown in Table 2.02.

TABLE 2.02: DISTRIBUTION OF TESTEES BY LOCAL LANGUAGE AND CLASS

LOCAL LANGUAGE CLASSP 1 P 2 P 3

Ateso 902 900 452

Leb-Acoli 1,088 1,072 555

Leb-Lango 1,202 1,200 603

Lhukonzo 301 282 148

Luganda 582 559 306

Lugbarati 302 299 151

Lumasaaba 598 595 300

Lusoga 1,970 1,973 981

Ngakarimojong 313 258 149

Runyankore-Rukiga 309 304 150

Runyoro-Rutooro 311 307 151

TOTAL 7,878 7,749 3,946

2.3.6 Sampling WeightsSampling weights were computed to reflect the probability of learners sampled.

At the first stage, sampling weight, W1i, for each sampled school was computed, i.e. the inverse of the probability of a school being selected from amongst the public schools in the district (number of public schools in the district divided by 15, the number of schools sampled).

At the second stage, sampling weight, W2j , for each learner was computed (Number of learners present in the class per school at the time of selection of learners divided by the number of learners sampled from the class, P 1 – 20, P 2 – 10, P 3 - 10).

At the last stage, the sampling weight, Wij, was computed by multiplying the weight for each school, W1i , and weight for each learner, W2j, i.e. Wij = W1i X W2j; where, Wij is the weight for the jth learner selected from the ith school.

2.4 Data Collection2.4.1 Training of AssessorsThe 189 Assessors and 12 Data Quality Assurance (DQA) officers were selected from among NAPE scorers of Literacy, language panellists, language item writers, other practising teachers, retired educationists and UNEB staff. The Assessors were selected through an interview during which their fluency in the local language and competence in the use of mobile data collection device (tablets) was evaluated.

There was a rigorous 10-day training of the DQA officers and Assessors at Kyambogo Primary School in Kampala. The training was conducted by officers from UNEB guided by the UNEB-EGRA manual. The training was intended to familiarise Assessors and DQA officers with the administration of an oral test and with the specific implementation and coding practices associated with EGRA.

Since EGRA is individually administered by Assessors, an Inter-rater Reliability (IRR) was computed twice to ensure consistency of Assessors with the gold standard1 in the administration of the EGRA tool. Piloting of

1 Pre-definedoralresponses


EGRA instruments was also done before scale-up to full data collection, to ensure the instruments performed as expected, assessors gained valuable hands-on experience and trainers identified gaps in assessors’ proficiency.

There were 27 teams each comprising 7 Assessors, one of whom was designated as Team Leader. The DQA officers or the Team leaders were also trained on how to observe reading lessons as well.

The distribution of Assessors and DQA officers by local languages assessed is shown in Table 2.03.

TABLE 2.03: DISTRIBUTION OF ASSESSORS AND DQA OFFICERS ACCORDING TO LOCAL LANGUAGES

LOCAL LANGUAGE NUMBER OF TEAMS DQAS NUMBER OF ASSESSORSAteso 3 1 21

Leb-Acoli 4 1 28

Leb-Lango 4 1 28

Lhukonzo 1 1 7

Luganda 2 1 14

Lugbarati 1 1 7

Lumasaaba 2 1 14

Lusoga 7 2 49

Ngakarimojong 1 1 7

Runyankore-Rukiga 1 1 7

Runyoro-Rutooro 1 1 7

TOTAL 27 12 189

2.4.2 Field work The field work lasted 21 days. For each language, the DQA officer(s) planned the school(s) to be visited daily using the list of sampled schools provided. The data collection exercise was guided by a field programme.

Data were collected using Samsung Galaxy Tab S28.0 on which data collection software (Tangerine) had been installed. An assessor would guide the learner through the assessment and then capture the responses on the tablet. Data were uploaded to the cloud by DQA officers at the end of every day’s work. On a daily basis, the data manager stationed at UNEB Headquarters checked the uploaded data for quality assurance. The quality assurance report was sent to the respective DQA to inform the next day’s debriefing with the assessors.

There was a team of Monitors comprising senior officers from UNEB, UTSEP, TIET, Basic and Secondary Education departments of Ministry of Education and Sports as well as retired educationists. These monitored the data collection process in all the 27 districts.

2.5 Statistical Data AnalysisThe analysis was done using STATA (Version 13.0) statistical package. The results were analyzed in terms of means and proportions in both local language and English.

The computation of the final proportion and mean took into consideration the complex survey design (weighted cluster means within strata) and population finite correction. Variances of the estimators that are smooth functions of the sample data: mean and proportion were estimated using the Jackknife error estimation approach.


CHAPTER 3LETTER IDENTIFICATION3.1 IntroductionThis chapter presents the achievement of learners in letter identification. A testee was presented with a stimulus (a sheet of paper with 11 letters of Leb-Acoli or English Alphabet) depending on whether the assessment was in Leb-Acoli or English. The assessor read out 5 of the 11, letters. The 5 letters were chosen in advance. The testee was asked to touch the letter that matched the name of the letter the assessor read out. The testee was allowed 5 seconds to respond before the assessor proceeded to another letter.

TABLE 3.01: MEASURE OF THE SKILL OF LETTER IDENTIFICATION

EGRA SUBTASK HOW IT WAS MEASURED

FOUNDATIONAL READING SKILL

HOW LEARNER DEMONSTRATES THE SKILL

Letter Identification Identifies at least 4 letters out of 5

Knowledge of letters

Learner touches the letter that matches the letter name read out by the assessor

3.2 The Achievement of P 1, P 2 and P 3 Learners in Letter IdentificationThe achievement of P 1, P 2 and P 3 learners in letter identification in Leb-Acoli and English is presented in Figure 3.01.

FIGURE 3.01: LEARNERS IDENTIFYING AT LEAST 4 OUT OF 5 LETTERS IN LEB-ACOLI AND ENGLISHFigure 3.0.1

Baseline Midline

P12016

12.9%

39.4%

P22017

46.8%

P32017

26.8%

P22016

P12016

P22017

P32017

P22016

Leb-Acoli English

9.0%

25.8%

54.4%

17.6%

The proportion of learners who were able to identify at least 4 out of 5 letters in Leb-Acoli and English was higher at midline than at baseline.

At midline the proportion of P 2 learners who were able to identify at least 4 out of 5 letters was higher in Leb-Acoli than in English.

The proportion of P 2 learners at midline who were able to identify at least 4 out of 5 letters in Leb-Acoli and English was higher than that of P 2 learners at baseline.


CHAPTER 4LETTER SOUNDS4.1 IntroductionThis chapter presents the achievement of learners in telling letter sounds. Letter sound knowledge is considered a prerequisite skill for beginning reading and has been found to be a strong predictor of reading growth (Adams, 1990). One of the main differences between successful readers and struggling readers is their ability to use the letter-sound correspondences to decode new words they encounter in text and to encode (spell) the words they write (Juel, 1991 and Ehri, 1998).

A learner was presented with a stimulus (sheet of paper) with 60 letters of the Leb-Acoli or English Alphabet in a random combination of the capital and lowercase depending on whether the assessment was in Leb-Acoli or English. The learner was asked to tell the sound associated with as many of the letters as they could identify within one minute. This subtask was timed to take not more than 60 seconds.

TABLE 4.01: MEASURE OF SKILL OF TELLING LETTER SOUNDS



HOW LEARNER DEMONSTRATES

Tell the sound of the letter

Sounds correctly one or more letters per minute

Letter-sound correspondence

Learner tells the sounds of the letters presented to her/him in both upper and lower case in a random order as fast as possible.

Mean number of letters sounded correctly per minute

The achievement of learners in telling letter sounds is presented using two benchmarks: the proportion of learners who sounded correctly one or more letters (section 4.2) and the mean number of letters sounded correctly per minute (section 4.4).

4.2 The Achievement of P 1, P 2 and P 3 Learners in Telling Letter SoundsThe percentages of P 1, P 2 and P 3 learners who sounded correctly one or more letters in Leb-Acoli and English is presented in Figure 4.01.

FIGURE 4.01: LEARNERS ABLE TO SOUND CORRECTLY ONE OR MORE LETTERS PER MINUTE Figure 4.0.3

46.7%

70.4%57.5%

Baseline Midline

P12016

59.5%

81.9%

P22017

77.2%

P32017

74.4%

P22016

P12016

P22017

P32017

P22016

Leb-Acoli English

61.0%


The proportion of learners who were able to sound correctly one or more letters per minute in Leb-Acoli and English was higher at midline than at baseline.

At midline, the proportion of P 2 learners who were able to sound correctly one or more letters per minute was higher in Leb-Acoli than in English.

The proportion of P 2 learners at midline who were able to sound correctly one or more letters per minute in Leb-Acoli and English was higher than that of P 2 learners at baseline.

4.3 Mean number of Letters sounded correctly per minuteThe mean number of letters sounded correctly per minute by P 1, P 2 and P 3 learners in Leb-Acoli and English is presented in Figure 4.02.

FIGURE 4.02: MEAN NUMBER OF LETTERS SOUNDED CORRECTLY PER MINUTEFigure 4.14

Baseline Midline

Leb-Acoli English

P12016

2.1% 5.2%

P22017

4.7%

P32017

3.2%

P22016

1.7% 4.2% 2.2%

P12016

P22017

P32017

P22016

2.9%

The mean number of letters sounded correctly per minute in Leb-Acoli and English was higher at midline than at baseline.

At midline the mean number of letters sounded correctly per minute by P 2 learners was higher in Leb-Acoli than in English.

The mean number of letters sounded correctly per minute by P 2 learners at midline in Leb-Acoli and English was higher than that of P 2 learners at baseline.


CHAPTER 5VOCABULARY5.1 IntroductionThis chapter presents the achievement of learners in Vocabulary. Vocabulary was assessed using two tasks: identification of objects and description of activities in pictures. Learners’ achievement in identification of objects is presented first, followed by description of activities in the pictures.

5.2 Vocabulary: Identification of ObjectsA stimulus (a sheet of paper with 4 objects in Leb-Acoli or English), depending on whether the assessment was in Leb-Acoli or English, was presented to the testee. The assessor read out the words that matched the objects on the stimulus. The testee was asked to touch the object that matched the word for the object the assessor read out. The testee was allowed 5 seconds to respond before the assessor proceeded to another object.

TABLE 5.01: MEASURE OF SKILL OF IDENTIFICATION OF OBJECTS




Tell the sound of the letter

Identifies at least 3 objects out of 4

Knowledge of vocabulary

Learner touches the object that matches the word for the object read out by the assessor

5.2.1 The Achievement of P 1, P 2 and P 3 Learners in Object IdentificationThe achievement of P 1, P 2 and P 3 learners in object identification in Leb-Acoli and English is presented in Figure 5.01.

FIGURE 5.01: LEARNERS ABLE TO IDENTIFY AT LEAST 3 OUT OF 4 OBJECTS

71.8%

90.2% 88.5%

Baseline Midline

P12016

99.4% 99.9%

P22017

99.8%

P32017

99.6%

P22016

P12016

P22017

P32017

P22016

Leb-Acoli English

95.7%


The proportion of learners who were able to identify at least 3 out 4 of objects in Leb-Acoli and English was higher at midline than at baseline.

At midline the proportion of P 2 learners who were able to identify at least 3 out of 4 objects was higher in Leb-Acoli than in English.

The proportion of P 2 learners at midline who were able to identify at least 3 out of 4 objects in Leb-Acoli and English was higher than that of P 2 learners at baseline.

5.3 Vocabulary: Description of Activities in PicturesA stimuli (a card with a picture of a person doing an activity) was flashed at the learner. The assessor instructed the testee to look at the card carefully and describe the activity in the picture. The testee was allowed 5 seconds to respond before the assessor proceeded to flash another card.

TABLE 5.02: MEASURE OF THE SKILL OF DESCRIPTION OF ACTIVITIES IN PICTURES




Description of activities in the pictures

Describes activities in at least 2 out of 3 pictures

Knowledge of vocabulary

Learner describes the activity in the picture on the card flashed by the Assessor

5.3.1 The Achievement of P 1, P 2 and P 3 Learners in Description of Activities in PicturesThe achievement of P 1, P 2 and P 3 learners in description of Activities in pictures in Leb-Acoli and English is presented in Figure 5.02.

FIGURE 5.02: LEARNERS ABLE TO DESCRIBE ACTIVITIES IN AT LEAST 2 OUT OF 3 PICTURES

5.6% 15.7% 10.2%

Baseline Midline

P12016

98.0% 99.2%

P22017

99.3%

P32017

99.5%

P22016

P12016

P22017

P32017

P22016

Leb-Acoli English

30.3%

The proportion of learners who were able to describe activities in at least 2 out of 3 pictures in Leb-Acoli and English was higher at midline than at baseline.

At midline the proportion of P 2 learners who were able to describe activities in at least 2 out of 3 pictures was higher in Leb-Acoli than in English.


CHAPTER 6LISTENING COMPREHENSION6.1 IntroductionThis chapter presents the achievement of P 1, P 2 and P 3 learners in Listening Comprehension for both local language and English.

A story was read to the testee. The testee was expected to listen to the story and orally answer four questions related to the text. Ten seconds were allowed for each response. If the testee answered some of the questions or none, the Assessor would read the story and the unanswered questions for a second time.

TABLE 6.01: MEASURE OF THE SKILL OF LISTENING COMPREHENSION




Listening Comprehension

Answers correctly three or more questions

Comprehension Learner responds correctly to questions asked about the story read to him/her

6.2 The Achievement of P 1, P 2 and P 3 Learners in Listening ComprehensionThe percentages of P 1, P 2 and P 3 learners who answered correctly at least 3 questions in Listening Comprehension in Leb-Acoli and English is presented in Figure 6.01.

FIGURE 6.01: LEARNERS ABLE TO ANSWER AT LEAST 3 LISTENING COMPREHENSION QUESTIONS IN LEB-ACOLI AND ENGLISH

85.4%92.6% 95.9%92.0%

1.2% 5.7%20.4%

3.6%

Baseline Midline

P12016

P22017

P32017

P22016

P12016

P22017

P32017

P22016

Leb-Acoli English

Figure 6..0.2

The proportion of learners who were able to answer at least 3 out of listening comprehension questions in Leb-Acoli and English was higher at midline than at baseline.

At midline the proportion of P 2 learners who were able to answer at least 3 out of 4 listening comprehension questions was higher in Leb-Acoli than English.

The proportion of P 2 learners at midline who were able to answer at least 3 out of 4 listening comprehension questions in Leb-Acoli and English was higher than that of P 2 learners at baseline.


CHAPTER 7ORAL PASSAGE READING AND COMPREHENSION

7.1 IntroductionThis chapter presents the achievement of P 1, P 2 and P 3 learners in fluency and reading comprehension. A testee was presented with a stimulus (a sheet of paper) with a passage in Leb-Acoli or English, depending on whether the assessment was in Leb-Acoli or English. The testee was expected to read the passage aloud, quickly and carefully in 60 seconds. After reading, the testee was asked some oral questions about the passage up to where s/he had stopped.

TABLE 7.01: MEASURE OF FLUENCY AND READING COMPREHENSION




Fluency Number of words read per minute

Reading with speed Reads the passage with speed and accuracy

Reading Comprehension

Answering correctly one question out of three

Reading the passage with understanding

Answers correctly the questions asked about the passage

7.2 The Achievement of P 1, P 2 and P 3 Learners in Fluency (1 or more words per minute)The percentages of P 1, P 2 and P 3 learners who could read correctly one or more words per minute in Leb-Acoli and English are presented in Figure 7.01.

FIGURE 7.01: LEARNERS ABLE TO READ CORRECTLY ONE OR MORE WORDS PER MINUTE

Figure 7.0.2

7.8%17.7% 14.0%

Baseline Midline

P12016

20.7%

55.4%

P22017

47.5%

P32017

28.6%

P22016

P12016

P22017

P32017

P22016

Leb-Acoli English

43.2%

The proportion of learners who were able to read correctly one or more words per minute in Leb-Acoli and English was higher at midline than at baseline.

At midline, the proportion of P 2 learners who were able to read correctly one or more words per minute was higher in Leb-Acoli than English.

The proportion of P 2 learners at midline who were able to read correctly one or more words per minute in Leb-Acoli and English was higher than that of P 2 learners at baseline.


7.3 The Achievement of P 1, P 2 and P 3 Learners in Fluency (20 or more words per minute)The percentages of P 1, P 2 and P 3 learners who could read correctly 20 or more words per minute in Leb-Acoli and English are presented in Figure 7.02.

FIGURE 7.02: LEARNERS ABLE TO READ CORRECTLY 20 OR MORE WORDS PER MINUTE

Figure 7.13

0.2% 0.5% 0.5%

Baseline Midline

P12016

0.3% 0.2%P2

2017

0.7%

P32017

0.1%P2

2016P1

2016P2

2017P3

2017P2

2016

Leb-Acoli English

7.4%

The proportion of learners who were able to read correctly 20 or more words per minute in Leb-Acoli and English was higher at midline than at baseline.

At midline the proportion of P 2 learners who were able to read correctly 20 or more words per minute was higher in English than in Leb-Acoli.

The proportion of P 2 learners at midline who were able to read correctly 20 or more words per minute in Leb-Acoli was higher than that of P 2 learners at baseline.

7.4 The Achievement of P 1, P 2 and P 3 Learners in Reading ComprehensionThe percentages of P 1, P 2 and P 3 learners who answered correctly one or more comprehension questions in Leb-Acoli and English are presented in Figure 7.03.

FIGURE 7.03: LEARNERS ABLE TO ANSWER CORRECTLY ONE OR MORE COMPREHENSION QUESTIONS

0.0% 1.2% 0.7%

Baseline Midline

P12016

0.5% 5.7%

P22017

8.6%

P32017

0.5%

P22016

P12016

P22017

P32017

P22016

Leb-Acoli English

7.6%

The proportion of learners who were able to answer correctly one or more comprehension questions in Leb-Acoli and English was higher at midline than at baseline.

At midline the proportion of P 2 learners who were able to answer correctly one or more comprehension questions was higher in Leb-Acoli than English.

The proportion of P 2 learners at midline who were able to answer correctly one or more comprehension questions in Leb-Acoli and English was higher than that of P 2 learners at baseline.


CHAPTER 8LEVEL OF ADHERENCE OF TEACHERS TO THE EGR MODEL8.1 IntroductionThis chapter is a presentation of the level of adherence of the teachers to the EGR model during the reading lessons. The EGR model entails the implementation of the 5Ts: Textbooks in the hands of the learners, Teaching in the mother tongue, Testing, Teaching techniques and Time on task. The P 2 teachers of 171 of the sampled schools were observed while conducting reading lessons with P 2 learners. The findings are discussed in the next section (8.2)

8.2 The Level of Adherence of Teachers to the 5TsOut of the 5Ts, time on task was not measured. The level of adherence of teachers to the 4Ts is shown in table 8.01.

TABLE 8.01: LEVEL OF ADHERENCE (PERCENTAGE) TO THE 4TS OF TEACHING READING

4TS INDICATOR BASELINE MIDLINEn % n %

Text books Text books/printed material in the hands of the learners 168 11.2 171 25.7

Tongue Teachers taught Reading lessons in the mother tongue 168 69.9 171 88.3

Testing Teachers have current assessment records of learners 168 49.7 171 40.4

Teaching Techniques

Teachers guide learners to:Beat the word (clap, beat, tap, or stamp the syllables of the word)

168 60.4 171 14.6

Blend letter to form syllables and words 168 38.7 171 21.6

Write a letter pattern in the air (or traces on floor or desk) 168 47.8 171 9.4

Read (individually) words from printed materials or book 168 51.0 171 60.8

At midline, for almost 9 in 10 lessons observed, the P 2 teachers taught reading lessons in the mother tongue. For 3 in 10 of the lessons observed, pupils had textbooks/printed materials in their hands (i.e. 1:1 pupil to text book ratio).

The proportion of learners who were able to read individually from printed materials or books was higher at midline than at baseline.


CHAPTER 9THE ROLES OF HEADTEACHERS AND TEACHERS IN PROMOTING EARLY GRADE READING

9.0 IntroductionThis chapter is a presentation of: the level of perceptions of headteachers about their roles in promoting Early Grade Reading; level of supervision of teachers during reading lessons; rate of supervision of teachers by different stakeholders during reading lessons and the percentage of teachers who have ever been trained to teach reading.

9.1 Head teachers’ Role in Early Grade Reading Head teachers in the EGRA sampled schools were interviewed about perceptions of their roles in promoting Early Grade Reading. Figure 9.01 shows the percentage of head teachers by what they perceived to be their role in promoting EGR at midline and baseline.

FIGURE 9.01: HEADTEACHERS’ PERCEPTION ABOUT THEIR ROLES IN PROMOTING EGR (%) Figure 9.0.1

Baseline Midline

Supervisor

89.9% 94.7%

37.8%36.0%

Logisticalsupport

Sensitization Leadership Coordination Trainingteachers

Others

32.0%30.1%

50.2%43.8%

82.7%69.9%

Roles

20.3% 17.3%10.0%8.9%

Generally, the Headteacher perception about their roles in promoting early grade reading improved at midline in contrast to the status at baseline. The critical areas of involvement and / or engagement in EGR, were also those cited as the key roles according to Headteachers’ reported perceptions. Most head teachers at midline viewed supervisory, logistical support and sensitization as their main roles in promoting Early Grade Reading. This was the same trend at baseline.

Calling for Continuous Professional Development seminars (CPDs), going to class to teach EGR in the absence of teachers, and emphasizing the use of local language as the medium of instruction among others were the other roles performed by head teachers in promoting early grade reading.


9.2 Teachers supervised during Reading lessonsAt midline, teachers were interviewed on whether they had ever been supervised during the reading lessons. The percentage distribution of teachers by ever having been supervised during the reading lesson at both midline and baseline is presented in figure 9.02.

FIGURE 9.02: PERCENTAGE DISTRIBUTION OF TEACHERS, WHO HAVE EVER OR NEVER BEEN SUPERVISED DURING READING LESSONSFigure 9.0.2

86.8%91.9%

13.2%8.1%

Yes No

Baseline Midline

At midline, 9 in 10 teachers reported that they had ever been supervised during the reading lessons.

A total of 320 teachers mentioned the category of officers who supervised them during the reading lessons. The percentage distribution of teachers by who supervised them during the reading lessons is shown in Table 9.01.

FIGURE 9.03: PERCENTAGE DISTRIBUTION OF TEACHERS, WHO HAVE EVER OR NEVER BEEN TRAINED TO TEACH READING

CATEGORY OF SUPERVISOR BASELINE (%) MIDLINE(%)

Headteachers 84.4 85.6

Centre Coordinating Tutors 49.3 37.5

Deputy Headteachers/Directors of Studies / Heads of Department 24.2 26.6

Inspectors of School 11.2 15.6

Ministry officials 3.8 2.5

School Management Committee / Sub-county staff 2.1 2.5

Associate Assessors 1.8 2.8

NGOs (DED, Action Aid, VSO, etc.) 0.9 0.6

There was an increase in the proportion of teachers who were supervised by headteachers at midline than at baseline. Similarly, the proportion of teachers who were supervised by deputy headteachers, Inspectors of Schools, School Management Committees and Associate Assessors increased at midline compared to that at baseline. It is worth noting that the proportion of teachers who were supervised by coordinating centre tutors dropped at midline in contrast to baseline.


9.3 Teachers having ever been trained to teach Reading Teachers were asked whether they ever attended training on how to teach reading at an early grade. The percentage distribution of those who have been trained is shown in figure 9.03.

FIGURE 9.03: PERCENTAGE DISTRIBUTION OF TEACHERS, WHO HAVE EVER OR NEVER BEEN TRAINED TO TEACH READINGFigure 9.0.3

92.2% 96.3%

7.8% 3.8%

Yes No

Baseline Midline

The proportion of teachers who reported that they were trained to teach reading was higher at midline than at baseline.

Apart from MoES/TIET, a number of other organizations have trained teachers to teach reading. They include RTI/SHRP, Mango Tree, Aga-Khan and the District Education Office. Many teachers (95.9%) reported that they applied what they learnt during their EGR training.


RECOMMENDATIONSTABLE 10.0 RECOMMENDATIONS AND RESPONSIBILITY CENTRE

SN RECOMMENDATION RESPONSIBILITY CENTRERoll out the EGR Model to all districts of the country MoES, Basic Education

UNEB take the lead in training of PTC tutors in assessment. TIET/UNEB

Closely monitor the implementation of the model Basic Education /DES/DEOs/DISs/ Headteachers/SMCs/CCTs

Strengthen regular practice in oral reading: provide more readers, integrate reading in and outside school activities.

Headteachers, Teachers / Parents

Provide continuous refresher courses to teachers and district officials. TIET, CCTs

Investigate the factors that may contribute to increased/improved support and sustainability of EGR

UNEB research department

Advocate for a vote in the budget to sustain and conduct EGRA annually. MoES (Basic Education)

Uganda National Examinations Board

Plot 35, Martyrs Way, Ntinda. | P.O. Box 7066, Kampala, Uganda. Tel: +256 417 773 100 /+256 414 289 397 | Email: [email protected] | Website: www.uneb.ac.ug

the achievement of learners in early grade reading in leb ......leb-acoli and english was higher...

Documents