edoren - nigeria education · technical report, volume i results and discussions. edoren, abuja....

EDORENEducation Data, Research and Evaluation in Nigeria

Final Baseline Technical Report Volume I: Results And Discussion

Impact Evaluation of Output 1: In-Service Training

Teacher Development Programme (TDP)

Sourovi De, Gunilla Pettersson, Robert Morris and Stuart Cameron

ImpactEvaluationofTDP’sIn-ServiceTeacherTrainingOutput:FinalBaselineTechnicalReport,VolumeI,ResultsandDiscussion

©EDOREN ii

Acknowledgments

SincerethanksareduetotheUKDepartment for InternationalDevelopment (DFID),particularlyEducation Adviser Esohe Eigbike and Laura Brannelly, Results Adviser Stuart Martindale anderstwhile Senior Education Adviser Emily Oldmeadow, for useful discussions and advice on themixed-methodsimpactevaluationdesign,andfeedbackonearlierdraftsofthisreport.

Thanksalso go to theManagingAgent,CambridgeEducation:particularlyBukolaOyinloye (TDPResultsandEvidenceTechnicalLead);NguyanFeese(TDPNationalProgrammeManager);MourieNishad(FormerTDPDeputyProgrammeManager);CelineThomas(TDPCommercialProgrammeManager); Abdurrahman Umar (TDP Pre-service Technical Lead); Robert McCormick (SeniorEvaluation Specialist); David Johnson (Instrument Development Specialist); John Shotton (Sub-divisionalDirector,MottMacDonald)andRonTuck(SeniorEducationAdviser,MottMacDonald).Special thanks are due to the TDP state in-service coordinators for their assistance duringfieldwork: Habeeb Saleh and Fati Abubakar (Jigawa); Hussaina Hassan and Mark Toscanini(Katsina);andKalliKori,MuhammedRajiSheriffandChineduMbonu(Zamfara).

Several members of Education Data, Research and Evaluation in Nigeria (EDOREN) and OxfordPolicy Management (OPM) contributed tirelessly at various stages of the quantitative andqualitative baseline survey design, implementation and analysis. In no specific order, they are:StuartCameron(Quantitative InstrumentDevelopmentLead);DavidMegill (SamplingSpecialist);RachelOuthred (LearningMetricsAdviser);AdrianBeavis (InstrumentDevelopmentAdviserandPeer Reviewer); Sourovi De (Project Manager and Education/Quantitative Specialist); GunillaPettersson (Education Lead Quantitative Analysis); Anna Duncan-Grant (Project Officer); PaulJasper (Quantitative Methods Consultant); Ian MacAuslan (EDOREN Evaluation Lead and PeerReviewer); Alex Hurrell (Impact Evaluation Adviser);Matthew Powell (Sampling Adviser); Peter-Sam Hill (Education Consultant); Girija Bahety (Quantitative Researcher); Robert Morris(Qualitative Researcher); Allan Findlay (EMIS Specialist); Terry Allsop andMonazza Aslam (PeerReviewers);SaltanatRasulova(QualitativeResearchMethodsAdviser);OladeleAkogun(EDORENCountry Director); Florian Friedrich (EDOREN Project Manager); Ijeoma Iroaganachi (EDORENProjectAdministrator);NandiChetden(EDORENProgrammeOfficer);JohnOde(EDORENProjectDriver);DanHooton,JamesOfferandAyoSangobanwo(SpearfishRiskManagers).

Thequantitativebaselinesurvey fieldworkwas implementedbythesurveyteamatOPMAbuja.Thanks and appreciation go to: Ifeatu Nnodu (Survey Manager); Charles Umar (FieldworkManager);SalawuSaheed(DeputyFieldworkManager);BabatundeAkanoandJusteNtiema(DataManagers);BusayoandJoshuaMoriyonu,andOkpeBartholomew(DataAssistants);andBukunolaAdekolu and Ibukun Ijaopo (Survey Administrators). Special thanks are due to the six statecoordinatorsforthequantitativebaselinesurvey:MusaAbubakarandTijaniGarba(Jigawa);FahadLawalandTimothyOyetunde(Katsina);andDanielAyalaandDawoodaKognet(Zamfara).Underchallenging conditions, they supervised the fieldwork in 330 primary schools in 42 LocalGovernment Authorities (LGAs) across the three states. The quantitative data collection wassuccessfullycompletedthankstotheimmensehardworkofthedatacollectorswhowerekindlyseconded fromtheir jobsat various stateeducationboards for thedurationof the trainingandfieldwork. The qualitative baseline survey fieldwork was implemented by the following sevenresearchers, who worked tirelessly in the field: Charles Umar, Dawooda Kognet, Fahad Lawal,NafinatuAbdullahi,SalawuSaheed,Shafa'atuMusa,andAhmedAbdulhamid.ThanksarealsoduetoJamilaSaliandFahadLawalforcarryingouttheunenviabletaskoftranscribingcountlesshoursofinterviewsandtranslatingthemfromHausaintoEnglish.


©EDOREN iii

Education secretaries and teacher facilitators (TFs) at various Local Government EducationAuthorities(LGEAs),aswellasstaffatthethreeStateUniversalBasicEducationBoards(SUBEBs),provided invaluable field advice, hospitality and support to the survey teams, thus ensuringsmoothprogressandsafetyoftheteams.Finally,sinceregratitudeisextendedtotheteamdriversMukhtar,AhmedandNasiruforsafelyconveyingtheresearchersduringtheirwork.

Needlesstosay,EDORENthanks,andextendswarmregardsto,theheadteachers,teachers,andpupilsfortheirtimeandpatiencewhileparticipatinginthisimpactevaluationbaselineresearch.

Recommendedcitation:De,S.,Pettersson,G.,Morris,R.,andCameron,S.(2016,June).TeacherDevelopmentProgramme(TDP)ImpactEvaluationofOutput1:In-ServiceTraining,FinalBaselineTechnicalReport,VolumeIResultsandDiscussions.EDOREN,Abuja.

EDOREN No.2,MafemiCrescent Tel +2348037220460EducationData,Research&EvaluationinNigeria Utakodistrict Email [email protected] Abuja,Nigeria Website www.edoren.org


©EDOREN iv

Executivesummary

Background

TheTeacherDevelopmentProgramme(TDP)isasix-year(2013–19)DepartmentforInternationalDevelopment(DFID) fundededucationprogrammeseekingto improvethequalityofteaching inprimary and junior secondary schools (JSSs) and in Colleges of Education (CoEs) in six states innorthernNigeria.TDPisbeing implementedbyMottMacDonaldand isoperating its firstphase,since 2014 in Jigawa, Zamfara and Katsina, with plans to extend it to Kano, Kaduna and Nigerstatesinlate2016.

This reportpresents the results fromamixed-methodsbaselinesurveyusingquantitativeand

qualitative researchmethods, to evaluate TDP’s in-service teacher training component in the

threePhase1states.Thisbaselinereportisorganisedintotwovolumes.VolumeI(thisreport)isintendedasastandalonereport,whichpresentsanoverviewoftheprogrammeandevaluation,and the baseline results for the programme’s treatment and control areas. It is designed to beaccessibletoallreaders.VolumeIIcoversthetechnicalandmethodologicaldetailsunderpinningthis impact evaluation, including further supplementary analyses not appropriate for Volume I,and is intended for those interested in methods, detailed statistical results and qualitativeaccountsforeachcasestudyschool.

TheobjectiveofTDP’sin-serviceactivities,whichisthelargestcomponentoftheprogramme,istoprovideschool-based,cost-effectivetrainingto62,000teachersby2019through:

• school-based interventions: collectively termed ‘New Classroom Activities,’ whichcompriseteachertraining,materialsforpupilsandteachers,andteachersupport;

• ‘trainer in the pocket’: which provides access to audio-visual resources through a basicmobilephone(perteacher)andtwomobileamplifier-radios(perschool)forplayingaudio-visualmaterialsintheclassroom;and

• continuoussupport:provisionofongoingsupporttoteachersthroughmultiplelayersandmechanisms(initialorientationworkshops,clustermeetings,classroomobservationduringschoolsupportvisitsbytrainers,andaudio-visualandprintedself-studymaterials),insteadof one-off training. The supportmechanism for continuous professional development isexpected to be institutionalised within schools (through head teacher leadership andmanagement training and peer support among teachers) and each of the participatingstates’educationsystems.

Thedesignof the in-service trainingcomponent is informedbya theoryof change (TOC) that

articulates how in-service teacher training activities can result in the desired impact

(improvements in pupil learning levels). Pupils can be expected to learnmore when they aretaughtbyeffectiveteacherswhoareskilledandknowledgeable,both intermsofpedagogyandsubject knowledge. TDP seeks to create effective teachers by combining the delivery ofpedagogicaltrainingalongsidethepromotionofasupportiveteachingenvironmentthroughheadteacher support, mentoring and supervision, peer interaction, visiting trainers, and access tolearningmaterials.

MoreonthedesignofTDPanditsTOC

• OverviewofTDP’srolewithinDFIDNigeria’seducationportfolio:Section2.1inVolumeI

• OverviewofTDPandTOCforin-serviceteachertraining:Figure2,Section2.2inVolumeI;AnnexB(InterventionFactsheet)inVolumeII.


©EDOREN v

Baselinesurveyobjectives

Thebaselinesurveycoversarangeofquantitativeandqualitativeresultsattheteacher,head

teacher,schoolandpupillevels.Itsobjectivesare:

• toestablishbaselinelevelsofteachereffectivenessandpupillearningbeforethestartofTDP’sin-serviceteachertrainingactivities,andespeciallycomparabilitybetweentheprogramme’streatmentandcontrolgroups,whichwillbeusedtoevaluateprogrammeimpactatthefollow-uproundsofdatacollectionin2018;

• to provide baseline results to help informwhether the coreassumptions underlying

theTOCofTDP’sin-serviceoutputholdinreality,thusofferingevidenceforpotentialadjustments to TDP’s design and implementation as deemed appropriate by theprogrammeandDFIDNigeria;and

• to provide an assessment of the status of pupil learning levels and teachereffectiveness in public primary schools in some of the most educationally

disadvantagedregionsinNigeriatothegovernmentandotherstakeholders.

Evaluationdesign

This baseline report forms part of amixed-methods theory-based evaluation thatwill enable

explorationof thepathways throughwhichanyobserved changesare likely tohaveoccurred

alongvariouslinksintheprogramme’sTOC.

The quantitative study design is based on a constrained randomisation of treatment (i.e.

participation inTDP in-service teacher training)andcontrolclustersofschools.Randomisationwillallowdifferencesinoutcomesbetweenthetreatmentandcontrolgroupsatfollow-uproundsof this evaluation to be attributed to the TDP in-service teacher training. In total, the intendedsample size for the quantitative baseline survey across the three stateswas 336 schools, 1,344head teachers and teachers, and 2,688 Grade 3 pupils. The survey administered five differentinstruments covering head teacher and teacher interviews, classroom observations, a Grade 2assessment(administeredtoGrade3pupils)ofEnglish literacy,numeracyandscientific literacy,and a TeacherDevelopmentNeeds Assessment (TDNA). The same teachers and head teachers,andpupilswhoweresurveyedatthebaselinewillbesurveyedagainattheendlineinJune2018.

TheoverallquantitativebaselineresultsforJigawa,KatsinaandZamfara,combined,showthat

therandomisationapproachwassuccessful, i.e.onaverage,thetreatmentandcontrolgroups

arebalancedintheircharacteristics.Thusthecontrolgroupmayserveasavalidcounterfactualforthetreatmentschools(andteacherandpupils) formeasuringprogrammeimpactonthekeyoutcomesofinterestattheendline(changesinpupillearninglevelsandteachereffectiveness).

The baseline qualitative fieldwork was carried out in nine schools, sampled from the

quantitativesurveyschoolsusingstratifiedpurposivesampling.Inparticular,withineachofthethree states, the 56 treatment schools were stratified into three categories according to theschool-level average for teachers’baseline scoreon the teacher subject knowledgeassessment,TDNA:top10%ofschools(‘high-performing’),middle10%(‘typicalschools’),andbottom10%ofschools (‘low-performing’).One school fromeach stratum ineach statewas selected (a totalofnine schools) for the qualitative sample. Fieldwork consisted of observations, key informantinterviews (KIIs) and focus group discussions (FGDs) with head teachers, teachers, pupils, localgovernment and state basic education officials and TDP teacher trainers and programme staff.While the quantitative survey established baseline levels of pupil learning and teacher


©EDOREN vi

effectiveness, the qualitative survey made it possible to assess the strength of assumptionsunderpinningtheprogrammeTOC,totestwhethertheyholdinpractice,andtoexplorepathwaysthroughwhichanyobserved changesare likely tooccur; aswell asdevelopingnewhypothesesandexploringunexpectedimpactsatendline.

Generalisabilityofsurveyresults

The findings of the baseline quantitative survey are representative only of the cluster of

treatment and control schoolswhere at least one of the TDP-selected (treatment) or control

teachersteachesEnglish,mathsorsciencetoGrade3pupils.Resultsarenotrepresentativeof

the three Phase 1 TDP statesmore broadly. This is because of the purposive selection of TDPclustersbyStateUniversalBasicEducationBoards(SUBEBs) ineachLocalGovernmentAuthority(LGA),ratherthanarandomselectionfromacomprehensivelistofpotentialTDPclustersineachstate.

Thequalitativefindingsarenotdesignedtoproduceresultsthataregeneralisableinrespectof

allschoolsinthethreestates.Thequalitativeresearchcomponentwasbasedonarelativelysmallsampleofnineschoolsandsamplingforthiscomponentwaspurposive,withtheaimofincludingschools with particular characteristics, as described above. Researchers investigated context inordertoconstructanargumentthatafindinginonesettingwaslikelytoapplyinanother.Theriskofvisitingatypical schoolsandgainingan incorrector incompleteunderstandingof therelevantprocessesremains,butissomewhatmitigatedbyvisitingschoolsindifferentstatesandcontexts,andbypayingcloseattentiontowaysinwhichthecontextofeachschoolmaybeatypical.

MoreontheevaluationdesignoftheTDPquantitativebaselinesurvey

• Overviewofmixed-methodsapproach:Section2,VolumeII

• Overviewofimpactevaluationdesignforquantitativesurvey,samplingstrategy,surveyinstruments,externalvalidityofresults,andpossibleriskstotheimpactevaluation:Section3,VolumeII

• Overviewof impact evaluationdesign for qualitative survey, sampling strategy, data collectiontools,analysisandlimitations:Section4,VolumeII

Results:Pupillearninglevelsandexperienceofschooling

Analysisoflearningoutcomeswasbasedonitemresponsetheory(IRT)usingRaschmodelling,which sorts pupils into three performance levels, each described by a set of competenciesexpectedbeforeGrade1,at theendofGrade1andat theendofGrade2.Thesampledpupils(eightrandomlyselectedineachtreatmentandcontrolschool)wereassessedwhentheyhadjuststartedGrade3basedontheGrade2Englishliteracy,numeracyandscientificliteracycurriculum–henceitisexpectedthattheywoulddemonstrateskillscorrespondingtothoseexpectedattheendofGrade2.

Englishliteracy

ThebaselinesurveyfoundthatEnglishliteracyskillsarepooramongpupilsintheTDPtreatment

and control schools.Only about 3% of pupils in Grade 3 were able to demonstrate the basicEnglishliteracyskillsexpectedofthembytheendofGrade2,and36%displayedemergentliteracyassociatedwiththecompletionoftheGrade1curriculum.Threeoutoffivestudents inGrade3


©EDOREN vii

wereatleasttwofullgradesbehind,indicatingthattheyhadnotdemonstratedprogressbeyondthelevelexpectedofpre-schoolchildren.

Numeracy

Thevastmajorityofpupilswereconsiderablybehindcurriculumexpectationsinnumeracy.Thebaselinesurveyfoundthatonly6%ofGrade3pupilsdemonstratedbasicnumeracyskillsexpectedattheendofGrade2and15%demonstratedemergentnumeracyassociatedwithcompletionofGrade1.Almostfouroutoffivepupilswerebehindbyatleasttwofullgrades,withgirlsandpupilsfrom poorer households significantly more likely to demonstrate lower levels of numeracy ascomparedtotheircounterparts.

Scientificliteracy

Pupils’performanceinthescientificliteracyassessmentwasalsopoorerthanexpectedfortheir

levelofschooling.About15%ofGrade3pupilswereabletoexplaineverydayphenomenawithunderstandingasexpectedat theendofGrade2,and67%wereable tounderstand these toalevelexpectedatGrade1.Theremaining18%ofpupilshadacquiredscientificliteracyskillsonlyadequate to recognisebasicphysicalpropertiesofeveryday items, andwerebehindbyat leasttwofullgrades.

Pupils’overallexperienceofschoolingandlearning

In all three subjects, girls and pupils from the poorest household wealth quintile performed

significantly worse than boys and pupils from the richest household wealth quintile,respectively.

Qualitative accounts frompupils depict a vicious cycle linkinghouseholdpoverty toexclusion

fromlearning inschoolandbroadersocialexclusion,corroboratedbyquantitativeresultsthat

suggest that it is differences in pupils’ own family backgrounds and community-level

characteristicsthatexplainthemajorityofvariationinpupillearning.

Teachersoftenallegedthatlearninglevelswerelowbecausepupilsdonotattendschoolregularlybecause of (paid or unpaid) work, or because their parents simultaneously enrolled them inIslamiceducationlessonsoutsidetheschool.

AlthoughprimaryschoolingisfreeinNigeria,schoolssometimeschargefeesforexamsorParent–TeacherAssociations(PTAs)chargesortowardsthecostofuniforms.Thereare,therefore,indirectschoolingcostsevenwhenthedirectcostsofschoolingarepresumablyminimalorevenfree–thisplacespoorerpupilsatadisadvantage.Pupils’nutritionalstatusislikelytoaffecttheirlearninginschool –many pupils reported coming to school hungry or with very small amounts of pocketmoneytobuyfoodduringtheday.

Mostpupils’accountsrevealedanacceptanceofthefactthatteachersoftendidnotcometoclassontime,oratall.Pupilsinschoolswithpoorinfrastructurewereconsciousoftheseproblems–ifgrantedmagicalpowers,mostpupilssaidtheywouldaddnewblocks,repairtheclassrooms,addconcretefloorsinschoolsthatdidnothavethem,oraddagateorwall.


©EDOREN viii

Moreonthepupillearningoutcomes

• Overviewofmethodofassessmentofpupils’learningoutcomes:Section3,VolumeII

• Overviewofresultsofassessmentofpupils’Englishliteracy:Section3.1,VolumeI

• Overviewofresultsofassessmentofpupils’numeracy:Section3.2,VolumeI

• Overviewofresultsofassessmentofpupils’scientificliteracy:Section3.3,VolumeI

• Overviewofpupils’overallexperienceoflearningandschooling:Section3.4,VolumeI

Results:Teachersubjectknowledge

Inorder tomeasure teachers’ subjectknowledge, teacherswereaskedtowritemodelanswers,mark test submissions, demonstrate knowledge through comprehension exercises and preparepupilworksheetsforprimary-levelEnglishliteracy,numeracyandscientificliteracy.

English

Averysmallgroup(0.4%)ofteachershadsufficientsubjectknowledgetobeeffectiveteachers

in English. A further 4% were found to have near-sufficient subject knowledge and 42% hademergingsubjectknowledge.Theremaining53%ofteachershadlimitedprofessionalknowledgeoftheirsubjectandareunlikelytobeeffectiveintheclassroomwithoutextensiveandcontinuoustrainingandsupport.

Mathematics

Onlyabout8%ofteachershadsufficientsubjectknowledgeofmathstobeconsideredeffectivein the classroom,with an additional 33% and 43% demonstrating near-sufficient and emergingsubjectknowledge,respectively.Theremaining17%hadlimitedsubjectknowledge,meaningthattheyscoredlessthan25%whentestedontheirknowledgeoftopicscoveredbyaGrade4pupil.

Scienceandtechnology

Onlyoneoutofthe1,158teachersassessedfortheirsubjectknowledgedemonstratedsufficient

subject knowledge in science and technology, with an additional 4% and 33% demonstratingnear-sufficient and emerging subject knowledge respectively. Nearly two out of three teachershad limited subject knowledge,meaning they scored less than25%whenpreparingworksheetsbasedontheGrade4sciencecurriculum.

Perceptionsaroundweakteachersubjectknowledge

The qualitative results further demonstrated that teachers, head teachers, and Local

GovernmentEducationAuthority(LGEA)officialsshowedlimitedacceptancethatlackofsubject

knowledge was an issue – few respondents showed an understanding of the challengesencounteredbyteachersregardingtheirsubjectknowledgeandpossiblewaystoaddressthese.

Assessmentandmonitoringofpupils’academicprogress

Theaverageteacherwasunabletocorrecterrorsinpupils’workandproviderelevantfeedback,

or identify learningneedsof individualpupilsandmonitor theiracademicprogressover time.


©EDOREN ix

Over80%ofteachersdemonstratedonlylimitedabilityinthisarea,comparedtotheonly0.3%ofteachersthatdemonstratedsufficientknowledge.

Moreonteachers’subjectknowledgeandabilitytoassessandmonitorpupils’academicprogress• DetaileddescriptionoftheTDNAinstrument, includingtopicscoveredunderthethreesubjects

andvalidityoftheinstrument:Section3,VolumeII

• Overviewof resultsof teachers’ subject knowledgeof English,maths and science: Section4.1,VolumeI

• Overviewofstakeholders’perceptionsofteachers’subjectknowledge:Section4.1.4,VolumeI

• Overviewofteachers’abilitytomonitorandassesspupils’progress:Section4.2.4,VolumeI

Results:Teachers’pedagogicalskillsanduseofinstructionaltime

The average teacher involved pupils in pupil-centred teaching practices, which characterise

effective teachers and classroompractice, for about aquarterof the total lesson time. Theseeffective teaching practices included assisting individuals or groups, explaining how somethingworksorhowtodoacertaintask,askingopenquestionsorgivingopenresponses,movingaroundamongst pupils, demonstrating or displaying work, and using printed or improvised materials.Simplywritingorreadingfromtheblackboard,consideredtobeaneutralteachingpractice,tookup40%oftotallessontime.

There was amajor loss of instructional time due to shorter lesson length, which necessarily

limitsthepotentialforin-classlearningtotakeplace.Classroomobservationsfoundthat45%oflessonsweremorethanfiveminutesshorterthanastandard35-minutelesson.

Teachers relied heavily on text books and curriculum guides, especially in the absence offoundationalsubjectknowledge,ratherthanbeingabletodevisetheirownlessonplanstoreachaspecified learninggoal.Theydemonstrated limitedability toadapt the textbookmaterial to theconditions and constraints faced in the classroom. In particular, teachers did not demonstrateideas about how pupils learn and about how teachers can respond to different learning levelswithin the same classroom to ensure all pupils achieve at least a minimum level. In general,teachers’priorpedagogical trainingdoesnotseemtohaveadequatelyequippedthemforthe

realities they faced in the classroom, including large class sizes, multi-grade teaching, limitedresources,andlimitedabilityofparentstosupporttheirchildren.

Moreonteachers’subjectknowledgeandabilitytoassessandmonitorpupils’academicprogress

• Detaileddescriptionoftheclassroomobservationinstrument:Section3,VolumeII

• Overview of results of teachers’ pedagogical skills and instructional time: Sections 4.2.1 and4.2.2,VolumeI

• Overviewofteachers’perceptionsoftheirownpedagogicalskills:Section4.2.3,VolumeI

Results:Teachermotivationandabsenteeism

Thisbaselinereportfoundmixedevidenceinregardtoteachersbeingintrinsicallymotivatedtotrytoimprovetheirteaching,totakepartinlearningopportunitiesandtoapplynewknowledgewhen they get it. Teachers gave varied descriptions of their attitudes towards the teachingprofession, ranging from those who described teaching as an inherently noble profession that


©EDOREN x

advancessociety,tothosewhoacceptedteachingjobsbecausetheywereunabletofindanythingelse.Oneteacherexplicitlybecameateacherbecauseitallowedhimtocarryonasidebusiness.

Therewasstrongevidencethatteachersfeltdemotivatedduetoextrinsicfactors.Amongthesedemotivating factors, teachers predominantly talked about low, late and even lack of salarypayments,thelackoflearningresourcesandpoorinfrastructure,lackofpromotionorperceivedunfairness in promotions, arbitrary transfers, the poor state of infrastructure and teachingresources,over-crowdedclassrooms,irregularattendanceofchildren,and‘untalented’pupils.

The average daily teacher absenteeismover the previous five days, according to the schools’

records,was14%.Themostcommonreasonscitedby teachers for theirabsencewere:ownorfamily illness (58%); collecting salary (20%); and social/religious obligations (10%). However,researchersfoundthatteacherscombinedtheirteachingworkwithfarmingorsmallbusinesses

outside the school. Some teacherswere undergoing further training or education at the sametimeasteaching,takingthemawayfromtheirregularjobs.Inmostschools,researchersalsosawteachersarrivinglate.Classroom absenteeism, where teachers are present in the school but missing from the

classroom,wasacommonobservation in thequalitativesurvey.Onepossiblereasonfor this isteachers’specialisedtrainingintheirNigeriaCertificateinEducation(NCE)oftendoesnotmatchthesubjectthatisneededwhentheyarepostedtoaschool,andshortagesofteachersinaschoolcanariseinspecificsubjectareasevenwhenthereisnooverallshortageofteachersintheschool.

Results:Contextualfactorsthataffectqualityofteaching

Urban or semi-urban case study schools in the qualitative study were generally found to be

muchbetterequippedthanruralschoolsintermsofphysicalinfrastructure.

Head teachers frequently noted that their request for additional infrastructure, repairs, and

resourcesfromtheLGEA/SUBEBwentunanswered.Politicalconnectionswerehighlightedasanimportantdeterminantofhowwellequippedschoolswere.

Textbookswerealsonotroutinelydistributedamongpupils.Thiswassometimesbecausetherewereinsufficienttextbooks.Intherarecaseswhereschoolresourcesweresufficient,therewasevidence that thesewere not alwaysmanaged and allocated efficiently (for example, unusedteachingresourcesandtextbooksstoredinaheadteacher’soffice).

Thequantitativesurveyfoundthatabout88%ofheadteachersreportedthattheirschoolswere

inneedofmajorrepairs.SchoolsinTDP’streatmentandcontrolclustershaveinadequatephysicalinfrastructure. Schools varynotably in size,with anaverageof 655pupils but varying from142pupilsforthebottomdecileto1,500ormoreforthetopdecile.Thereareonaverage42pupilsperclassroom,and59pupilsperteacher,comparedtotheofficialpolicyof35pupilsperteacher.Onlyabout11%ofschoolshadanelectricitysupply.Inteacherinterviews,70%ofteachersnotedthattheirschool’sbuildingwasina‘poorcondition’,and33%saidtheyhaveinadequatematerialsto do their job properly. Inadequate classroom resources and poor school infrastructure wererankedamongtheworstconstraintsfacedbyteachers.


©EDOREN xi

Moreonteacherabsenteeismandmotivation,andcontextualfactors• Overview of current levels and drivers of teacher absenteeism from schools and classrooms:

Sections4.3.3and4.3.4,VolumeI

• Overviewofcurrentlevelsanddriversofteachermotivation:Sections4.3.1and4.3.2VolumeI

• Overviewofcontextualfactorsthataffectthequalityofteaching:Section4.4,VolumeI

Results:Schoolleadershipandmanagement

Effectiveschoolleadershipandmanagement(SLM)iscrucialforthesuccessofTDP’sin-service

training activities. One of the key outputs of the TDP’s in-service teacher training activities isenhancedSLM. Inaddition, it cannotbeassumedthat teacherswillbemotivated toadoptnewteachingpracticeswithoutappropriateleadershipandmanagementfromheadteachers.EffectiveSLM is therefore an important factor in determining whether TDP training leads to improvedsubjectandpedagogicalknowledge.

Headteachersarenotextrinsicallyincentivisedandinmanycasesnotintrinsicallymotivatedto

improve SLM or encourage teachers’ adoption of the teaching practices that TDP seeks to

promote.Thereisariskthatheadteacherswillnotexercisetheirabilitytoinfluencesuchchangewhere they encounter resistance tonewpractices andwhere theyperceive a risk of disruptingotherwisepositiverelationshipswithintheschool.Thiswouldmeanthatnewlyacquiredsubjectandpedagogicalknowledgeisnotorsuperficiallyappliedbyteachersinaclassroomcontext.

Head teachersaregenerallyunable toeffectivelyaddress thewidespreadproblemof teacher

classroomabsenteeism.Thissuggeststhat,evenifteachersattendTDPtrainingandimprovetheirsubject and pedagogical knowledge, there is a significant risk that teachers will not apply thisknowledge in a classroom context because they spend large parts of the school day outside ofclasses.

Headteachersremainatleastpartlyabletoinfluenceteachingtechniquesandtocorrectsubject

knowledgemistakesthroughlessonobservationsandfeedback.Thissuggeststhatheadteacherswill be able tomotivate andpersuade teachers that already attend lessons andhave improvedsubject and pedagogical knowledge to adopt the positive teaching practices that TDP seeks topromote.

MoreonSLM• Overviewofschoolleadership,managementandinfrastructure:SectionError!Referencesource

notfound.,VolumeI

Results:Qualityofprogrammeimplementation

Quality of implementation and robust delivery of outputs are vital to TDP’s success.A ‘lighttouch’ process evaluationwas undertaken as part of the baseline qualitative study,which tookplaceroughlysixmonthsaftercommencementofimplementationofin-servicetrainingactivities,tostudyvarious implementationprocessesandpracticaldynamics,and in turn toprovide initialimpressionsofhowimplementationofthein-serviceoutputisprogressing.

Anumberofchallengesrelatingtotheprogramme’sdeliverymodelweakenitsTOCandreduce

itschancesofmeetingoutcome-andimpact-levelgoals.Weakparticipationinclustermeetings


©EDOREN xii

duetofactorslikelanguagecomplexities,andbarrierssuchasgender,arepersistent.Thereisalsolimitedevidenceofpeer-to-peerlearningmaterialisinginpracticeduetolowteachermotivationandhighabsenteeism,leadingtolimitedtimedevotedtopeerinteraction.TDP’strainingmaterialstry to address the constraints faced by classroom teachers (e.g. large class sizes) and proposetechniques for effective teaching in themidst of these obstacles. However, observations in theearly stages of implementation reveal limited application of these techniques in classrooms, orsuperficialadoptionatbest.Confusionamongteachersandtrainersinregardtotheconceptsandterminologyusedbyvariousteachertrainingprogrammesiscommon.Whilemechanismsexistforrouting feedback on the training from teachers to developers of materials and programmemanagers, it is not clear whether these then prompt action and rectification. Finally, teacherfacilitators (TFs) have strong backgrounds in classroom teaching and school administration butencounter some of the same pedagogical limitations in their skills as are faced by teachersthemselves, and they face an immense workload in organising and delivering cluster meetings(alongsidedischargingtheirdayjobs,mostlyintheLGEAs).It is worth emphasising the point that teachers in northern Nigeria have incredibly tough

professional duties to discharge – they are expected to teach children where classroomssometimesdonotexist, textbooksareoftenscarce,andclass sizesare large.Teachers’ intrinsicmotivationtobebettereducators isoftenerodedbybroad-basedpovertyand lowsalaries,andlong delays in receiving them. This leads to per diems from cluster meetings becoming a keyincentiveforparticipationintraining.Anumberofthesechallengesappeartobewellbeyondthe

programme’sdirectambitofcontrol.Forexample,issuesoflowsalariesandlatepaymentrelateto federal and state level governance and public financial management issues and it is notstraightforwardforTDPtoinfluencethis,despitethesechallengeshavinganimmenseimpactonprogrammesuccess.

ImplicationsofbaselinefindingsforTDP

One of the primary objectives of this mixed-methods report is to provide baseline results forpotential adjustments to TDP’s design and implementation, as deemed appropriate by theprogrammeandDFIDNigeria.Thissectionhighlightstheimplicationsofthebaselinefindingsfortheprogramme.

Pupillearninglevels

PupillearninglevelsinEnglish,mathsandscienceareverylowinthetreatmentandcontrolareas.Qualitative accounts from pupils suggest a vicious cycle linking household poverty to exclusionfromlearninginschoolandbroadersocialexclusion.

Thispictureoflearninglevelsandpupils’experienceofschoolingatbaselineconfirmsthatthe

core objective of the programme of improving pupil learning levels is highly relevant in the

programme LGAs. Pupils’ own family backgrounds and community-level characteristics, whichappear to be the key drivers of low pupil learning, may be beyond the scope of a teacherdevelopment intervention,yetare likely to impingeontheeffectivenessofsuch interventions–thisisindeedasoberingfindingforaprogrammedrivenbyschool-basedinterventionsaloneandneedstobetakenintoaccountwhenassessingtheprobabilityofsuccess.Itmaybeimportanttoensurethatpupilsfrompoorerbackgroundsareconsideredasatargetgroupinthecoreobjective,aswellasgirls,andtoensurethat theteacher trainingconsidershowschoolscanaddress theirparticularlearningneedsinwaysthataremoresociallyinclusive.


©EDOREN xiii

Teachereffectiveness

This baseline survey investigatedwhether teachers aremotivated to attend school and lessonsregularly; whether they have sufficient subject knowledge and pedagogical skills; and whetherthey have access to sufficient infrastructure and materials. For a vast proportion of schoolsstudied as part of this baseline survey none of the assumptions hold up fully with strong

evidence.

Someofthe issuesraisedhereare likelytoremainbeyondthescopeofaprogrammesuchas

TDP.Forinstance,infrastructureproblemsandlargeclasssizesrequireactionatthefederal,state,andlocalgovernmentlevel,possiblycombinedwithactionstobuildthecapacityofSchool-BasedManagementCommittees(SBMCs)toholdschoolstoaccount.

ActionforTDPcould,however,focusonhowtheteacherdevelopmentmaterialsandactivities

cantakeintoaccounttherealityofthedifficultcontextsinwhichteacherswork,andthevery

limitedpedagogicalandsubjectknowledgetheycurrentlypossess.

Itisworthinvestigatingwhetherthereisspacepoliticallytopushforthegovernment’slanguage

policy–ofteachinginHausainGrades1–3andinEnglishthereafter–toactuallybeenforced,which would mean producing and using Hausa textbooks and assessments for early gradelearning. This would still be a radical change as teachers may not be familiar with textbookswritten in Hausa, but it would at least mean engaging with the reality of teachers’ currentunderstanding of English andwould be a recognition of the fact that 99% pupils in this surveyreportedHausaastheirhomelanguage.

SLM

ManyofthefactorsthatshapeaTDPschool’sSLM–suchasteacherrecruitmentpractices,andtheresponsivenessofLGEAs/SUBEBstoheadteachers’requests–arebeyondthescopeofTDP.

However,therearepotentiallessonsthatcanbelearnedfromthebaselineresearchtoinform

futureTDPactivitiesregardingSLM:

• Teacherclassroomattendance isawidespreadproblembut inmostcaseshead teachersalone do not have the means to resolve it. TDP could request that good classroomattendance be an additional criterion for selecting future beneficiaries of TDP training,therebymaximisingthechancesthatnewknowledgeisappliedinaclassroomcontext.

• Head teachers are rarely held to account and therefore are not incentivised to improveperformance.ThisfactmakesTDPmorereliantonheadteachers’ownmotivation,whichisalso lacking inmanycases.TDPcouldconsider includingactivitiesdesignedspecificallytomotivateheadteachers.

• Headteachersaresometimesunclearabouttheirrightsandresponsibilitiesinrelationtoother actors in the education system, particularly LGEAs/SUBEBs. TDP’s planned headteacher training activities could increase their awareness of these rights andresponsibilities and equip themwith the skills necessary to navigate such a constrainingsystem.

• AlargenumberofTFsareLGEAstaff.GiventhatmanyoftheconstraintsoneffectiveSLMarecausedbydynamicsatthesystemlevelTDPmaywishtoconsiderhowtheseTFscould


©EDOREN xiv

foster an honest dialogue between influential communitymembers, SUBEBs, LGEAs andschoolstocreateacommonunderstandingofthedriversofschoolperformance.

Qualityofprogrammeimplementation

As has been said, duality of implementation and robust delivery of outputs are vital to TDP’ssuccess.Anumberofchallengesrelatingtotheprogramme’sdeliverymodelweakenitsTOCandreduce itschancesofmeetingoutcome-and impact-levelgoals,eventhoughanumberofthesechallengesappear tobewellbeyondtheprogramme’sdirectambitofcontrol. It shouldalsobeborneinmindthatthisprocessassessmentwasundertakenintheearlystagesofimplementationanditisexpectedthatsomeoftheoperationalteethingissueswillbealleviatedastheprogrammematures.

However,therearesomechallengeswhichappeartobemorewithintheprogramme’sabilitytoaddressintheshortormediumterm.

• To begin with, teachers and their trainers would find it immensely helpful to receivematerialsinadequatequantitiesandontime,toavoidthemhavingtopayout-of-pockettomakephotocopies,buystationery,andsoon.

• Thedeliverymodelwillbeenhancedconsiderablyiffeedbackmechanismsfromteachersregardingthetrainingareroutedtothosewhocanaddressfeedbackmoreeffectively,andiffeedbackisfolloweduponwithactionsanddecisions.

• Theprogrammemightfurtherwanttoconsiderwhatvalueisbeingaddedbytheideaoftrainingsometeachersinaschoolwhileleavingoutothers,giventhediscussiononlimitedpeerexchangesamongselectedteachersandpossibilityofhostilitybetweenTDPandnon-TDPteacherswithinaschool.

• Finally, the lack of a direct emphasis on subject knowledge in the TDP’s training

curriculum is a concern, and something for TDP and DFID to consider and address.Ineffectiveness of training is often driven by limitations imposed by teachers’ ownabsorptive capacity for new materials and pedagogical techniques. In a context whereteachers’ own subject knowledge is grossly insufficient, more training on newerpedagogicaltechniqueswillonlybuildonaweakfoundationofsubjectknowledge,andwilllimit absorption and adoption of new pedagogical content, thus weakening chances forprogrammesuccess.

Plansforfollow-upquantitativeandqualitativeresearch

Therearetwoplannedfollow-upevaluationactivitiesinthepipeline.

Formativeprocessassessmenttoinformtheprogramme’sfutureimplementationandscale-up:betweenAprilandJuly2016aseriesofimplementation-focusedstudiescoveringoperationalandprocessquestionswillbeundertaken,to informingtheprogramme’sfuture implementationandespecially its scale-up (scheduled for late 2016). The TDP evaluation’s steering committee hasagreedonthefollowingthreequestionsforaprocessassessment:


©EDOREN xv

• Given the low levels of subject knowledge and pedagogical skills among teachers asestablishedbythebaselinesurvey,howcouldTDPmaketheteacherdevelopmentteams,teachertrainers,clustermeetingsandschoolsupportvisitsmoreeffective?

• TowhatextentdoTDP trained teachersunderstand the contentof theprint andaudio-visualtrainingmaterialstheyhavebeengiven?Towhatextentdotheyusethesematerialsbothinsideandoutsidetheclassroom?Howcantheprogrammemakethematerialsmoreusefulandmoreused–intermsofcontent,languageandusage?

• How could TDP improve school leadership (especially vis-à-vis management andgovernance)inTDPschools–intermsofensuringmoreeffectiveheadteachers,educationsecretaries,inspectors,qualityassuranceofficers,SBMCsandparents?

Endlinemixed-methodssurveys: Thesewillbeconducted in June2018asplanned, toallow formeasurementofanyprogrammeimpactonteachereffectivenessandonpupillearninglevels.Thebaseline panel of pupilswhowere at the beginning ofGrade 3 inOctober 2014will be on thevergeofsittingtheirprimaryschoolleavingexamsattheendofGrade6inJune2018.

MoreonimplicationsofthesurveyresultsforTDPandfutureevaluationandresearchactivities• Conclusionanddiscussionofbaselinesurveyresults,implicationsfortheprogramme,andfollow-

upquantitativeandqualitativeresearch:Section7,VolumeI


©EDOREN xvi

Tableofcontents

Acknowledgments ii

Executivesummary iv

Listofboxes,figuresandtables xviii

Listofabbreviations xx

1 Introduction 11.1 Objectivesofthebaselineanalysis 11.2 Structureofthisvolume 1

2 Programmebackground 32.1 TDP’srolewithinDFIDNigeria’sportfolioofeducationprogrammes 32.2 OverviewofTDP’sin-serviceteachertrainingoutput 52.3 Objectivesoftheimpactevaluation 82.4 Overviewoftheimpactevaluationdesign 92.5 Pointersonhowtointerprettheresults 14

3 Pupillearninglevels 163.1 PupillearninglevelsinEnglishliteracy 173.2 Pupillearninglevelsinnumeracy 193.3 Pupillearninglevelsinscientificliteracy 223.4 Pupils’experiencesofschoolingandlearning 25

4 Teachereffectiveness 304.1 Teachers’subjectknowledge 314.2 Pedagogicalskills 364.3 Teachermotivationandattendance 434.4 Contextualfactorsthatinfluencethequalityofteachingandlearning 50

5 SchoolLeadershipandManagement(SLM) 595.1 AssumptionsrelatingtoSLMintheTDPTOC 595.2 Headteachers’motivationandincentivestoidentify,incentiviseandinfluence

teacherbehaviour 605.3 Headteachers’abilitytoidentify,incentiviseandinfluenceteacherbehaviour 655.4 Managementofschoolinfrastructureandresources 69

6 TDP’sdeliverymodelforin-servicetraining 726.1 Scopeandrationale 726.2 Clustermeetingsandpeer-to-peerlearning 736.3 Contentandusageofmaterials 756.4 TFsareappropriatelyselectedandtrained,andhaveamanageableworkload 78

7 Conclusionanddiscussion 807.1 Pupillearninglevelsandoverallexperienceofschooling 807.2 Teachereffectiveness 81


©EDOREN xvii

7.3 SLM 837.4 TDP’simplementationmodelforin-servicetraining 847.5 Follow-upquantitativeandqualitativesurveys 86

Bibliography 87


©EDOREN xviii

Listofboxes,figuresandtables

Box1Keybackgrounddocumentsandcompanionreports..............................................................8Box2Evaluationquestions(atbaseline)aboutpupillearninglevels..............................................16Box3BaselinebackgroundcharacteristicsofGrade3pupils..........................................................16Box4State-levelresultsonpupillearninginEnglishliteracyforJigawa,KatsinaandZamfara......19Box5State-levelresultsonpupillearninginnumeracyforJigawa,KatsinaandZamfara..............21Box6State-levelresultsonpupillearninginscientificliteracyforJigawa,KatsinaandZamfara...24Box7Baselineevaluationquestionsaboutpupils’context,backgroundandattitudestowardsschoolingandlearning.....................................................................................................................25Box8BaselineresultsforteachersubjectknowledgeinJigawa,KatsinaandZamfara...................35Box9Lossofinstructionaltimeduetoshorterlessonlength.........................................................39Box10BaselineresultsforteacherpedagogyinJigawa,KatsinaandZamfara...............................40Box11BaselineresultsforteacherandheadteacherschoolabsenteeisminJigawa,KatsinaandZamfara............................................................................................................................................48

Figure1StatestobecoveredinTDP’sPhases1and2......................................................................4Figure2TDPTOCforOutput1:In-serviceteachertraining...............................................................7Figure3Quantitativeandqualitativedatacollectionatbaseline....................................................10Figure4Distributionofpupilsbyliteracyperformancelevels.........................................................18Figure5Distributionofpupilsbynumeracyperformancelevels....................................................20Figure6Distributionofpupilsbyscientificliteracyperformancelevels.........................................23Figure7Teachers’subjectknowledgebylevelofachievement......................................................34Figure8Lessonlengthisfrequently(much)shorterthanexpected................................................39Figure9Abilitytoassessandmonitorpupils’progress...................................................................42Figure10Overallteachermotivationscorebyqualification,genderandstate..............................44Figure11Teachermotivationscoresandteacherbackgroundcharacteristics...............................45Figure12Teachereffectivenessandpupillearninglevels:Simplebivariateregressionanalysis....58Figure13Frequencyofschoolinspections......................................................................................61Figure14SLM,basedonheadteacherself-reporteddata..............................................................66

Table1Overviewofinstrumentsandrespondents.........................................................................11Table2Instrumentsadministeredforeachparticipantgroup........................................................12Table3Summaryofintendedsamplesizesforthequantitativesurvey.........................................12Table4Schoolselectionprocess......................................................................................................13Table5Definitionsofstrengthofevidenceusedbyresearchers....................................................15Table6Englishliteracyassessmentframework...............................................................................17Table7Numeracyassessmentframework......................................................................................20Table8Scientificliteracyassessmentframework............................................................................22Table9Teacherbackgroundcharacteristics....................................................................................30Table10Evaluationquestionsonteachersubjectknowledge........................................................31Table11MappingofTDNAtopics,exercisesandquestions............................................................32Table12TDNAlevelsofachievement,descriptorsandscoreranges..............................................32Table13Teacherandheadteachersubjectknowledge:%ofcorrectanswers..............................33Table14Summaryofstrengthofevidenceunderpinningassumptionsrelatingtoteachers’subjectknowledge........................................................................................................................................36Table15Evaluationquestionsonpedagogy....................................................................................37Table16Teachers’positiveinteractionwithpupilsduringlessons(meanestimates)....................38


©EDOREN xix

Table17Summaryofstrengthofevidencerelatingtoteachers’pedagogicalskills.......................42Table18Evaluationquestionsonteachermotivation.....................................................................43Table19Teacherabsenteeism(meanestimates)............................................................................48Table20Summaryofstrengthofevidencerelatingtoteachers’motivationandattendance.......50Table21Evaluationquestionsonthecontextthatinfluencesthequalityofteachingandlearning..........................................................................................................................................................51Table22Schoolcharacteristics:Size,PTRandclasssize..................................................................52Table23Summaryofstrengthofevidencerelatingtocontextualfactorsinfluencingteachingandlearning............................................................................................................................................56Table24SummaryofassumptionstestedregardingSLM...............................................................60Table25Summaryofstrengthofevidenceunderpinningassumptionsrelatingtoheadteachers’motivationandincentives................................................................................................................64Table26Summaryofstrengthofevidenceunderpinningassumptionsrelatingtoheadteachers’ability................................................................................................................................................69Table27SummaryofstrengthofevidenceunderpinningassumptionsrelatingtoSLMofschoolinfrastructureandresources............................................................................................................71Table28Summaryofstrengthofevidenceunderpinningassumptionsrelatingtoclustermeetingsandpeer-to-peerlearning................................................................................................................75Table29Summaryofstrengthofevidenceunderpinningassumptionsrelatingtocontentandusageoftrainingmaterials...............................................................................................................77Table30Summaryofstrengthofevidenceunderpinningassumptionsrelatingtoappropriateselection,trainingandmotivationofTFs.........................................................................................79


©EDOREN xx

Listofabbreviations

CoE CollegeofEducation

DFID UKDepartmentforInternationalDevelopment

EDOREN EducationData,ResearchandEvaluationinNigeria

EiA EnglishinAction

ES EducationSecretary

ESSPIN EducationSectorSupportProgrammeinNigeria

FGD Focusgroupdiscussion

GEP3 Girls'EducationProject,Phase3

ICT Informationandcommunicationtechnology

IE ImpactEvaluation

IQTE IntegratedQuranic,Tsangayaeducation

IRT Itemresponsetheory

JICA JapanInternationalCooperationAgency

JSSs Juniorsecondaryschools

KIIs Keyinformantinterviews

LGA LocalGovernmentAuthority

LGEA LocalGovernmentEducationAuthority

MDE Minimumdetectableeffect

N Numberofobservations

NCCE NationalCommissionforCollegesofEducation

NCE NigeriaCertificateinEducation

NEI NorthernEducationInitiative

NERDC NigeriaEducationalResearchandDevelopmentCouncil

NGN NigerianNaira

NGO Non-governmentorganisation

OECD OrganisationforEconomicCo-operationandDevelopment


©EDOREN xxi

OPM OxfordPolicyManagement

PISA ProgrammeforInternationalStudentAssessment

PTA Parent–teacherassociation

PTR Pupil–teacherratio

RED DFID'sResearchandEvidenceDivision

SAVI StateAccountabilityandVoiceInitiative

SBMC School-BasedManagementCommittee

SLM Schoolleadershipandmanagement

SPARC StatePartnershipAccountabilityResponsibilityandCapability

SUBEB StateUniversalBasicEducationBoard

TDNA TeacherDevelopmentNeedsAssessment

TDP TeacherDevelopmentProgramme

TDT TeacherDevelopmentTeam

TF Teacherfacilitator

TOC Theoryofchange

UNICEF UnitedNationsChildren'sFund

USAID UnitedStatesAgencyforInternationalDevelopment


©EDOREN 1

1 Introduction

Thisbaselinereportpresentsquantitativeandqualitativeevidencefromtheimpactevaluationofthe in-service teacher training component of TDP in three northern Nigerian states: Jigawa,Katsina and Zamfara. It covers a range of results and discussion at the pupil, teacher, headteacher,schoolandeducationadministrationlevels.

This baseline report is organised into two volumes. Volume I presents an overview of theevaluation, and the baseline results for the programme’s treatment and control areas. It isdesignedtobeastandalonereportaccessibletoallreaders.AsupplementaryVolumeIIcoversthetechnical and methodological details underpinning this impact evaluation, and furthersupplementary analyses not appropriate for Volume I, and is intended for those interested inmethods, detailed statistical results, and the detailed qualitative background accounts for eachcasestudyschool.

1.1 Objectivesofthebaselineanalysis

Theoverallobjectivesofthebaselineanalysisare:

• toestablishbaselinelevelsofteachereffectivenessandpupil learningbeforethestartofTDP’s in-service teacher training activities – especially comparability between theprogramme’s treatmentandcontrolgroups–whichwillbeusedtoevaluateprogrammeimpactatthefollow-uproundsofdatacollectionin2018;

• toprovidebaseline results tohelp informwhether thecoreassumptionsunderlying the

TOC of TDP’s in-service output hold in reality, thus offering evidence for potentialadjustments to TDP’s design and implementation as deemed appropriate by theprogrammeandDFIDNigeria;and

• toprovideanassessmentofthestatusofpupillearninglevelsandteachereffectivenessinpublic primary schools in some of the most educationally disadvantaged regions in

Nigeriatothegovernmentandotherstakeholders.

1.2 Structureofthisvolume

This report (Volume I: Results and Discussion) constitutes presentation and discussion of theintegrated quantitative and qualitative baseline results for treatment LGAs of TDP’s in-servicetraining output. While the quantitative results provide evidence on pupil learning, teachereffectiveness and SLM at baseline, the qualitative research discusses whether the coreassumptions underlying the logic of TDP’s in-service TOC holds in practice. This report will besupplemented by a series of aggregate and state-level policy briefs on teacher capacity andperformance,andpupillearningintheTDPprogrammeLGAs.

Thisvolumeofthebaselinereportisstructuredasfollows.Section2providesanoverview,andadiscussionofthecontextof,theTDPprogramme.Italsodiscussestheobjectivesofthisbaselinesurvey and describes its audience. Section 2.4 describes the overall evaluation design, thequantitativemethodsunderlyingbaselineresults,thesamplingstrategy,andtheprocessesofdatacollection, cleaning and analysis. It also provides a summary of the survey instruments, a briefdiscussion of the representativeness of results, and possible risks to the impact evaluation.Sections3–6presentthemixed-methodsbaselineresultsforpupils,teachersandschools.Section


©EDOREN 2

7 concludes by outlining potential implications of the baseline results for TDP’s design andimplementationandthenextstagesofthisimpactevaluation.

Gender-neutral language has been used throughout this report to refer to teachers, pupils andparents;wherenecessary,foreaseofreading,thefemalenounandpronounhavebeenusedtorefertoallgenders.


©EDOREN 3

2 Programmebackground

TheTDPisasix-year(2013–19)DFID-fundededucationprogrammeseekingtoimprovethequalityof teaching inprimary schoolsand JSSs,andCoEs in six states innorthernNigeria.TDP isbeingimplemented byMottMacDonald and in the first phase,which started in 2014, is operating inJigawa,ZamfaraandKatsina,withplanstoextendittoKano,KadunaandNigerstatesinlate2016.

2.1 TDP’srolewithinDFIDNigeria’sportfolioofeducationprogrammes

TDParises fromthe recognition that ‘children inNigerianschoolsarenot learning’and that thequalityofteaching,whichisofcentralimportancetolearningachievement,isa‘seriousconcern’(DFID Nigeria 2012). Despite large investments in education in recent years, the quality ofteachers,teaching,andteachertrainingcontinuestobeofgraveconcernforNigeria(HumphreysandCrawfurd2014).1 Improvingpupils’ learningoutcomesisatthecentreofDFID’sapproachtoaid programming in education, which is based on a learning framework for improving pupillearning outcomes principally through ‘strong and accountable education systems’ and ‘goodteachersandgreatclassroompractices’(DFID2013).

Consequently,throughtheTDP,DFIDNigeriaseekstoimprovethequalityofteachinginprimaryschoolsandJSSs,and inCoEs, insomeofthemostdisadvantagedstatesofnorthernNigeria.AsshowninFigure1,theprogrammefocusesonsixnorthernstates:Jigawa,Kaduna,Kano,Katsina,Niger and Zamfara. The TDP is being implemented in Jigawa, Katsina and Zamfara during theprogramme’sfirstphase(whichstartedin2014),withtheremainingthreestatestobecoveredbythe programme’s second phase (late 2016), with the programme ending in 2019. Volume IIdiscussesthecontextualcharacteristicsoftheTDPstates(especiallythelevelsofpoverty,conflictandinsecurity,aswellastheprogramme’sgeographicalcoverage,intermsofLGAsinthePhase1states).

1A full reviewof the literatureon thestateofNigerianpublicprimaryschool teachers isbeyond thescopeandpurposeof thisbaselinereport,andhasbeendiscussedinEDOREN(2014).


©EDOREN 4

Figure1StatestobecoveredinTDP’sPhases1and2

TheobjectiveoftheTDPisto increasetheeffectivenessofteachers, inordertoultimatelyraisepupillearninglevels,by:

• improving in-service teacher training in the three core curriculum subjects of English,maths,andscienceofprimaryandJSSteachers(Output1:in-servicetraining);

• developingmoreeffectiveteachereducatorsofprimaryandJSSteachersbyreformingpre-serviceteachereducation(Output2:pre-serviceteachereducation);and

• strengthening evidence-based research to influence and inform policies on teachereffectiveness(Output3:researchandevidence).

TDP’sin-serviceteachertrainingcomponentisitsbiggestoutput,representingapproximately80%of the programme’s resources.2 Given that TDP’s ‘core business’ is teacher training,complementarities are expectedwith otherDFID-funded educationprogrammes inNigeria. TheEducationSectorSupportProgrammeinNigeria(ESSPIN)–whichoperatesinsixstates,includingJigawa (a TDP Phase 1 state) and Kano and Kaduna (TDP Phase 2 states) – focuses on schoolquality improvement,accountabilityandgovernance.Whilenotpartof itsprimaryfocus,ESSPINalso conducts training activities in the area of teacher professional development. The Girls’EducationProject,Phase3(GEP3)–whichisimplementedbytheUNChildren’sFund(UNICEF)andoperatesinfivestates,includingtwoTDPPhase1states(ZamfaraandKatsina)andoneTDPphase2Error!Referencesourcenotfound.,VolumeII,containsafactsheetonTDP’sin-servicecomponent.3GEP3doesnothaveteachertraininginterventionsinKatsinaandZamfara.Thereis,however,anearlylearningpilotfocusingon


©EDOREN 5

2 state (Niger)– focusesoncreatinganenablingenvironment forgirls’education inNigeria.3 Intheareaofgovernanceandadministrationof the teachingworkforceTDPwillbuildon systemsreform initiatives already underway through other DFID programmes in Nigeria, such as StatePartnership Accountability Responsibility and Capability (SPARC-I and II) and the StateAccountability and Voice Initiative (SAVI). Theirwork includes encouraging states to implementexistingemploymentconditionstransparently;ensuringthatteachersarepaidontime;ensuringthatrecruitment,promotionanddeploymentistransparentandfair;andensuringthatthesystemisheldmoreaccountable. Finally, in its implementationandevidencegeneration, it is expectedthat TDP will also have complementarities and engagement with education programmes beingfunded by other international development partners, such as the US Agency for InternationalDevelopment(USAID), theBritishCouncilandtheJapan InternationalCooperationAgency(JICA)(DFIDNigeria,2012).4

2.2 OverviewofTDP’sin-serviceteachertrainingoutput

TDP’s in-service teacher training output aims to provide a school-based in-service trainingprogrammesupportedbythreekeyactivities:

• ‘trainer in the pocket’, which provides access to audio-visual pedagogical resourcesthroughabasicmobilephone(per teacher)andtwomobileamplifier-radios (perschool)forplayingaudio-visualmaterialsintheclassroom;5

• continuous support through theprovisionofongoing support to teachersusingmultiplelayers and mechanisms (initial orientation workshops, cluster meetings,6 classroomobservationduringschoolsupportvisitsbytrainers,andaudio-visualandprintedself-studymaterials),insteadofone-offtraining.Thesupportmechanismforcontinuousprofessionaldevelopment is expected to be institutionalised within schools (through head teacherleadershipandmanagement trainingandpeer supportamong teachers)andeachof theparticipatingstates’educationsystems;and

• school-based interventions which comprise peer-to-peer learning among teachers,materials for pupils and teachers, and mentorship and teacher support through headteachers.

The in-service teacher training output is intended to provide regular and ongoing training andsupporttoabout62,000teachersby2019(TDP2014).Thesein-serviceactivitieswillberolledoutinparticipatingschools inPhase1statesbyselectingand training, ineveryparticipatingschool,twoteachersforEnglishandtwoteachersformathsonGrade1–3materialsinYear1(2014).InYear2 (2015) the same four teacherswillbe trainedonGrade4–6materials, and subsequentlytwo of these teachers will be trained on Grade 4–6 science materials. The programme will

3GEP3doesnothaveteachertraininginterventionsinKatsinaandZamfara.Thereis,however,anearlylearningpilotfocusingonearlygradeliteracyandnumeracyinHausa.4USAID’sNorthernEducationInitiative(NEI)operatesintwonorthernstates(SokotoandBauchi)andprovidesteachertrainingforearlygradereadinginHausa.JICAprovidedteachertraininginmathsandscienceinthreestates(Niger,KadunaandPlateau),withanambitiontocoverallNigerianstates;thisprogrammehasnowended.TheBritishCouncilisalsoworkinginnorthernNigeriainsupportofEnglishlanguageteachertraining.5 The audio-visual materials will be produced on secure digital cards, with the aim of achieving wide distribution and usage.Teachersandheadteachersparticipatingintrainingwillbeprovidedwitha‘technologypack’(aphone,SIMcardandcharger)thatwillbeheldbyschoolsbutshouldbemadeavailableforteacherstotakehomeforstudying.Asmentionedabove,eachschoolalsogetstwoamplifierstoplayaudio-visualmaterialsintheclassroom,keptunderthecareoftheheadteacher.Thisisthe‘trainerinthepocket’approachthathasbeenusedintheDFID-fundedEnglishinAction(EiA)programmeinBangladesh.6Frequentclustermeetingsbetweentrainersandteacherswillbefacilitatedbygroupingprogrammeschoolsintoclustersbasedontheirgeographicalproximity.


©EDOREN 6

establish permanent cadres of teacher trainers (called TFs) responsible for the coordination,developmentanddeliveryofthein-servicetraining.

The TOCunderlying TDP’s in-service training component, represented in Figure2below, is thatpupils learnmorewhen they are taught by effective teachers, and that teachers becomemoreskilledandknowledgeable (both in termsofpedagogyand subject knowledge) through training(DFID Nigeria 2012). Other things being equal, pupils’ satisfaction and interest in schoolingimprovesas teachereffectiveness isenhanced,andthis increasespupilattendanceand learningachievement, and reduces ratesofdropout.Asa combined result, teachers’motivation risesastheyfeelmoreeffectivewhentheirpupils’learningoutcomesimprove,andthisaffectsteachersinanumberofways,includingbyreducingteacherabsenteeism.TheTDPTOCfurtherassumesthattrainingeffectivenessisimprovedthroughheadteachersupport;mentoringandsupervision;peerinteractionandsupportfromsimilarlytrainedteacherswhoareabletosupporteachother;andregularinvolvementbyvisitingteachertrainers,whoactnotonlyastrainersbutalsoasmentorsandsupervisors.


©EDOREN 7

Figure2TDPTOCforOutput1:In-serviceteachertraining7

7ThelinksshadedingreyrefertothecomponentsoftheTOCwhicharebeingdirectlyassessedbythisimpactevaluation.

TDPactivitiesandoutputs

Improvedteachersubjectcontentknowledge

Improvedteacherpedagogicalknowledge

ImprovedlearningoutcomesinEnglish,mathsandscienceforcohortstaughtbyselectedteachersinTDPschools2014-2019

Improvedteachereffectivenessinclassroom

Improvedteachereffectivenessinoutsideclassroomsupport

Improvedheadteacherleadershipandmanagement

TDPoutcomes TDPfinalimpact TDPintermediateimpact

TDPactivitiesfollowingthein-servicestrategyareexpectedtoleadtooutputsinfiveareas

Scaleoutcomes

TDPmodelappliedinotherstatesinNigeria

ImprovedlearningforcohortsinschoolsinNigeria2016-2019

Sustainabilityoutcomes TDPmodelappliedsustainablyinTDPschools,inotherschoolsinTDPstates,andinNigeria

ImprovedlearningforfuturecohortsinTDPschools,inotherschoolsinTDPstates,andinNigeria

TDPmodelappliedinotherschoolsinTDPstates

Improvedlearningforcohortsinschoolsinotherstates2014-2019

TDPin-servicetrainingimprovesperformanceofotherteachersinTDPschools

ImprovedlearningforcohortstaughtbyotherteachersinTDPschoolsstates2014-2019

TDPoutputsinfiveareas: 1.Collaborationandpartnership 2.Trainingandsupport 3.Materialsdevelopment 4.Technologyuseandmanagement 5.INSETprogrammeimplementation


©EDOREN 8

2.3 Objectivesoftheimpactevaluation

InNovember2014educationconsultantsfromEducationData,ResearchandEvaluationinNigeria(EDOREN) finalised the evaluation framework for TDP’s in-service component, which uses atheory-based,mixed-methods approach to assess if TDP’s in-service teacher trainingmodel hasimproved teacher effectiveness and learning levels of pupils in primary education in northernNigeria(EDOREN2014).

Thisimpactevaluationhasthreemainpurposes:• formative: to help inform the implementation of TDP in its first phase of in-service

activities,andthedesignandimplementationinitssecondphase;• summative:tohelpinformTDP,DFIDNigeriaandothereducationstakeholdersifTDP’sin-

service teacher training component has led to improvements in teacher effectiveness inEnglish,mathsandscienceandpupillearninglevels;and

• learning from the TDP about what might work for improving teacher effectiveness inNigeriaandelsewhere.

Theprimaryusers of this impact evaluation areDFIDNigeria and the TDPwithother keyusersbeing: state ministries of education, state commissioners of education, SUBEBs in the six TDPstates, the Federal Ministry of Education, the Universal Basic Education Council, the NigeriaEducationalResearchandDevelopmentCouncil(NERDC),theNationalCommissionforCollegesofEducation(NCCE),ESSPINandGEP3.AdditionalaudiencesfortheimpactevaluationincludeLGEAstaff,headteachers,teachers,pupilsandtheirparents,theDFIDResearchandEvidenceDivision(RED), other education donors in Nigeria, and other education projects in Nigeria andinternationally.Box1KeybackgrounddocumentsandcompanionreportsThe two volumes of this baseline report outline TDP, its context and its impact evaluationmethodology,andpresentthequantitativeandqualitativebaselineresultsandpossibleprogrammeimplications.Readersmightalsofindthefollowingcompanionreportsanddocumentsofinterest:• TDPquantitativebaselinesurveystate reports for Jigawa,KatsinaandZamfara,whichdiscuss

detailed state level results (De, Pettersson, et al., 2015a; De, Pettersson, et al., 2015b; De,Pettersson,andMorris2015).

• ReviewoftheliteratureonbasiceducationinNigeria(HumphreysandCrawfurd2014),whichexamines issuespertinent to theconstraints facedbyprimary teachers,pupils and schools inNigeria.

• TDPin-servicetrainingimpactevaluationframework(EDOREN2014),quantitativeanalysisplan(DeandPettersson2015a)andqualitativeconceptnote (De,Cameron,etal.,2015)onwhichthedesignandanalysisofthisbaselinesurveyarefounded.

• TDPin-servicetrainingimpactevaluationinitialdesignnote(McCormick2014).• TheTeacherManagementStudyforKadunaandKatsina(WattsandAllsop,2015).• TheKatsinaTeacherSupply,DemandandUtilisationStudy(2014–2025)(Bennell,Anyawu,and

Dodo2014).Note:Fullreferencesaregiveninthebibliography.


©EDOREN 9

2.4 Overviewoftheimpactevaluationdesign

This sectiongivesanoverviewof themain featuresof theoverall impactevaluationdesignandthendiscusseskeyelementsofthebaselinesurveys.Theaimistogiveadequate informationtoset the context for the results that follow, from Section 3 onwards. The technical design anddetailsarediscussedinrelevantchaptersinVolumeII.

2.4.1 Mixed-methodsdesign

GiventhefocusonunderstandingtheimpactofTDP’sin-serviceteachertrainingcomponent,andthe need to distinguish it from other education programmes, the centrepiece of this impactevaluation is an experimental design based on random assignment of schools to treatment orcontrol status.Thisapproachallowsquantitativemeasurementandattributionofany impactofTDP’s in-service training on teacher effectiveness and primary grade pupil learning levels in itsPhase1states.This formspartofthetheory-basedmixed-methodsevaluationthatwillalsousequalitativeresearchtotestwhethercoreassumptionsunderlyingtheprogramme’sTOCholds inpractice;explorepathways throughwhichanyobserved changesare likely tohaveoccurred; aswellasdevelopingnewhypothesesandexploringunexpectedimpacts.

Figure3illustrateshowthequantitativeandqualitativedatasourcesarecombinedtoprovideanunderstandingof(any)TDPimpactandpotentialmechanismsthroughwhichtheprogrammemayhave influenced the outputs and outcomes of interest. It shows that while the quantitativeresearchfocusesoncollectingdataattheschoollevel,coveringheadteachers,teachersandpupilsin a representative sample of programme impact evaluation areas, the qualitative researchcollectsdatainasmallpurposivesampleoflocalgovernmentsandschools.8

8 As a result of this staggered sequence and timingof the quantitative andqualitative surveys, the evaluationwas also able toconducta‘lighttouch’processevaluationundertakenaspartofthebaselinequalitativestudy,whichtookplaceroughlysixmonthsafter commencement of implementation of in-service training activities. This gave the authors an initial opportunity to studyvariousimplementationprocessesandpracticaldynamics,andinturntoprovideinitialimpressionsofhowimplementationofthein-service output is progressing. Having said this, it is indeed atypical of baseline surveys, by their very nature, to observe andcommentonimplementationofanintervention.Thus,thecorefocusofthisbaselinesurveyreportistoestablishbaselinelevelsofteacher effectiveness and pupil learning before the start of TDP’s in-service teacher training activities, especially comparabilitybetween the programme’s treatment and control groups, which will be used to evaluate programme impact at the follow-uproundsofdatacollectionin2018.


©EDOREN 10

Figure3Quantitativeandqualitativedatacollectionatbaseline

Source:Authors.Note:KII=keyinformantinterview;FGD=focusgroupdiscussion

2.4.2 Overviewofquantitativeandqualitativeinstruments

During the quantitative survey, the head teacher and selected teachers at each school wereinterviewed.Eachteacherandheadteacherwhoteacheswasalsoobservedwhiletheytaughtaclass.Followingthecompletionoftheschoolsurvey,allteachersandheadteachers(irrespectiveofwhether they teachornot)wereadministeredaTDNAatanexaminationcentre. Inorder toassesspupillearninglevelsforthisbaselinesurvey,eightofallthepupilswhostartedGrade3inSeptember2014,andwhowerebeingtaughtEnglish,mathsorsciencebyatleastoneTDP/controlteacher,wererandomlyselectedforthecombinedEnglish,mathsandscientific literacy learningassessment..

Table1liststhequantitativeinstrumentsandrespondents,andprovidesbriefdescriptionsoftheinstruments’contents.

Programme (TDP) and donor (DFID)

KIIs with TDP and non-TDP teacher interviews, case study TDP teacher: classroom observation, a

photo story "day in the life of a teacher",

vignettes

FGDs with pupils: life of a poor / non-poor pupil; what I do when I am not

at school

Source of data

Programme (TDP) and donor (DFID)

TDP teacher interviews, classroom observations, teacher

assessments

Grade-2 English literacy, numeracy

and scientific literacy pupil assessments

Three states: Jigawa, Katsina, Zamfara

Sample of LGAs

Sample of schools

Sample of teachers

Sample of pupils

Qualitative Quantitative

KIIs with TDP Abuja and state staff, ESSPIN, GEP

staff; DFID education and results advisors

Head teacher interviews on school

leadership and management

KIIs with State Universal Basic Education Board

(SUBEB) staff

KIIs with Education Secretaries

KIIs with head teachers; TDP teacher facilitators; teacher training cluster meeting observations


©EDOREN 11

Table1Overviewofinstrumentsandrespondents

Instrumentdescription RespondentperschoolHeadteacherinterviewandschoolrecordchecksHeadteachergender,age,yearsofexperience,academicqualifications,trainingundertaken

Headteacher(oneperschool)Frequency/typeofinteractionwithandsupervisionofteachersNumberofpupilsregisteredandteachersemployed,pupil–teacherratio(PTR),schoolinfrastructureandresources,SBMCsTeacherattendancefromschoolrecordsPortraitphotographofheadteacherforpanelidentificationatendline

TeacherinterviewTeachergender,age,yearsofexperience,academicqualifications,trainingundertaken

Threesampledteachers(teachermotivationquestionswerealsoadministeredtoheadteacherswhoteach)

Frequency/typeofin-servicetrainingreceivedInteractionwithandsupervisionbyheadteacherSelf-reportedabsenteeismTeachermotivationPortraitphotographofteacherforpanelidentificationatendline

TDNAinEnglish,maths,andscienceandtechnology

Assessmentofsubjectknowledgeandabilitytomeasureandanalysepupilacademicprogress

Threesampledteachersandheadteachers(irrespectiveofwhethertheyteachornot)

ClassroomobservationKeyteacherbehaviourintheclassroom,includingteachertalk,teacherlanguage,teacheractions Threeteachersselectedby

TDP,andheadteacherswhoteach

PupilactivitiesinclassroomInstancesofpraiseandreprimandbytheteacher;useofteachingaids;etc.PupillearningassessmentEnglishliteracy,numeracyandscienceEnglishliteracy:earlyliteracy,readingwithcomprehension,writing,other

EightrandomlysampledGrade3pupils

Numeracy:pre-numeracyandGrades1and2levelnumeracyquestionsScientificliteracy:pre-scienceandGrade2levelquestionsPupilgender,age,languageandhouseholdassets

Portraitphotographofpupilforpanelidentificationatendline

The qualitative part of the impact evaluationmade use of threemain researchmethods –KIIs,FGDs andobservations. Table 2 summarises the types of instruments thatwereused to collectinformationfromdifferentparticipantsinthequalitativeresearch.


©EDOREN 12

Table2InstrumentsadministeredforeachparticipantgroupInstrumen

t

Head

teache

rs

TDPteache

rs

TDPcasestud

yteache

r

Non

-TDP

teache

rs

Pupils(girls)

Pupils(boys)

LGEA

edu

catio

nsecretary

SUBE

Bofficials

TDPTF

TDPclusterm

eetin

gs

TDP,DFID,ESSPINand

GE

Pprogrammestaff

KIIs x x† x x† x x x x

FGDs x x

Observation x x

†TheseweregroupinterviewsthatincludedsomeinteractiveelementsmoretypicalofFGDs.

2.4.3 Samplingstrategiesandsamplesizes

The overall sampling strategy9 for the quantitative survey was shaped by practical programmeconsiderations and resource constraints. Within these parameters, the design is intended tomaximise the statistical power of the impact indicator difference-in-difference measures (andreducetheminimumdetectableeffect(MDE),tomaketheevaluationassensitiveaspossibletodetectingsmallchanges).ThisisexplainedfurtherinVolumeII.Theintendedtotalsamplesizesforthequantitativebaselinesurveyforallthreestateswere:336headteacherinterviews;classroomobservations of head teachers who teach (up to 336); 1,008 teacher interviews and classroomobservations;1,344TDNAsadministeredtobothteachersandtoheadteachers;and2,688Grade3pupilstestedonGrade2learningassessments(Table3).Thesameteachersandheadteachers,andpupilswhoweresurveyedatthebaselinewillbesurveyedagainattheendlineinJune2018.

Table3Summaryofintendedsamplesizesforthequantitativesurvey

Instruments/evaluationclusters Perschool

Treatment(perstate)

Control(perstate)

Total(perstate)

Total(threestates)

Evaluationclusters n.a. 14 14 28 84

Totalsampledschools(foursampledpercluster)

n.a. 56 56 112 336

Headteacherinterviews 1 56 56 112 336

Headteacherswhoteach:classroomobs. Upto1 Upto56 Upto56 Upto112 Upto336

Teacherinterviews 3 168 168 336 1,008

Teacherclassroomobservations 3 168 168 336 1,008

TDNA(teachersandallheadteachers) 4 224 224 448 1,344

Pupillearningassessment 8 448 448 896 2,688

9 It is important to note that these reports have used “cluster” in an operational/programmatic sense (borrowing from TDP orESSPIN) which refers to a collection of schools grouped together by the implementers for logistical purposes. However these“clusters” arenot the sameas the term“clusters”normally referred to in traditional samplingmethodology.Rather thanbeingsamplingunitsdrawnfromalargerpopulationinordertorepresentthatpopulation,the84“clusters”andtheschoolsinthemarethewholepopulationthatthesurveydatarepresents.


©EDOREN 13

Forthequalitativecomponentofthisimpactevaluationtheapproachtosamplingschoolsisbasedon stratified purposive sampling. The sample for the qualitative strand is nested within thecontexts used for sampling by the quantitative strand. In other words, for each state, the 56treatmentschoolsselected for inclusion in thequantitativestudywere first listed indescendingorderof theaverageof teachers’baseline scores in the teacher subject knowledgeassessment,TDNA. These schoolswere then divided into three strata, representing the top 10% of schools(‘high-performing’), middle 10% (‘typical schools’, i.e. 45%–55%), and bottom 10% of schools(‘low-performing’).Twotypesofschoolwereremovedfromeachshortlist:schoolsthatweremorethan90minutes’ travel fromthestatecapitalwere removed for logisticalandsecurity reasons;andschoolsthatwerelocatedinLGEAsthatweredeemedinsecurebysecurityconsultantswerealso removed from each shortlist. Finally, a school was randomly selected from within eachstrata’sshortlisttostudyinmoredepthaspartofthequalitativestudy.Theprocesswasrepeatedwithin each of the three TDP pilot states, Jigawa, Katsina and Zamfara, giving a total of nineschoolsacrossthethreestates.Table4belowsummarisesthisprocess.Table4Schoolselectionprocess

Samplingframeforqualitativestudy

Filter Filter Filter Repeat

56treatmentschoolsperstate

Avg.TDN

Ascore

Top 10%[6]

Accessibility

Top[~2–4]

Rand

omse

lection 1High-performingschool

x3

(for

each

state)

Mid 10%[6]

Mid [~2–4]

1Typically-performingschool

Bott. 10%[6]

Bott. [~2–4] 1Low-performingschool

Source:EDORENauthors.

2.4.4 Generalisabilityofbaselineresults

Thefindingsofthebaselinequantitativesurveyarerepresentativeonlyoftheclusteroftreatmentand control schools where at least one of the TDP-selected (treatment) or control teachersteachesEnglish,mathsor science toGrade3pupils.Resultsarenot representativeof the threePhase 1 TDP statesmorebroadly. This is because the TDP clusterswerepurposively chosenbySUBEBsineachLGA,ratherthanbeingrandomlysampledfromacomprehensivelistofpotentialTDPclusters(intowhichallschoolscanbeallocated).Furthermore,inbothtreatmentandcontrolschools, if treatment (or control) teachers did not teach Grade 3 pupils, the entire school wasreplacedduringfieldwork.Replacingschoolsinthiswaycouldintroduceabias,albeitonethatisbalanced across treatment and control schools, meaning that the impact results would not berepresentativeofTDPimpactoverallbutonlyofschoolswheretheselectedteacherstaughtGrade3.

As for the qualitative research, this component was not designed to produce results that aregeneralisable in the same sense as quantitative data. The qualitative research component wasbasedonarelativelysmallsampleofnineschoolsandsamplingforthiscomponentwaspurposive,withtheaimofincludingschoolswithparticularcharacteristics,ratherthanbeingrepresentativeof all schools in the three states. This issue was further compounded by the inability of theresearch teams to access all or any LGA as theywished, due to security concerns. Thus,moreremote schools further away from the state capitalwere less likely to ultimately feature in thefinalsampleofschools.Generalisabilitywasthusderived from linkingqualitative findings to the


©EDOREN 14

TOC.Insomecasesitwasmoreappropriatetotalkofwhetherfindingsweretransferableratherthangeneralisable:researchersneededto investigatecontext inordertoconstructanargumentthatafindinginonesettingwaslikelytoapplyinanother.Theriskofvisitingatypicalschoolsandgaining an incorrect or incomplete understanding of the relevant processes remains, but ismitigatedbyvisitingschoolsindifferentstatesandcontexts,andbypayingcloseattentiontowaysinwhichthecontextofeachschoolmaybeatypical.

2.5 Pointersonhowtointerprettheresults

2.5.1 Whatthequantitativefiguresandtablesshow

Allthequantitativefiguresandtablespresentedintheresultssectionshowthemeanestimateorproportionas relevant foreach indicator.Unlessotherwisestated, theword ‘average’hasbeenused in this report to signify central or typical values in the distribution. All figures and tablesdisplay the sample size (N) – that is, the number of respondents who answered a particularquestion for each indicator. In some tables, the estimates have asterisks, which indicate astatistically significant difference between the groups shown: *significant at 10% level**significantat5%level***significantat1%level.Themoreasterisksthatareshown,themorelikelyitisthattheobserveddifferenceisduetorealdifferencesbetweenthegroupsratherthantochance,drivenbywhowasinterviewedortested.Sometablesandfigurescompareestimatesfordifferentsub-populations,forexamplemaleandfemalepupils.

To give information on the spread of results, the detailed statistical tables in Error! Referencesourcenotfound.MofVolumeIIcontainpercentileestimates.Theseareestimatedvaluesbelowwhichagivenpercentageofobservationslie.Forexample,the10thpercentile(labelledP10inthetables)istheestimatedvaluebelowwhich10%ofobservationslie.Similarly,the90thpercentile(labelledP90)istheestimatedvaluebelowwhich90%ofobservationslie.Thesetablesalsoshow95%confidenceintervals,whicharetherangeofvaluesforwhichthereisa95%probabilitythatthe estimated confidence interval encompasses the true value of the population parameter.Standard errors are also reported in these tables and these aremeasures of the accuracywithwhichasamplerepresentstheunderlyingpopulation.Thesestatisticshavebeencalculatedtakinginto account single-stage sampling for schools and teachers, and two-stage sampling for pupilswithinschools.

2.5.2 Weightedestimates2.5.2 Weightedestimates

To provide estimates of key indicators that are representative of the treatment and controlclusters in the 14 LGAs selected by the programme, the observed values were analysed usingsurveyweights.Forinstance,ifafixednumberofpupilsaresampledfromGrade3ofeachschoolirrespective of the number of pupils enrolled in the class, as is the case in this survey, unlessweightsareusedthesampleofpupilswouldbeoverrepresentedbypupils insmallschools.Therelevantweightstousediffersdependingonwhetheranalysisiscarriedoutattheschool,teacheror pupil level, and survey weights were calculated for each of these levels. For a detaileddiscussionofthesamplingframeandstagesseeAnnexD(VolumeII)ofthisreport.


©EDOREN 15

2.5.3 Referencetableforassessingstrengthofevidence

One of the objectives of the baseline analysis is to ascertain whether the core assumptionsunderlyingtheTOCofTDP’sin-serviceoutputholdinreality.Attheendoftheresultssectiontheauthorshavesummarisedtheassumptionspertainingtothesection,commentedonwhethertheassumptions were met/not met/inconclusive, and the strength of evidence underlying thisjudgement. Clear standards were agreed between researchers for assessing the strength ofevidence that is applied to conclusions aboutwhether assumptions in the TOC held. These aresummarisedinTable5below.

Table5Definitionsofstrengthofevidenceusedbyresearchers

Strengthofevidence

Definition

StrongEvidence is both frequently explicitly mentioned in interviews and supported by researchers’ owninferencesfrominterviews,contextandbackgroundliterature.

Mixed

Evidenceiseither(a)bothexplicitlymentionedinaminorityofinterviewsandsupportedbyresearchers’own inferences from interviews, context and background literature; or (b) frequently explicitlymentionedin interviewsbutnotnecessarilysupportedbyresearchers’owninferencesfrominterviews,contextandbackgroundliterature.

OR

There is strong/moderate/weak evidence that an assumption holds in at least two cases but there isstrong/moderate/weakevidencethatthesameassumptiondoesnotholdinatleastanothertwocases.

WeakEvidenceiseitherexplicitlymentionedinaminorityofinterviewsorsupportedonlybyresearchers’owninferencesfrominterviews,contextandbackgroundliterature.


©EDOREN 16

3 Pupillearninglevels

Thissectionaddressesasetofevaluationquestionsabouthowmuchpupilsare learning(Box2;andseealsotheevaluationmatrixinAnnexAofVolumeII).10ThisinformationisderivedfromtheEnglishliteracy,numeracyandscientificliteracyassessmentsconductedaspartofthequantitativebaselinesurvey. It isrequiredboth inordertogiveapictureofcurrent learning levelsto informthe programme’s design, as well as to make comparisons at endline in order to evaluate theprogramme’s impact. In the final part of the section the qualitative baseline study is used toexamine the situations in which children are living, and the aspects of their context that mayprevent them from attending school regularly, and learning once they are in school. If thesecontextual factorsactasseverebarriers to learning, thenthe impactofmoreeffective teacherswillbelimited.

Box2Evaluationquestions(atbaseline)aboutpupillearninglevels

• WhatarethecurrentlevelsoflearningforGrade3pupilsinEnglishliteracy,numeracy,andscientificliteracy?–answeredusingquantitativesurvey(pupillearningassessment)

• Is there any significant difference in pupil learning levels between pupils in the treatment andcontrol groups, boys and girls, or poorer and richer pupils? – answeredusing quantitative survey(pupillearningassessment)

• What aspects of pupils’ backgrounds, context and attitudes help explain the current levels oflearning?–answeredusingqualitativesurvey(FGDswithGrade3pupilsinseparategroupsforboysandgirls)

Asbackground,Box3 summarises somebasic informationon thebackgroundofGrade3pupilsgatheredduringthequantitativesurvey(formoresupplementarystatisticaltables,seeAnnexMVolumeII).Box3BaselinebackgroundcharacteristicsofGrade3pupils

• TheaverageageofpupilswasnineyearsatthebeginningofGrade3.About45%ofpupilswereage-appropriateforGrade3(eighttonineyearsold),while17%wereunder-age(lessthaneightyears)and36%wereover-age(olderthannineyears).Almost25%ofpupilswereunabletotelltheirage.

• Nearly all (99%) of pupils named Hausa as themain language spoken at home. The rest (1%)namedFulfulde,KanuriorMangaasthemainlanguagespokenathome.

• Almost60%ofthesampledpupilswereboys.• In termsof household assets, the averagemalepupil comes froma lesswell-off household as

comparedtotheaveragefemalepupil.• Fouroutofthe2,575sampledpupilsshowedoneormoreformsofdisability,includinginability

to speakorwrite.Theywere included in thesamplenonetheless,andassessment itemswhichrequiredtheseabilitieswereskippedforthesepupils.

Source:Quantitativeimpactevaluationbaselinesurvey(October2014),pupillearningassessment.Note:ReportedpercentagesareweightedtorepresentallTDPtreatmentandcontrolschools.

10Thisreportusespupillearninglevelsinsteadofpupillearningoutcomestoavoidconfusionwithoutcomeindicatorsinthelogframe.


©EDOREN 17

3.1 PupillearninglevelsinEnglishliteracy

Thissectionaddressesthequestion:AttheendofGrade2havepupilsachievedthebasicEnglishliteracy skills expected at their grade level? To examine this, pupils were sorted into the threeperformancelevelsshowninTable6.Theseare:Level0,whichrequiresonlyelementaryskillstoanswer items correctly (pre-literacy); Level 1, which contains Grade 1-level items that requiresome emerging literacy skills to answer correctly; and Level 2, which comprises Grade 2-levelitems that require basic literacy skills to answer correctly.11 A performance level is achieved ifpupils aremore likely thannot tobeable todemonstrate the skills linked to thatperformancelevel.12 Thisbaseline survey testedGrade3pupils at thebeginningof theirnew school yearonitemsfromtheGrade2curriculumandbelow.HenceitisexpectedthattheywoulddemonstrateEnglishliteracyskillscorrespondingtoLevel2.

Table6Englishliteracyassessmentframework

Performancelevel Descriptionofpupilabilityassociatedwithperformancelevel

Level0:pre-literacy

Names the letter at the beginning of a familiar word (own name);answerssimpleoralquestionsusingcompletesentences(e.g.:‘Whatis your name?’); names and writes at least two letters from thealphabet; points to some body parts based on oral cues; orallynamessomeanimalsandeverydayobjectsbasedonpicturecues.

Level1:emergingliteracy

Readssometwo, threeor four letterwords; identifiessomesimplewrittenwordsafterhearingthem;soundsoutornamesmostloweranduppercaseletters;identifiesfirstlettersofeverydayobjectsandanimals; copies short sentences; answers simple questions in fullsentence based on visual stimuli; listens to a short story (oneparagraph)andanswerssimplequestions.

Level2:basicliteracy

Writesanswerstooralquestionsusingsomegrammarconventions;writeseveryday items inplural; spells (written) some two, threeorfourletterwordsbasedonoralcues;readsatleastsomewordsfromapassage.

3.1.1 Results:PupillearninglevelsinEnglishliteracy

Data presented in Figure 4 show that only about 3% of pupils in Grade 3 demonstrated basicEnglishliteracyskillsexpectedtohavebeenacquiredbytheendofGrade2(performanceLevel2).Almost36%ofpupilsfell inthemiddleperformanceband(Level1)anddemonstratedemergentEnglishliteracyskillsexpectedtobeacquiredbytheendofGrade1.ThisimpliesthattheseGrade3pupilshadnotacquiredthebasicEnglishliteracyskillsexpectedattheendofGrade2,andthuswerefallingbehindbyroughlyonefullgrade.Atthebottomofthescale(Level0)werethelargestandmostcriticalgroupofGrade3pupils(61%),whoonlydemonstratedpre-literacyskillsandhadyet to acquire emergent literacy skills expected at the end of Grade 1. They were below theexpectedgradelevelbyabouttwofullgrades.11Theseperformancebandsaremutuallyexclusiveandcollectivelyexhaustive.12Theterm‘achievingatbandlevel’meansthatpupilsinaparticularbandaremorelikelythannottobeabletodemonstratetheskills linked to that performance band. In general, the probability of a pupil responding correctly to a question from a lowerperformance level than the pupil's location in the learning distribution is greater than 0.5, while the probability of respondingcorrectlytoaquestionfromaperformancelevelhigherthanthepupil'slocationislessthan0.5.


©EDOREN 18

Figure4Distributionofpupilsbyliteracyperformancelevels

Comparing the distribution of performance in English literacy for different subgroups of pupilsrevealedasimilarpatternforboysandgirlsatperformanceLevels0and1.Therewere,however,significantgenderdifferences inthetopperforminggroup(Level2),withtheproportionofboysthat have achieved Grade 2-level skills (4%) being significantly larger13 than the correspondingproportion of girls (2%).14 The proportion of pupils from poorer households15 that onlydemonstratedpre-literacyskills(80%)wassignificantlylarger16thantheproportionofpupilsfromricher households (50%). At the other extreme, the proportion of pupils from the pooresthouseholdsatperformanceLevel2(basicliteracy)is0.4%,whichissignificantlysmaller17thanthatfrom the richest households (5%). There were no statistically significant differences in theproportions of pupils across various performance levels between the treatment and controlgroups,implyingthatthepupil learninglevelsinEnglishliteracywerebalancedacrosstreatmentandcontrolgroups.1819

Box4belowsummarisesstate-levelresultsonpupillearninginEnglishliteracyforJigawa,KatsinaandZamfara.Thesearediscussedinmoredetailinthecorrespondingstatereport(De,Pettersson,etal.2015a;De,Pettersson,etal.2015b;De,Pettersson,andMorris2015).

13Ata10%levelofsignificance.14SeesupplementarytablesinAnnexMofVolumeIIfordetailedstatisticaltables.15Anasset indexwascreatedusingprincipal componentanalysis.Basedon this, the lowest20%werecategorisedas the ‘poor’quintileandthetop20%asthe‘rich’quintile.Theassetindexwasbasedonphotosshowntochildrenofvarioushouseholdassets(abicycle,amat,achairetc.)duringthepupilassessmentandaskingthemiftheyownedtheassetornot.16At1%levelofsignificance.17At1%levelofsignificance.18Disaggregatingpupilperformancebymainlanguagespokenathomemayhaveprovidedaninterestinginsightintothecomplexrelationship between learning levels and language spoken at home, specifically when this is different from the language ofinstruction in classrooms. However, with almost 99% of pupils being predominantly Hausa speakers this analysis could not beundertaken.Thisisexpectedinthenorth-westzoneofNigeria,whichcomprisesthethreeTDPstatesunderstudy,whereHausaisthepredominantlanguage.TheotherlanguagesnamedbypupilswereFulfulde(0.7%)andKanuri(0.3%).19SeesupplementarytablesinAnnexMofVolumeIIfordetailedstatisticaltables,includingdisaggregationbytreatmentandcontrolgroups.

60.8

60.5

61.2

79.9***

49.8

35.8

35.3

36.5

19.7

45.2***

3.4

4.2*

2.3

0.4

5***

0 20 40 60 80 100

All(N=2571)

Boys(N=1633)

Girls(N=942)

Poorest20%(N=573)

Richest20%(N=503)

Percentageofpupils

Only3%ofpupilsdemonstratebasicliteracyskills

Level0:Pre-literacy Level1:Emergentliteracy Level2:Basicliteracy


©EDOREN 19

Box4State-levelresultsonpupillearninginEnglishliteracyforJigawa,KatsinaandZamfara

Only a small group of Grade 3pupils in treatment and controlschools demonstrate the basicEnglish literacy skills expected attheendofGrade2.

Among the three states, pupils inJigawaperformsignificantlyworsecomparedtopupils inKatsinaandZamfara. Only 0.1% of Grade 3pupils in TDP treatment andcontrol schools in Jigawademonstrate basic literacy skillsexpected at the end of Grade 2,comparedto5%inKatsinaandto4%inZamfara.

These impact evaluation survey results are representative of the TDP treatment and controlpopulations.However,thesepopulationsarenotinthemselvesrepresentativeofthethreePhase1TDPstatesmorebroadly.

Foradetaileddiscussionofthebaselineresultsatstatelevelseetheindividualstatereports(De,Pettersson,andMorris2015;De,Pettersson,etal.2015a;De,Pettersson,etal.2015b)

3.2 Pupillearninglevelsinnumeracy

This section answers the question:At the end of Grade 2 have pupils achieved basic numeracyskills expected at their Grade level? To examine this, pupils were again sorted into threeperformance levels, as shown in Table 7. These levels include pre-numeracy items that requireonlyelementaryskillstoanswercorrectly(Level0);Level1,whichlargelycontainsGrade1-levelitems that require some emergent numeracy skills to answer correctly; and Level 2 mainlyconstituted of Grade 2-level items that require basic numeracy skills to answer correctly.20 Asmentionedabove,sampledpupilswhohadjuststartedatGrade3wereexpectedtodemonstrateLevel2numeracyskills.

20Theseperformancebandsaremutuallyexclusiveandcollectivelyexhaustive.

61.4

51.9

76.2

60.8

34.9

43.1

23.7

35.8

3.7

5

0.1

3.4

0 20 40 60 80 100

Zamfara

Katsina

Jigawa

Allstates

BasicEnglishliteracy:scoresinJigawaweresignificantlylowerthanintheothertwostates

Level0:Pre-literacy Level1:Emergentliteracy Level2:Basicliteracy


©EDOREN 20

Table7Numeracyassessmentframework

Performancelevel Descriptionofabilitylevelassociatedwithbenchmark

Level0:pre-numeracy

Recognisesmostone-/two-digitnumbers;countsorallyfromoneto10;performssimplequantitydiscriminationtasks;measureslengthsusingnon-standardmethods(palm,steps)

Level1:emergentnumeracy

Measures lengthsusingconventionalmethods (rulers);counts from100 to 110; completes simple missing sequences of single-digitnumbers; recognises basic 2-D shapes; performs money sums forsmall amounts (up to Nigerian Naira (NGN) 10); andaddition/subtractionsumsofone/twodigits

Level2:basicnumeracy Performs two-/three-digit addition/subtraction sums, money sumsforamountsorchangeuptoNGN500,one-digitmultiplicationsums

3.2.1 Results:Distributionofpupillearninglevelsinnumeracy

ThedistributionofpupilsbynumeracyperformancelevelisshowninFigure5below.ThetopbarshowsthatonlyasmallpercentageofGrade3pupils(6%)inTDPtreatmentandcontrolschoolsdemonstrated basic numeracy skills expected at the end of Grade 2. A larger group of pupils,around15%, demonstrated emergent numeracy skills expected at the endofGrade1, and thisgroupwasroughlyonegradebelowthelevelexpected.Thelargestgroupofpupilsbyfar(79%),hadnotyetachievedemergentnumeracyskills,andwerethusbelowtheexpectedgradelevelbyroughlytwogrades.

Figure5Distributionofpupilsbynumeracyperformancelevels

79

75.4

84***

90.2***

69.9

15.3

18.1***

11.5

9.1

22.5***

5.7

6.6

4.5

0.64

7.6***

0 20 40 60 80 100

All(N=2571)

Boys(N=1633)

Girls(N=942)

Poorest20%(N=573)

Richest20%(N=503)

Percentageofpupils

Lessthan6%ofGrade3pupilsdemonstratebasicnumeracyskills

Level0:Pre-numeracy Level1:Emergentnumeracy Level2:Basicnumeracy


©EDOREN 21

Comparingthedistributionofdifferentsubgroupsofpupilsrevealedthatboysperformedbetterthan girls overall. Theproportionof girls (84%)withpre-numeracy skillswas significantly largerthantheproportionofboys(75%)atLevel0,whiletheproportionsofboysatLevel1(emergentnumeracy) was significantly larger than the proportions of girls at these levels.21 No significantdifferences were found in the proportion of boys vis-à-vis girls performing at Level 2 (basicnumeracy).

Differencesinlearninglevelsacrosssubgroupswereparticularlystarkbetweenpupilsfrompoorerversus richer households. The proportion of pupils from the 20% of poorest households thatdemonstratedpre-numeracyskills(Level0)was90%,significantlylarger22thantheproportionofpupilsfromtherichest20%ofhouseholds(70%).Conversely,whilealmost8%ofpupilsfromtherichest 20%of householdsdemonstratedbasic numeracy skills, less than1%of pupils from thepooresthouseholdsdid so.Therewereno significantdifferences in theproportionsofpupilsatvariousperformancelevelsofnumeracyacrossthetreatmentandcontrolschools,suggestingthatpupillearninglevelsinnumeracywerebalancedacrossthetreatmentandcontrolgroups.

Box5belowsummarisesstate-levelresultsonpupillearninginnumeracyforJigawa,KatsinaandZamfara.Thesearediscussedinmoredetailinthecorrespondingstatereport(De,Pettersson,etal.2015a;De,Pettersson,etal.2015b;De,Pettersson,andMorris2015).

Box5State-levelresultsonpupillearninginnumeracyforJigawa,KatsinaandZamfara

OnlyasmallgroupofGrade3pupilsintreatmentandcontrolschoolsdemonstratethebasicnumeracyskillsexpectedattheendofGrade2.

Amongthethreestates,pupilsinJigawaperformsignificantlyworsecomparedtopupilsinKatsinaandZamfara.Fewerthan1%ofGrade3pupilsinTDPtreatmentandcontrolschoolsinJigawa(fivepupils)

demonstratebasicnumeracyskillsexpectedattheendofGrade2(Level2),comparedto4%inZamfaraand10%inKatsina.

TheseimpactevaluationsurveyresultsarerepresentativeoftheTDPtreatmentandcontrolpopulations.However,thesepopulationsarenotinthemselvesrepresentativeofthethreePhase1TDPstatesmorebroadly.

Foradetaileddiscussionofthebaselineresultsatstatelevelseetheindividualstatereports(De,Pettersson,andMorris2015;De,Pettersson,etal.2015a;De,Pettersson,etal.2015b).

21Bothat1%levelofsignificance.SeesupplementarytablesinAnnexMofVolumeIIforsignificancetests.22At1%levelofsignificance.

79

87.2

71.3

83.2

15.3

11.9

18.8

13.1

5.7

0.9

9.9

3.6

0 20 40 60 80 100

Allstates(N=2571)

Jigawa(N=837)

Katsina(N=872)

Zamfara(N=862)

Percentageofpupils

Numeracy:PerformanceinJigawawassignificantlyweakerthaninKatsinaandZamfara

Level0:Pre-numeracy Level1:Emergentnumeracy Level2:Basicnumeracy


©EDOREN 22

3.3 Pupillearninglevelsinscientificliteracy

Alongside assessments in English literacy and numeracy, the sampled Grade 3 pupils wereadministeredanassessmentofscientificliteracyskills.23AsshowninTable8,thescientificliteracyassessment frameworkusedforthisanalysis linksperformance levels tocompetencies,withtheexpectationthatattheendofGrade2apupilwoulddemonstrateabilitiescorrespondingtoLevel2.

Table8Scientificliteracyassessmentframework

Performancelevel Descriptionofabilitylevelassociatedwithperformancelevel

Level0:observes Orallydescribesorrecognisesbasicpropertieslikesize,taste,weight,lengthandcolourofeverydayobjects

Level1:understandsUnderstandsinteractions,everydayphysicalandchemicalprocessesoreffectsthathavebeenexperiencedusinganabstractscientificconcept

Level2:explainswithunderstanding

Understandsandorallyexplainsprocessesoreffectsintermsofanon-observableproperty(i.e.phenomenathatpupilsmaynothaveexperiencedorseenfirst-hand)oranabstractscienceconcept

3.3.1 Results:Pupillearninglevelsinscientificliteracy

Data presented in Figure 6 show that 15% of pupils in Grade 3 demonstrated the ability tounderstand and orally explain everyday processes and phenomena using abstract scientificconcepts.Thisgrouphadroughlyachievedthe levelofscientific literacyexpectedat theirgradelevel.Around67%ofpupilsfell inthemiddleperformanceband(Level1)anddemonstratedtheability to understand interactions and everyday physical/chemical processes, but not explainthem. This meant that these Grade 3 pupils had not yet acquired the scientific literacy skillsexpectedattheendofGrade2.Atthebottomofthescale(Level0)wasagroupofGrade3pupils(18%)whodemonstratedtheabilitytoorallydescribeandrecognisebasicphysicalpropertiesofeverydayobjects,butwerenotabletouseabstractscientificconceptstounderstandthenaturalworldaroundthemorexplainthesephenomena.

23AccordingtoOECD-PISA,1999,scientificliteracyisoftendefinedbroadly,andisthe‘…thecapacitytousescientificknowledge,toidentifyquestionsandtodrawevidence-basedconclusionsinordertounderstandandhelpmakedecisionsaboutthenaturalworldandthechangesmadetoitthroughhumanactivity.’Ourscientificliteracytestcoveredthetopicsofsize,weight,colourandtastediscrimination;themotionofobjectsinwaterandair;workandeffort;andeverydaychemicalprocessesrelevanttothenorthernNigerian context (dyeing cloth). The items all related to a specific strand of scientific literacy, namely the use of scienceunderstandingstodescribeandexplainnaturalphenomena,andtointerpretreportsaboutphenomena.


©EDOREN 23

Figure6Distributionofpupilsbyscientificliteracyperformancelevels

Resultsrevealedsimilarproportionsofboysandgirlsinthemiddleperformancebandforscientificliteracy (Figure 6) but there were significant gender differences (at 5% and 10% levels,respectively) in the size of the highest performing group (17% boys and 12% girls), and of thelowestperforminggroup(16%boysand20%girls).24Theproportionofpupilsfromtherichest20%ofhouseholdsinthehighestperforminggroup(16%)wassignificantlylargerthantheproportionofpupilsfromthepoorest20%ofhouseholds(10%).25Therewerenosignificantdifferencesintheproportionsofpupilsatvariousperformancelevelsofscientificliteracyacrossthetreatmentandcontrolschools,suggestingthatpupillearninglevelsinscientificliteracywerebalancedacrossthetreatmentandcontrolgroups.

While the scientific literacy results are seeminglybetter than theEnglish literacyandnumeracyresults caution is recommended in drawing comparisons across the three subjects because thethreepartsofthetestassesseddifferentcompetenciesandskills.Forinstance,onereasonfortherelativelybetterperformanceonscientificliteracycouldbethepresenceofanumberoftestitemsthatassessedpupilsonsize,weightandlengthdiscrimination(e.g. ‘whichpencil is longer?’withpictorialaids)whichareprobably competenciespupilspickup fromtheirenvironment,perhapseven before they enrol in school. To answer these items correctly also does not requiresimultaneousapplicationofanotherfunction,likereadingorwriting.Incomparison,someoftheLevel 0 English literacy and numeracy items required skills like writing an English letter, oridentifyingsingle-digitnumbers,whicharearguablyskillsthatarenotpickedupnaturallyfromtheenvironment and which do require simultaneous functions of writing or reading. Hence, it ispossible that sampledpupils effectively found the scientific literacy test easier than the Englishliteracyandnumeracytests.

24SeesupplementarytablesinAnnexMinVolumeIIforsignificancetests.25At10%levelofsignificance.

17.9

16.2

20.3*

23.5

18.7

67.1

66.8

67.6

67.0

65.4

15

17**

12.2

9.6

15.9*

0 20 40 60 80 100

All(N=2571)

Boys(N=1633)

Girls(N=942)

Poorest20%(N=573)

Richest20%(N=503)

Percentageofpupils(%)

15%ofgrade3pupilscanexplaineverydayphenomenawithunderstanding

Level0:Observes Level1:Understands Level2:Explainswithunderstanding


©EDOREN 24

Box 6 below summarises state-level results on pupil learning in scientific literacy for Jigawa,KatsinaandZamfara.Thesearediscussed inmoredetail in thecorrespondingstatereports (De,Pettersson,etal.2015a;De,Pettersson,etal.2015b;De,Pettersson,andMorris2015).

Box6State-levelresultsonpupillearninginscientificliteracyforJigawa,KatsinaandZamfara

Only15%ofGrade3pupilsintreatmentandcontrolschoolsdemonstratedthescientificliteracyskillsexpectedattheendofGrade2.

Amongthethreestates,pupilsinJigawaperformedsignificantlyworsecomparedtopupilsinKatsinaand

Zamfara.Fewerthan6%ofGrade3

pupilsinTDPtreatmentandcontrolschoolsinJigawademonstratedscientificliteracyexpectedattheendofGrade2(Level2)comparedto17%inZamfaraand19%inKatsina.



17.9

16.2

20.3

23.5

18.7

67.1

66.8

67.6

67.0

65.4

15

17

12.2

9.6

15.9

0 20 40 60 80 100

All(N=2571)

Boys(N=1633)

Girls(N=942)

Poorest20%(N=573)

Richest20%(N=503)

Percentageofpupils(%)

15%ofGrade3pupilscanexplaineverydayphenomenawithunderstanding

Level0:Observes Level1:Understands Level2:Explainswithunderstanding


©EDOREN 25

3.4 Pupils’experiencesofschoolingandlearning

The following sections of this report will argue thatmuch of the explanation for low levels oflearning lieswith thewayspupilsare taught, theiraccess to learningmaterials, teachingqualityandinstructionaltime,andultimatelywithoverallschoolmanagement.Partoftheexplanationforlowlevelsoflearning,however,islikelytolieinpupils’backgroundsandthelevelsofsupporttheirfamiliesareabletoprovide.Thisincludesissuesrelatingtowhetherpupilsattendschoolregularly,and reasons, such as child work, that might prevent them from attending; whether they havesufficient nutrition, rest, and time for learningoutside school; andwhether parents are able tosupporttheirlearningdirectly,throughhelpwithhomeworkorreinforcingpositivebehaviour(Box7).Theseaspectsofpupils’livesmaybebeyondthescopeofateacherdevelopmentintervention,andyetarelikelytoimpingeontheeffectivenessofsuchinterventions.Theyneedtobetakenintoaccountinprogrammedesignandinassessingtheprobabilityofsuccess.

Thissectiondrawsmainlyontheresultsfromqualitativediscussionswithgroupsofpupilsinnineschoolsacrossthethreestatestopresentpupils’perspectivesonschoolingandlearning,includinghow pupils see their schools and teachers, and their attitudes towards problems that mightconstrain their learning. In each school, a group of girls and a group of boys participated,witharoundeightpupilsineachgroup.

Box7Baselineevaluationquestionsaboutpupils’context,backgroundandattitudestowardsschoolingandlearning

Attendanceandchildwork• Dopupilsattendtheirschoollessonsregularly?Whatarethereasonsiftheydon’tattend?• Howmightotherout-of-schoolobligationslimittheirattendanceorthetimethattheycanspendon

learningoutsideschool?Pupils’householdbackground• Dopupilshavesufficientnutritionandresttimetobeabletoconcentrateduringlessons?• Dopupilshaveopportunitiesoutsideofschoolhourstoreinforcetheirlearningthroughdiscussion

andhomework?Whatsupportdoparentsprovidetotheirchildren?• Doparentsencouragechildrenbyreinforcingpositivebehaviourandlearningoutcomes?Pupils’attitudestoschoolandteachers• Whatarepupils’attitudestowardstheirschoolsandteachers?• Whatproblemsdopupilsperceiveintheschoolthatmightconstrainlearning?

3.4.1 Attendanceandchildwork

Whenaskedtoranktheproblemstheyface,teachersplacedpupilabsenteeismamongthemostimportant.Observed attendancewas typicallymuch lower than thenumbers of pupils officiallyregistered ineachclass.Thequalitative interviewswerecarriedoutwithpupils in schoolat thetimeofthesurveyteam’svisit,andsoprovideonlyapartialpictureofthepressuresthatmightbeplacedonpupils’attendance.Teachersandheadteachersoftenallegedthatpupilsdonotattendschool regularly because of (paid or unpaid)work, or because their parents prefer them to besimultaneouslyenrolledinlessonsoutsidetheschoolfocusedonIslamiceducation.


©EDOREN 26

Most pupils acknowledged that they carried out household chores. Their householdresponsibilitiesincludedhelpingonthefarm,washingdishes,fetchingwater,cleaning,sweeping,buying and pounding ingredients, running errands, and helping with cooking. Some of theseactivitiesmaybecarriedoutmorebyonegenderthananother,butinqualitativeinterviewstherewasnocleardivision.Asmallnumberofbothboysandgirlsalsosaidthattheyhawkedgoods,andoneboyclaimedtohelpwithcarrepairsinagarage.However,mostofthepupilssaidthatthesehouseholdresponsibilitieswereundertakenoutsideoftheirschoolhoursanddidnotaffecttheirschooling. Some admitted that tasks such as fetching water, in relation to farming, wouldsometimesmakethemlatefor,orevenmiss,school. Inafewschools,basedondiscussionsandafter looking at pupils’ notebooks, it was clear thatmost of the pupils in the focus group hadmissedthepreviousday’sschool.

Manypupilsattendedpart-timeIslamiceducationlessonsinthemorningoreveningunderalocalmallam (teacher), although pupils claimed that this did not affect their attendance at thegovernment school. Teachers and head teachers raised cases of pupils who had left thegovernmentschoolaltogethertoattendatsangaya,26althoughinoneschooltwoboysstayingataresidentialtsangayaschoolhadbeenallowedtoattendthegovernmentschoolduringtheday.

3.4.2 Householdresourcesforschooling

Although primary schooling is free inNigeria, schools sometimes charge fees for exams or PTAcharges. Additionally, pupils are expected to wear uniforms. There are, therefore, indirect orunofficialschoolingcosts,evenwhentheofficialcostsarezero.

Pupils’nutritionalstatusislikelytoaffecttheirlearninginschool.Asdescribedbelow,somepupilsare described as coming to school hungry and without money for food, due to their parents’limitedabilitytosupportthem.Furthermore,intheorypupilsneedtimeandsupportfromparentsto complete homework, but pupils’ and teachers’ accounts suggested that pupils rarely dohomeworkoutsideoftheschool,insteadtheyusethetimewhenteachersarenotpresentintheclassroomtocompleteassignments.

Finally,parents’encouragementandmonitoringoftheirpupilsislikelytobeimportantinensuringtheyattend regularly; in several schools, teachers ranked lackof support fromparentsasbeingamong the main constraints that they faced, and this was often cited as a reason for poorattendanceinclass.

3.4.3 Pupils’experiencesofschooling

Aspartof thequalitativediscussion,pupilswereaskedto imagineanddescribeahappyandanunhappy child. For the happy child, pupils tended to describe a pupil who has adequate studymaterials(e.g.stationary,schoolshoes,etc.),acleanuniform,freetimetoplay,apackedlunchorpocket money for food, and friends in school, and who is loved by teachers and elders, asillustratedbythequotesbelowfromaboys’FGD:

26 Tsangaya is a traditional form of qur’anic religious educationwhere the teachermoveswith his pupils in the belief that anitinerantlifeisessentialforthemtofullyconcentrateontheirstudy.Theitinerantchildrenattendingtsangayaeducationarecalledalmajirais.


©EDOREN 27

Interviewer:Now let us think of a boywho is happywith his school, and even happy athome.Letuscloseoureyes forawhileandthinkaboutour friendwho ishappywithhisschool,andevenhappyathome.Tellmeaboutourfriend…

Pupils:Heissmilingandhappy…hehasacap,hisuniformisveryclean,hecomestoschoolwithwritingmaterialsschoolbag,pens,andpencils….Hehasabicycle;hecomeswith20nairapocketmoney,andhisparentsloveshimaswellastheteachers,heistheheadteacher’spet.Interviewer:Whatkindoffooddoeshecometoschoolwith?Pupils:Indomienoodles,riceandstew,breadsometimesyam,sometimessweetandwaterinhisbag.Interviewer:Let’stalkabouthisfamily,hislifestyleathome,hishobbies.Pupils:Herunserrandsforhismother;hepaysattentionintheclass.

Fortheunhappychild,pupilstendedtobuildupapictureofachildwhoisasocialandacademicoutcast: poor, dirty, unintelligent, disobedient, aggressive, a cheat, a thief, and irreligious. Theunhappypupils’hungeranddifficulthouseholdbackgroundmakeshimorher,inpupils’accounts,behaveaggressivelyandrudelytowardsteachers,whothenpunishthepupilormakehimorherrepeat grades, and this in turn worsens the pupil’s attitude towards the school. Rather thansympathisingwiththis imaginarypupil, thediscussantssaidtheywouldavoidtheunhappypupiland not befriend him or her because s/he is perceived as troublesome and has an unkemptappearance.Thefollowingquotesillustratethis.

Interviewer:Letustalkabouthislifeintheclassplease?Doeshecometoschooleveryday?Pupils:Heisdullintheclass,comestoschoolwithoutagooduniformandnopocketmoney,hisbagcontainsonlysomepiecesofpapers,andevenifheaskshisparentsdon’tprovideforhim,manytimesheabsentshimselffromtheschool.Interviewer:Why?Pupils:Becausehehatesschool,hewillprefertogotofarmandworkandgetpaid,hisheadisverydirty.[Boys’FGD]

Interviewer:Whydoeshecomelatetoschool?Pupils:Becausetheyfloghim.Interviewer:Becausetheyfloghim?Pupils:Yes,theyfloghiminschoolandwhenhegoeshometheyalsofloghimsohecomestoschoolangry.[Boys’FGD]

Pupils:Shecriesalot,sometimesteacherssendherbackhomeifshekeepscryingintheschool,herhouseisfarawayfromtheschool,anditisamudhouse.Interviewer:Whydoyouthinksheisnothappy,orwhydoyouthinkshecriesalotintheschool?Pupils:Shedoesn’twanttocometoschool;shehasnogooduniform,noshoes.[…]Interviewer:Howdoteacherstreatherintheclass?Pupils:Theysometimesusedtobeatherbecauseeveniftheytalktohershewouldneverstopcrying,sheisdull,shedoesn’tcopynotesoranswerquestionsintheclass.Interviewer:Whydoyouthinkshecriesalot?


©EDOREN 28

Pupils:Shehatesschoolmaybethatiswhyshecriesalot,sheabstainsfromschool,shehasaworn-outuniform.[Girls’FGD]

The accounts of both pupils and teachers indicated that pupils have some agency in decidingwhether or not to attend school. This would suggest that pupils’ experience of school is animportant influenceonattendance.Thepupilswholackparentalsupportaredepictedashavingdistressing experiences in school and, in some cases, as dropping out or attending schoolinconsistently.

Theassociationbetweenchildworkandthispatterninpupils’accountswasnotstraightforward.In several cases farm work was associated with the unhappy child, while the happy child wassometimes depicted as willingly helping with household chores. This suggests a distinctionbetween accepted forms of children’swork, compatiblewith a good experience in school, andotherformsofworkwherepupils’andteachers’viewsweremoreambiguous.

Comingtoschoolhungryandwithnopocketmoneyiscloselylinkedtothepupilbeingunhappyand isdepictedasdrivinga seriesofbadbehaviours.Asnoted in theprecedingsections,pupilsfrompoorerhouseholdsweresignificantlylesslikelythanpupilsfromricherhouseholdstoachievetestscoresattheexpectedgradelevelinEnglishliteracy,numeracy,andscience.Pupils’accountsare suggestive of the causal links that might underlie this pattern. They depict a vicious cyclelinkinghouseholdpoverty–manifestedparticularlybycomingtoschoolhungry,withoutmoneytobuyfood,andwithoutaproperuniform–toexclusionfromlearningandbroadersocialexclusion.

3.4.4 Pupils’attitudestowardstheirschoolandteachers

Pupils did not articulate in detailwhat they liked or disliked about teachers, except to identifysome teacherswhowere particularly violent in theway they punished pupils, and otherswhomade lessonsmore funbyusingactivities like singingor jokes.Asmentionedabove, thehappychilddescribedbypupilsisonewhoislovedbyhisorherteachers,whileanunhappychildisonewhoisoftenhurtphysicallyorhumiliatedbytheteachersfornotbeingabletoanswerquestions.Thequotesbelowillustratethis.

Pupils:MalamA---,hebeatusalot,heusedcableandothersophisticatedinstrumentstobeatuswithit,wethankGodtheytransferredhimtosecondaryschoolnow.[Boys’FGD]

Interviewer:Let’stalkabouthis[theunhappychild’s]relationshipwithteachers,howdohisteacherstreathim?Pupils:Iftheyaskaquestionandhecouldn’tanswer,teachersusetobeathimandgivehimallsortofpunishmentlikefrogjumpandtherest,andsometimestheteachersthreatenedtolockhimupintheofficefortwodays.[Boys’FGD]

Mostpupils’accountsrevealedanacceptanceofthefactthatteachersoftendidnotcometoclassontime,oratall.While itwasrareforpupilstocriticisetheirteachersdirectly,theywerefrankabout classes not being held regularly, and occasionally, as in the quotation below, recognisedhowbadlythiswasaffectingtheirlearning.AsexplainedinSection4below,oneinseventeacherswere typically found to be absent during the quantitative study. In the FGDs, pupils toldresearchers that when teachers were absent they would copy notes from the previous day orcompletehomework,orjustwait.Inobservationsofsomeclasseswherenoteacherwaspresent,pupilsplayednoisily.


©EDOREN 29

Interviewer:Doyouwantyourteacherstobechanged?Pupils:Wewanttheoldteacherstobechangedbecausetheysleepandtheydon’tenterclasseveryday.Interviewer:Oldasinhow?Agewise,ortheystayedlongerintheschool?Pupils:Theystayedlongintheschool,sowewantnewteachersthoughourHausateacherisnewbutshehasneverenteredourclass.[Boys’FGD]

Pupils in schools with poor infrastructure were conscious of the problems this caused for theteachers,andwerenotthemselvesinapositiontodoanythingaboutthis.Whenaskedwhattheywoulddo if they couldmagically transform their schools,mostpupils said theywould addnewblocks,repairtheclassrooms,addconcretefloorsinschoolsthatdidnothavethem,oraddagateorwall.Onegroupof children ina rural schoolwished that their schoolwouldbecome ‘likeanurban school’, showing that theywere aware that not all schools suffered similar problems totheirs. More often, though, pupils recognised the effects of the infrastructural issues on theirlearningbutcoulddolittleotherthantoacceptthesituation:

Interviewer:Youfindmathseasytounderstand?Pupils:Nonotverywell.Interviewer:Whydon’tyouunderstanditverywell?Pupils:Theboardhasaholesowecannotseeverywell.Interviewer:Isitonlythemathsteacherthatwritesontheboardwithaholeoralltheteachers?Pupils:Somedividetheboardintotwoandwriteonthepartwithoutahole.Interviewer:Don’tyoutellthemyoucannotsee?Pupils:Theysaidweshouldn’tmakenoiseinclass[Girls’FGD]

Aswill be explained in the following sections, pupils typically spent a lot of time copyingnotesfrom the board. While some were able to understand and read out the notes that they hadwritten,mostwerenotable to identifybasic information suchas thedateor title. Somepupilswerenotsurewhichexercisebookrelatedtowhichsubject.Thissuggestsnotonlythatlearningoutcomesarepoor,butthatpupilsspendmuchoftheirtimeonfutileexercisesthatdonothingtoaddresstheirlearningneeds.


©EDOREN 30

4 Teachereffectiveness

InordertounderstandhowtoimproveteachinginnorthernNigeriaitisimportanttounderstandhow effective teachers currently are, the factors that might enable them to become moreeffective,orprevent them fromdoing so, and the context inwhich theyareworkinggenerally.This analysis is guided by examining assumptions regarding four key aspects of teachereffectivenessthatouranalysissuggestsunderpinsthein-serviceinterventionbeingstudied.Priorevidencesuggeststhattheseassumptionsneedtoholdfortheprogrammetoworkasexpected.Iftheydonothold,theprogrammewillneedtotakethis intoconsideration,andperhapsinsomecases,reviseitsTOCordesign.Theseassumptionsare:

• Subject knowledge: Teachers have the foundational subject knowledge to be able tounderstandtrainingandcurriculummaterials(Section4.1).

• Pedagogy: Teachers have a foundation of pedagogical skills to be able to apply newtraining; and they have feedback mechanisms that enable them to reinforce positivelearningandcorrectmistakes(Section4.2).

• Motivation: Teachers aremotivated –whether through external incentives like salaries,promotions,etc.ortheirowncommitment–toattendschoolandlessonsregularly,totryto improve their teaching, to take part in learning opportunities and to apply newknowledgewhentheygetit(Section4.3).

• Context: Teachers have access to sufficientmaterials, their class sizes are not too large,and the social and political context does not prevent the adoption of new teachingpractices(Section4.4).

ThissectionofthereportexaminestheseassumptionsinturninordertounderstandhowlikelyitisthattheTOCfor in-servicetrainingwillberealisedinpractice. Itdrawsondatafromboththequantitativeandqualitativebaselinestudiesconductedin2014and2015.SomebasicbackgroundinformationontheteachersisprovidedinTable9.

Table9Teacherbackgroundcharacteristics

Forteachers(N=908)theaverage… Oftheseteachers…

• …ageis37years.• …teachingexperienceis12years.However,

thereisalargevariationinteachingexperience,rangingfromlessthanoneyear(10thpercentileofteachers)upto36years(90thpercentileofteachers).

• …workexperienceinhercurrentschoolisaboutfiveyears.

and

• …teachersaremorelikelytoteachboththelowerprimaryGrades1,2and3andupperGrades4,5and6thantospecialiseineitherthelowerortheuppergrades.

• …only18%arefemale.• …themajority(67%)holdanNCE

qualification.• …48%reportedreceivingin-serviceteacher

trainingduringthelasttwoyears.

Source:Quantitativeimpactevaluationbaselinesurvey(October2014)teacherinterviews.


©EDOREN 31

4.1 Teachers’subjectknowledge

Forpupilstobeabletolearnwhileintheclassroom,teachersmusthavesufficientknowledgeofthesubjecttheyareteaching.Recentstudiesfromsub-SaharanAfrica,includingforsomestatesinNigeria,findthatfrequentlyteachersdonothave‘adequate‘workingknowledge’ofthesubjectstheyteach’anddonothave‘asufficientlevelofliteracytoreadforinformationfromoneormoresources,tointegrateinformation,andtosummariseitinaform,eitherwrittenorverbal,thatcanbepresented to studentsorused in theplanningof lessons’ (Johnson2010; JohnsonandHsieh2014;OPM2015).

ThequantitativebaselinesurveyestablisheslevelsofteachersubjectknowledgeinEnglish,mathsandscience,andofteachers’knowledgeofhowtoassesspupils’academicprogresspriortothestart of theTDP.Aspart of thequalitative research, teachers andother respondentswere alsoaskedabouttheissueofteachers’subjectknowledgeandwhethertheysawitasaconstrainttotheir effectiveness as teachers. The aimwas to understandwhether they acknowledged issuesaroundsubjectknowledgeandhadanystrategies fordealingwiththem.Afull listofevaluationquestionsispresentedinTable10below.

Table10Evaluationquestionsonteachersubjectknowledge

Question Sourceofdata

What are current levels of teacher subject knowledge on the primaryschoolcurriculuminEnglish,mathsandscience? Quantitative(TDNA)

Are there any differences in subject knowledge between treatment andcontrolgroupsofteachers,andbygender? Quantitative(TDNA)

Do teachers, head teachers, LGEA officers and TDP TFs understand theissuesaroundteachersubjectknowledge?

Qualitative(interviewsandFGDs)

Howdoteacherswithpoorsubjectknowledgemanageintheclassroom?Doteachersthemselves,headteachers,LGEAofficersandTDPfacilitatorshave strategies or ideas for ensuring that teachers have the subjectknowledgetheyneed?

Qualitative(lessonobservations,interviews)

4.1.1 Measuringteachersubjectknowledge

The estimates of teacher subject knowledge come from a TDNA administered at examinationcentrestotheheadteachersandthreeteacherssampledfromTDPtreatmentandcontrolschools.DrawingfromJohnsonandHsieh(2014),theTDPTDNAcoveredGrade4levelsubjectknowledgein English, maths and science, and also tested teachers’ ability to assess and monitor pupils’academic progress. The TDNA had four parts, as described below and in Table 11. The TDNAinstrumentisexplainedinmoredetailinSection3(VolumeII)ofthisreport.


©EDOREN 32

Table11MappingofTDNAtopics,exercisesandquestions

Subject Topicscovered

English Readingforfactualinformation,interpretingwords,phrasesorsentences,writingamodelletterinEnglish

Maths Addition,subtraction,multiplication,division,fractions,decimals,time,measurement,unitconversions,readinggraphs,squareroots,exponents

Science Thelifecycleofinsects,changesinweather,functioningofthehumandigestivesystem,measurement,theprocessofevaporation,formsandsourcesofenergy

Assessingandmonitoringpupils’academicprogress

Analysisofpupiltestsscores,includingmakingsimplegraphstomonitorperformance,shortassessment,pointingoutthestrengthsandweaknessesineachletter.Commentingonthefollowing:purpose,organisation,grammar,spellingandpunctuationTheseresultsareexaminedinthefollowingsectiononteachers’pedagogicalskills.

Source:TDPbaselinesurvey(October/November2014),TDNAinstrument.

4.1.2 TDNAlevelsofachievement

Toexaminedifferences in teacher subjectknowledge, fourachievement levelsweredefined fortheGEPTDNAs(JohnsonandHsieh2014),andthesewereusedfortheTDPTDNAtoo(Table12).AchievementLevel1capturesteacherswhohave‘sufficientprofessionalknowledge’(TDNAscore75%–100%):thatis,theymeetthebenchmarkminimumknowledgethresholdandareconsideredtobeeffective in theclassroom.AtachievementLevel2are teacherswhohave ‘near-sufficientprofessional knowledge’ (TDNA score 50%–74.9%) andwhowould benefit from some in-schoolsupport and in-service training tomake them effective in the classroom.Achievement Level 3includes teachers who have ‘emerging professional knowledge’ (TDNA score 25%–49.9%) andwho, tobeeffective,would requirea combinationof school-based in-service trainingandmorefundamentalprofessionaldevelopment.Finally,teachersatachievementLevel4(TDNAscore0%–24.9%),have limitedprofessional knowledgeandwouldneed substantial and sustained trainingandsupportinordertobecomeeffectiveintheclassroom.

Table12TDNAlevelsofachievement,descriptorsandscoreranges

Achievementlevel Descriptor TDNAscore(%)

Level1 Sufficientprofessionalknowledge 75<=score<=100

Level2 Near-sufficientprofessionalknowledge 50<=score<75

Level3 Emergingprofessionalknowledge 25<=score<50

Level4 Limitedprofessionalknowledge 0<=score<25

Source:(JohnsonandHsieh2014)Note:Scoressignifypercentageofquestionscorrectlyanswered

4.1.3 Results:Teachers’subjectknowledgeinEnglish,mathsandscience

Table13showsthatlevelsofteachersubjectknowledgeinEnglish,mathsandsciencearelow:theaverageTDNAscoreforscienceis21%,forEnglishitis23%,andformathsitis45%.FemaleandmaleteachersperformsimilarlyinEnglish(averagescoreof22%,comparedto23%)andinscience


©EDOREN 33

(averagescoreof23%,comparedto22%),but femaleteachersperformsignificantlyworsethanmaleteachersonmaths(averagescoreof43%,comparedto45%)inthethreestateseventhoughthisdifferenceisrelativelysmall.Bycomparison,theESSPINcompositesurveyconductedin2014,which covered six states – Jigawa, Kaduna, Kano, Enugu, Kwara and Lagos – found that femaleteachers performed significantly better than male teachers on English and maths, but thesegenderdifferencesaredrivenbyfemaleteachersinthesouthernstatesandarenotpresentinthenorthern states (De and Cameron 2015). There are no significant differences in average TDNAscores for English,maths or science between teachers in treatment and control schools,whichsuggeststhatteachersubjectknowledgelevelsacrossthetwogroupsisbalanced.

Table13Teacherandheadteachersubjectknowledge:%ofcorrectanswers

Indicator Overall Treatment Control Male Female N

Teacherandheadteachersubjectknowledge(logframeOutcomeIndicator1.3)

TDNAscores(%correctanswers,maxscore100%)

English 23.1 22.8 23.4 23.2 21.9 1158

Maths 45.1 44.8 45.4 45.4* 42.7 1158

Science 21.7 21.2 22.2 21.5 22.7 1158Source:Quantitativebaselinesurvey(October2014),TDNA.Note:(1)Estimatesareforteachersandallheadteachers;(2)statisticallysignificantdifferencesbetweengroupsaremarkedwithasterisks:*significantatthe10%level;**significantatthe5%level;***significantatthe1%level.

TheTDNAfindsthatalargemajorityofteachersdonothavesufficientornear-sufficientsubjectknowledgetoteachEnglish,scienceormathseffectively.Applyingthefourachievementlevels(asgiven in Table 12), it is evident that teacher subject knowledge is very weak across all threesubjects,andmoresoinEnglishandsciencethaninmaths(Figure7).

Only0.1%ofteachershavesufficientsubjectknowledge inscienceand0.4%(5outof1,158) inEnglish.Thegroupofteacherswithsufficientsubjectknowledgeinmathsislargerat8%(90outof1,158),butthisstillmeansthatfewerthanonein10teachershavesufficientsubjectknowledgeofmathstobeconsideredeffectiveintheclassroom.Thegroupofteachersthathavenear-sufficientsubjectknowledge,meaningthattheycouldwithsometargetedsupportandtrainingmoveintothegroupof teacherswithsufficientknowledge, isalsoverysmall.ForEnglishonly5%,and forscienceonly4%,ofteachershavenear-sufficientknowledgetobeeffectiveintheclassroom.Thiscompares to a much larger group for maths, for which 33% of teachers have near-sufficientknowledge.Therearelargegroupsofteacherswithonlyemergingsubjectknowledgeforallthreesubjects:43%ofteachersformaths,42%forEnglishand33%forscience.Furthermore,thegroupsofteacherswhohavelimitedprofessionalknowledge,andwhowithoutextensiveandcontinuoustraining and support are unlikely to be considered effective in the classroom, comprise 63% ofteachersforscienceand53%forEnglish.Althoughformathsthegroupofteacherswith limitedsubjectknowledgeisnotablysmaller(17%),thisstillmeansthatnearlyone-in-fiveteachersinthetreatmentandcontrolschoolsareunlikelytobeabletoteachmathseffectivelywithoutextensiveandcontinuoustrainingandsupport.


©EDOREN 34

Figure7Teachers’subjectknowledgebylevelofachievement

Further examination of teachers’ subject knowledge shows that performance is significantlycorrelatedacrossthethreesubjects.That is, ifateacherdoesrelativelywellononesubjectshealsotendstodowellontheothertwosubjects.27,28Thereverseisalsotrue:teacherswhoperformrelatively poorly on one particular subject tend to do relatively poorly on the remaining twosubjects.State-levelbaselineresultsforteachersubjectknowledgearediscussedinBox8belowandareavailable inmoredetail in the individualstatereports (De,Pettersson,etal.2015a;De,Pettersson,etal.2015b;De,Pettersson,etal.2015b).

27CorrelationcoefficientsformathsandEnglish,andformathsandscience,TDNAscoresaresignificant,andareabout0.6inbothcases.28TheTDNAwasadministeredinEnglishandthereforethereisapossibilitythatitnotonlyassessessubjectknowledgeandabilityto comprehendpupil progress but, to someextent, teachers’ English language skills too. Themaths sectionwouldbe relativelyunaffectedbythis,as itemswerenumerical,withminimal instructions(inEnglish).TheEnglishexercisesaimedtoassesssubjectknowledgeofEnglishandneededtobeadministeredinEnglish.Forthescienceandpupilassessmentexercises,thelanguagecouldpotentiallybeanissueinthattheseitemsmightseemliketestsoflanguagecompetencyinEnglish,ratherthantestsofknowledgeof scienceorability toassesspupils.Nonetheless, theofficialmediumof instruction inNigeria isEnglish fromGrade4onwards(FederalRepublicofNigeria2004),andthusteacherswouldbeexpectedtounderstandthebasicassessment instructions in theTDNAandtobeabletoanswerquestionsinEnglish.

62.9

17

53.3

32.7

42.7

41.6

4.3

32.6

4.8

0.1

7.8

0.4

0 20 40 60 80 100

Science

Maths

English

Percentageofteachers(N=1158)

Teachers'subjectknowledgeislargelyinsufficientinallthreesubjects

Level4 Level3 Level2 Level1


©EDOREN 35

Box8BaselineresultsforteachersubjectknowledgeinJigawa,KatsinaandZamfara

• Teachersubjectknowledgeisweakinallthreestates• ButteachersperformrelativelybetterinmathsthaninEnglishandscienceinallthreestates:

o theaveragescoreinEnglishis28%inKatsina,22%inJigawaand19%inZamfara;o theaveragescoreinscienceis23%inKatsina,22%inJigawaand20%inZamfara;ando theaveragescoreinmathsis49%inKatsina,45%inJigawaand40%inZamfara.

• Theseresults,althoughnotdirectlycomparable,areroughlyinlinewiththosefrompreviousassessmentsofteacherdevelopmentneeds.TheaverageTDNAscoreformathswas41%,andforEnglishitwas18%,inJigawain2010(Johnson2010)andinKatsinain2014theaveragemathsTDNAscorewas48%,andforEnglish8%(JohnsonandHsieh2014).



4.1.4 Perceptionsaboutteachers’weaksubjectknowledge

Given these findings on teachers’ limited subject knowledge in the quantitative survey, a keyquestion for the qualitative research was whether teachers themselves, head teachers, LGEAofficers,andTDPTFs,recognisethisasaconstraintandhavestrategiesfordealingwithit.

Therewasvariedbutlimitedacceptanceamongrespondentsthatlackofsubjectknowledgewasanissue.Theirresponsesmay,tosomeextent,reflectasenseofembarrassmentorareluctanceto criticise. Nevertheless, the lack of awareness of the problem or of possible solutions ispotentially problematic for attempts to improve teachers’ subject knowledge. Some TFsacknowledgedthatteachers’limitedabilityinEnglishwassometimesaproblemduringtraining,orthattheirteachersweredifficulttotrainbecausetheylackedtheexpectedfoundationalskillsorweresimply‘lazy’and‘incompetent’.

Some LGEA education secretaries claimed that there was no problem with teachers’ subjectknowledge. One had observed teachers with limited subject knowledge writing incorrectinformation on the blackboard, and argued that teachers undertaking their NCE by distancelearning(comparedtotheclassroom-basedNCE)resultedinworseteachers.Someheadteachersclaimedthattherewasnoissuewithteachers’subjectknowledgebecausetheyhadall,ornearlyall,achievedNCE,whileothersadmittedthiswasnotamajorconcernforthem.Accordingtoonerespondent‘therearesometeachersthatcannotreadouttheirnameseventhoughtheyhaveanNCE’ [TDP teacher], including somewhohaveobtained theirNCE fraudulently.Onedescribedateacherwholackedsubjectknowledgeandwas‘notwillingtolearnevenwhenyouoffertoteachhim’[headteacher].Thisteacherwasplacedinthefirstgrademathematicsclasstoteach‘somebasicthings’.

Few teachers acknowledged their own difficulty in terms of subject knowledge, although onementionedusingadictionarytolookupdifficultwordsandonevisitedteachersatotherschools‘to teach me the subject I don’t know’ [non-TDP teacher]. In the teachers’ constraint rankingexercise,teacherstendedtoplace‘poorteachersubjectknowledge’inthemiddle–meaningtheywerefairlyambivalentabouttheseverityoftheproblem.

Teachers did, however, discuss the poor subject knowledge of other teachers, especially thosewhohadundertakentheirNCEbydistancelearning.Oneteacherhadbeenwarnednottomixup


©EDOREN 36

lowercaseanduppercaselettersinEnglish,butdidnotappeartohavebeenofferedanyfurthersupport to strengthen his writing ability. Some teachers acknowledged that they may notunderstandeverything in the curriculum,particularly if theywereasked to teacha subject thatwasnottheirusualsubject.Theycomplainedofnothavingbookstoresearchsubjectstheydidnotknow,ortheysaidthatinsuchcasestheydidnotreceivesupportfromotherteachersortheheadteacher.Tosomeextentitisreassuringthatinthesecases,teachershadatleastidentifiedaneedforsupport,buttheystilldidnotreceivethesupporttheywanted.

Inmyownopinionthereisaneedtoaddresspoorsubjectknowledgeoftheteachersaswell as the curriculum, to be honestwith you 35%–40%of teachers have issues andchallenges with their subject knowledge, you will find such cases in a seminar andworkshops,many of us cannot present a simple thing, there is a need to boost theconfidenceoftheteachersandmakethemmoreeffective.[TDPteacher]

Whenaskedwhetheritwascommonforteacherstobeteachingsomethingincorrectlybecauseofamistake in the textbook,one teacher said ‘somethingof thisnaturehappened tooneofourformerteachers,andattheendtheteacherrealisedhewasjustteachingrubbishbecauseofthatmistake in the text book’ [TDP teacher]. Teachers do not appear to have either the subjectknowledge or pedagogical skills to deviate from the textbook, evenwhen the textbookmay attimesbeinappropriateorcontainerrors.

Table14Summaryofstrengthofevidenceunderpinningassumptionsrelatingtoteachers’subjectknowledge

Assumptions Assumptionsatisfied

Strengthofevidence

Subjectknowledge:Teachershavethefoundationalsubjectknowledgetobeabletounderstandtrainingandcurriculummaterials

• Teachershavethefoundationalsubjectknowledgetobeabletounderstandtrainingandcurriculummaterials

No Strong

• Teachers,headteachers,andLGEAofficialsunderstandthechallengesencounteredbyteachersregardingtheirsubjectknowledgeanddemonstratepossiblewaystoaddressthis

Mixed

4.2 Pedagogicalskills

Teachers spend a considerable proportion of their time in the classroom and their teachingpractices influence pupil learning (Bayer et al. 2012). Thismakes it important to understand ifteachersuseteachingpracticesthatareconducivetopupillearningintheclassroom.Thereexistssome literaturewhich discusses a set of teaching practices considered to characterise effectiveteachers and classroom practice (Siraj et al. 2014;Westbrook 2013).29 These practices include:organising teaching timewell; having shared and clearly stated goals,with guidanceonhow toachievethem;usinghomework;creatingapositiveclassroomclimate;managingpupilbehavioureffectively; using group work for collaborative and peer learning; using personalised learningbased on pupil needs; appropriate use of teaching resources; using open-ended questions todevelop a deeper understanding of concepts and to encourage pupils to participate; usingcontinuousassessmentofpupilacademicprogress;andusingplenariesat theendof lessons to

29ThesepedagogicalpracticeshavebeenprovidedasexamplesonlyandarenotmappedtothepedagogicaltechniquesdeliveredthroughTDP’straining.


©EDOREN 37

summarise and repeat key learning points. A related question is whether teachers have theknowledgerequiredtoanalyseandmonitorpupilacademicprogressinordertoaddresslearningneeds. Evidence from selected states in Nigeria find that teacher knowledge in this area isgenerallyweak(JohnsonandHsieh2014).

TheTDPwillprovidein-servicetrainingtopromoteeffectiveteachingpracticesaimedatmovingfrom predominantly teacher-centred to more pupil-centred practices. To understand teachingpracticesbeforetheTDPbegins,andtobeabletomeasureanychangesinteachingpracticesovertimethatareattributabletotheprogramme,thebaselinequantitativesurveyexaminedteachingpractices at the baseline using timed classroom observations in the quantitative survey. Inaddition, qualitative lessonobservationswere conducted, inwhichobserverswatcheda lesson,madequalitativenotesonthewaysinwhichteachersandpupilsinteracted,andlaterinterviewedthe teacher about his or her methods of teaching. This section seeks to address a range ofquestionsaroundpedagogyandassessment(seeTable15).

Table15Evaluationquestionsonpedagogy

Question Sourceofdata

Whatarecurrentlevelsofteachereffectivenessintheclassroom?Quantitativeandqualitative(lessonobservation)

Aretheredifferencesinteachereffectivenessintheclassroombetweentreatmentandcontrolgroupsofteachers,andbygender?

Quantitative(lessonobservation)

Whatareteachers’ownperceptionsabouteffectivemethodsofteaching? Qualitative(interview)Whatproportionofteachersknowshowtoassessandmonitorpupils’academicprogress?

Quantitative(TDNA)

Isthereanydifferenceintheproportionsofteacherswhoknowhowtoassessandmonitorpupils’academicprogressbetweentreatmentandcontrolgroups,andbygender?

Quantitative(TDNA)

Whatareteachers’ownperceptionsabouttheirpupils’performance?Whatdoteacherssayabouthowtheymonitorandassesspupilperformance?

Qualitative(interviewandFGD)

4.2.1 Results:Teachers’positiveinteractionwithpupilsduringlessons

ThebaselinesurveyincludedobservationsofTDPorcontrolgroupteachersteachingalesson.Theobservationrecordedtheproportionoftimethattheteacherwasengagedininteractionthatwasclassed as effective (see Section 3 in Volume II for details of the classroom observationinstrument).

Theaverageteacher inTDPtreatmentandcontrolschools involvedpupils inpositive interactionfor 24% of the total lesson time (Table 16). This suggests that there is considerable scope tointroducemoreeffectiveteachingpracticesintheclassroom.Maleteachersonaverageengagedpupils in positive interaction for a larger proportion of total lesson time than female teachers,significantatthe10%level.Therewerenosignificantdifferencesbetweenteachersintreatmentandcontrolschools.


©EDOREN 38

Table16Teachers’positiveinteractionwithpupilsduringlessons(meanestimates)

Indicator Overall Treatment Control Male Female N

Teacherpedagogy(logframeOutcomeIndicator1.1)

Timeteacherinvolvespupilsinpositiveinteractionduringlesson(%oftotallessontime)

24.2 24.0 24.5 24.5* 22.9 1054

Source: Quantitative impact evaluation baseline survey (October 2014), classroom observation. Note: (1) Theabove indicators include teachers as well as head teacherswho teach a primary class regularly; (2) statisticallysignificantdifferencesbetweengroupsaremarkedwithasterisks:*significantatthe10%level;**significantatthe5%level;***significantatthe1%level.

Additionalanalysisofteachers’pedagogicalskillsisdiscussedinVolumeII,includingteachers’useofpraiseandreprimands,teacherresourcesintheclassrooms,presenceofmulti-gradeteaching,andteachers’actionsattheendofthelesson.

4.2.2 Results:Lessonlengthandlossofinstructionaltime

An additional result of this baseline survey relates to lesson length:most lessons tended to beconsiderablyshorterthanthestandard35minutes.30Box9discussesthelossofinstructionaltimeduetoreducedlessonlength,whichinmanycasesissubstantial,andhasconsequencesforpupillearning.

30ThelessonlengthcriterionusedherehasbeenborrowedfromtheESSPINlogframe,whichconsidersastandardlessontofinishwithinfiveminutesofastandard35-minutelesson(i.e.lessonlengthis30–40minutes)(Cameron2015a).


©EDOREN 39

Box9Lossofinstructionaltimeduetoshorterlessonlength

The amount of instructional time pupils receive from their teachers affects the amount of learningthat can occur during a lesson (Robinson, Lloyd, and Rowe 2008). In the three states, the averagelesson length was 28 minutes, and a large proportion of observed lessons (more than 45%) wereshorter thanwithin fiveminutes of a standard 35minute lesson (Figure 8), implying large losses ininstructionaltime.

Around15%oflessonswere18minutesorshorter,equivalenttolosingnearlyhalfofthelessontime,whichimpliesasubstantiallossofinstructionaltime,withseriousimplicationsforpupillearning.Thisloss of instructional time is additional to losses due to teachers being absent from school and toteachers who are present at school and scheduled to teach but who are not in the classroom(classroomabsenteeism,notmeasuredbythesurvey).

About54%of lessonsmetthecriterionof lastingforat least30minutes;andsomeoftheselessonsmay have even also lasted longer (the classroom observation instrument was designed to stoprecordingclassroomactivitiesafterthe36thminute).

Figure8Lessonlengthisfrequently(much)shorterthanexpected

Source:Quantitativeimpactevaluationbaselinesurvey(October2014),classroomobservation.Note:(1)N=1,054allteachersandheadteacherswhoteach;(2)excludeslessonsthatwerenineminutesorshorter.

Lessonsthatwereshorterthannineminutes(fewerthan2%oflessonsobserved)areexcludedfromFigure8andtheanalysis.Thiswasdonetoaddressthepossibilitiesthatinsomecasesteachersmayhaveshortenedlessonsbecausetheywerebeingobservedorbecauseofenumeratorshavingarrivedafter the start of the lesson. Excluding these cases did not significantly alter the results presentedabove.Moreover, theaverage lesson lengthof28minutes isconsistentwithanalysiscarriedoutbythis report’s authors on classroom observation data for northern Nigerian states covered by theESSPINcompositesurvey,whichfoundan(unweighted)averagelessonlengthof26minutesinJigawa,28minutesinKadunaand30minutesinKano.

Box10containsasummaryofstate-levelbaselineresultsforteacherpedagogyinJigawa,KatsinaandZamfara.

2.8%5.4% 7.0%

10.6% 9.4% 11.2% 11.8%

5.2%

36.6%

0%

5%

10%

15%

20%

25%

30%

35%

40%

12 15 18 21 24 27 30 33 36

Percen

tageoflessonsobserved

Lessonlength(minutes)


©EDOREN 40

Box10BaselineresultsforteacherpedagogyinJigawa,KatsinaandZamfara

• Teachers spent amere 24%–25% of total lesson time in positive interactions with pupils in allthreestates.

• However,themajorityofteachersappeartocreateapositiveclassroomclimate:o InZamfara,88%ofteachersusedpraisemorethanreprimands,inJigawa78%ofteachers

didsoandinKatsina72%didso.• Theaveragelessonlengthwas28minutesinallthreestates,whichisshorterthanthestandardof

35minutesandindicatesanotablelossofinstructionaltimeTheseimpactevaluationsurveyresultsarerepresentativeoftheTDPtreatmentandcontrolpopulations.However, thesepopulationsarenot inthemselvesrepresentativeof thethreePhase1TDPstatesmorebroadly.Foradetaileddiscussionofthebaselineresultsatstatelevelseetheindividualstatereports(De,Pettersson,andMorris2015;De,Pettersson,etal.2015a;De,Pettersson,etal.2015b)

4.2.3 Teachers’perceptionsoftheirpedagogicalskills

In the qualitative study, teachers were in many cases unable to articulate in any detail whatmethods theyuse for teachingpupils and ensuring that they learn. Exceptionswere theuseof‘energisers’orexercisestokeeppupils interested;usingacontainerofsoiltodemonstratewhatsoil is; and using song to capture pupils’ interest and help them to learn faster. Someof thesemethodsmay have been picked up during the early TDP training, or in Jigawa from ESSPIN, orother formsof pre- and in-service training, ormayevenhavebeen the result of teachers’ owninitiative.Aswithsubjectknowledge,teacherswhohadgonethroughNCEwereoftenconsideredtohave‘allthenecessarymethodology’.

Inparticular,teachersdidnotdemonstrateideasabouthowpupilslearnandabouthowteacherscanrespondtodifferentlearninglevelswithinthesameclassroomtoensureallpupilsachieveatleastaminimumlevel.Whenaskedabouthowtheywouldhelpapupilwhodoesnotunderstand,some said their pupils were simply untalented; others expressed frustration with having trieddifferentteachingtechniqueswithoutsuccess;and(perhapsmostcommonly)somesaidthattheywouldrepeatthesameinformationuntilthepupilabsorbsit.Oneteacherdescribedastrategyofplacingpupilsingroupswithone‘talented’pupilwithineachgroup.

INTERVIEWER:Ok,whatdoyouunderstandbyuntalentedpupils?TEACHERS:WhatIunderstandiswhenyouteachapupilandhedoesnotunderstand,honestlywehaveafewofthemthatareuntalented,theyhardlyunderstand.INTERVIEWER:Likewhenyougivethemwork,cantheydoit?TEACHERS:Whenyougivethemworksometimestheycan’tevenwrite,youhavetoholdtheirhandstoshowthem.[Non-TDPteacher]

INTERVIEWER:Towhatextentdoyoufeelthetopicsyoucoveredinthelessonsyoutaughtpreviouslyarewellunderstood?TEACHERS:IamalwayshappytorevisewiththepupilswhatItaughtthem,buttherearesomepupilswhoneverunderstandwhatyouteachnomatterhowyouteachthem.INTERVIEWER:Howdoyoutreatthosepupilswhoneverunderstand?TEACHERS:Ialwaystrymybesttograduallygettheirattentiontilltheyunderstand.INTERVIEWER:Sir,howdoyougettheirattention?TEACHERS:IdothatbyrepeatingseveraltimesoneaftertheotheruntiltheyunderstandwhatIamteachingthem.[Non-TDPteacher]


©EDOREN 41

Teachers spokemostly in Hausa during lessons, interspersed with sections of English from thetextbook.Veryfewteacherscouldcorrectlystatethefederalgovernment’slanguagepolicy,whichistoteachinthelanguageoftheenvironment,i.e.mothertongue,fromGrades1to3andtheninEnglishfromGrade4onwards.TeachersinsteadbelievedthatthepolicywastoteachinEnglish,and were apologetic that they had to switch to Hausa to make the pupils understand. Someteachers said they read everything in English and then repeated it in Hausa,which seemed aninefficientuseofinstructionaltime.Someoftheteachers’frustrationwithpupilswhowereunabletounderstandalessonmayhavebeencausedbyteachinginEnglish,withanemphasisonwrittenEnglishinparticular,whenpupilswereunabletorecogniseEnglishwordsinwritingorspeech.Thelesson observations confirmed that there was a large amount of repetition of set phrases inEnglish,usuallycopiedfromthetextbookontotheboard,withpupilscopyingthesametextattheendoftheclassafterhavingrepeateditseveraltimes.

Teachers, particularly thosewho had received training from ESSPIN or TDP, talked about child-centredlearningorthe‘playway’method,butoftenseemedtounderstandthismainlyintermsofusingimaginativelessonmaterials,suchasmoney(formathematics)orwaterorsoil(forscience),or of using songs, poems and jokes. Even where teachers had adopted more child-centredmethods, they still tended to finish their lessonsbywritingnoteson theboardand leaving thepupilstocopythem,anactivitywhichtookalargeamountoftimeandmayhavereflectedaneedforteacherstodemonstratetoparentsthattheirpupilsaredoingsomethingproductiveinschool.In principle, pupils were also supposed to use their notes to prepare themselves for tests andexams, but the notes available for observation by researchers were often scant andindecipherable.

4.2.4 Doteachersknowhowtomonitorandassesspupils’academicprogress?

TheTDNAtest includedquestions formeasuring teachers’ (andhead teachers’)ability toassessand monitor pupils’ academic progress (see Section 4.1 above, and Section 3 in Volume II fordiscussionofitemscoveringthisskill).Theirabilityinthispartofthetestwasweak,theaveragescorebeing15%.Theaverageteacher,atthebaseline,isunabletocorrecterrorsinpupils’workandproviderelevantfeedback,ortoidentifylearningneedsofindividualpupilsandmonitortheiracademicprogressovertime.Therearenosignificantdifferencesbetweenteachersintreatmentandcontrolschools.

Examiningteachers’abilitytoassessandmonitorpupilprogressusingtheachievementlevels,theresultsareevenstarker.Thevastmajority(80%)ofteachersonlyhavelimitedknowledgeinthisarea (Figure9) andanswered less than25%of thequestions correctly.At theotherendof therange, a mere 0.3% of teachers have sufficient knowledge to assess and monitor pupilperformance.


©EDOREN 42

Figure9Abilitytoassessandmonitorpupils’progress

Theseresultsaresupportedbyfindingsfromotherassessments.InJigawain2010noteacherwasconsidered to have sufficient knowledge to identify pupil learning problems or use graphs andtables to monitor pupil progress, and in Katsina in 2014 only 0.2% and 1.5% of teachers hadsufficientknowledgeofhowtocorrectandprovidefeedbackonpupilwritingandmonitorpupilprogressrespectively(Johnson2010;JohnsonandHsieh2014).31

In the qualitative survey, only one of the nine schools was regularly conducting continuousformative assessment in the form of class tests. Elsewhere, feedback mechanisms by whichteacherscouldjudgetheprogressoftheirpupilswerelacking.Teachersclaimedtoknowwhetherpupils were learning or not from their test or class work. In the classroom observations (bothquantitative and qualitative) it was noted that teachers did spend a large amount of timecorrecting pupils’ notebooks. However, in qualitative observations they usually spent very littletime checking each notebook before ticking it and handing it back. Some teachers noted thedifficultyinmarkingworkproperlywhenclassesareverylarge:

Ifyoucollect137booksyoucan’tmarkthemallwithinoneperiod[headteacher].

From observation of notebooks of pupils, and from talking to the pupils about these notes, itappearedthattheydidnotknowhowtowrite lettersorthemeaningofwhattheyhadwritten.Thiswasthecaseevenwhenteachershadmarkedtheworkascorrect.Thissuggeststhatteachersareeitherunabletopayproperattentiontothepupils’work,forexamplebecauseoflargeclasssizes,orlacktheknowledgetomarktheworkappropriately,orboth.Theymayalsonothavethepedagogicalskillsandsubjectknowledgeneededtosupportpupilswhoselearninglevelsarelowrelativetotheexpectationssetinthecurriculum.

Table17Summaryofstrengthofevidencerelatingtoteachers’pedagogicalskills


Strengthofevidence

Pedagogy:Teachershaveadequatepedagogicalknowledgetobeabletoapplynewtechniqueslearntintraining;andtheyhavefeedbackmechanismsthatenablethemtoreinforcepositivelearningandcorrectmistakes

• Teachershaveafoundationofpedagogicalknowledgeonwhichtheycanbasefurtherdevelopment

No Strong

• Teachersknowhowtomonitorandassesspupils’progress No Strong

31Thesecomparisonsofresultsacrossthedifferentstudiesshouldbecaveatedbythefactthattheinstruments,althoughsimilarinstructure,hadasomewhatdifferentscopeandcontents.

80.2 14.7 4.8 0.3

0% 20% 40% 60% 80% 100%

Assessmentandmonitoringofpupil

progress

ProportionofteachersbyTDNAachievementlevel(N=1158)

80%ofteachersshowedlimitedabilitytomonitorpupils'progress

Level4 Level3 Level2 Level1


©EDOREN 43

4.3 Teachermotivationandattendance

TheTDPTOCarguesthatTDPmaterialsandsupportcanimproveteachers’motivationbyraisingteacherself-esteem(McCormick2014).Theevaluationframework(EDOREN2014)elaboratesonthismechanism,suggestingthatteachers’motivationisincreasedasteachersfeelmoreeffectiveand see their pupils’ learning outcomes improve. However, this link in the TOC assumes thatteachersaremotivated(externallythroughsalary,promotions,etc.orinternallythroughtheirowncommitment)toattendschoolandlessonsregularly,totrytoimprovetheirteaching,totakepartinlearningopportunitiesandtoapplynewknowledgewhentheygetit.Thequantitativebaselinesurveyadministeredateachermotivationinstrumenttothesampledteachersinbothtreatmentandcontrolschoolsinthethreestates,withasampleof1,077teachers.Inthequalitativebaselinestudy,27teacherswereaskedtotalkabouttheirmotivationinFGDsandone-on-oneinterviews.Thissectionpresentstheresultsoftheteachermotivationscale,andadiscussionofteachers’ownnarratives about the causes and effects of (lowor high)motivation. It then examines teachers’attendanceinschoolandintheclassroom,usingofficialattendanceregistersconsultedinschoolsduring thequantitativebaselinesurvey,observationsduring thequalitativestudy,and teachers’andheadteachers’viewsontheissue.Thissectionusesquantitativeandqualitativebaselinedatato examine teachers’ existing levels anddrivers ofmotivation aswell attendance in school andclassroom(Table18).

Table18Evaluationquestionsonteachermotivation

Questions Datasources

Whatarethecurrentlevelsanddriversofteachers’motivation?

Quantitative(teachermotivationscale)andqualitative(teacherFGDs,headteacherinterviews,casestudyinterviewsandself-directedphotography)

Doesteachers’motivationvarybetweenstates,betweenmaleandfemaleteachers,andbetweencontrolandtreatmentgroups?

Quantitative(teachermotivationscale)

Whatarecurrentlevelsofteacherabsenteeismfromschool?

Quantitative(teacherinterviewsandattendanceregisters)

Isthereanysignificantdifferenceinteacherabsenteeismfromschoolbetweentreatmentandcontrolgroupsofteachers,andbygender?

Quantitative(teacherinterviews)

Whyareteachersabsentfromschoolandclassrooms?

Qualitative(teacherFGDs,headteacherinterviews,casestudyinterviewsandself-directedphotography)

4.3.1 Currentlevelsofteachermotivation

For the purposes of the baseline survey, teacher motivation is defined as the propensity ofteachers to start andmaintain behaviours that are directed towards fulfilling their professionalgoals, and in particular towards achieving better learning outcomes for the school’s learners,basedonBennellandAkyeampong(2007)andHoyandMiskel(1991).Aspartofthequantitativebaselinestudyamotivationinstrumentwasdeveloped,relatingtodifferentaspectsofmotivation,


©EDOREN 44

thatcouldbeadministeredverballytoteachers(seeVolumeIIandCameron,2015b).Theresultingdatawereusedtocalculateanoverallmotivationscoreforeachteacher.32

Thisscoreisusefulforcomparingdifferentgroupsofteachers(Figure10).Teachermotivationissignificantly higher for NCE-qualified teachers than for teachers who do not have an NCEqualification, and is higher in Katsina than in Jigawa or Zamfara. Although significant, thesedifferencesaresmallinscale:forexample,themeanmotivationscoreinZamfaraislessthan0.3standard deviations below that in Katsina. Differences betweenmale and female teachers, andbetweenJigawaandZamfara,aresmallandstatistically insignificant.There isalsonosignificantdifferenceinoverallteachermotivationscoresfortheTDPtreatmentandcontrolgroups.

Figure10Overallteachermotivationscorebyqualification,genderandstate

Examining the relationship between overall teacher motivation and teaching experience,classroompedagogy,absenteeismandsubjectknowledgeprovidesinterestinginsights.Thereisnosignificant correlation between teacher motivation and experience, teacher motivation andpositive interaction with pupils in the classroom or teacher motivation and (self-reported)absenteeism(Figure11,seeCameron(2015b)forfullresults).However,therelationshipbetweenoverall teachermotivationandperformanceoneachpartof theTDNA–English,maths,scienceand ability to monitor and assess pupils’ academic progress – is small, though positive andsignificant.33

32Thescoreisnormalised–calculatedinsuchawaythatensuresithasanaverage(mean)of500andastandarddeviationof100.33Thelackofcorrelationbetweentheoverallteachermotivationscoreandabsenteeismandpositiveinteractionintheclassroomissomewhatsurprisingandwarrantsfurtherexploration.InsubsequentTDPresearch,multivariateregressionmodelscouldbefittedtoexplorethedatainmoredepth.Questionsforfurtherresearchmayinclude:arepupillearninglevelshigherforpupilstaughtbymoremotivatedteachers?WhyareNCE-qualifiedteachersandteacherswithbettersubjectknowledgemoremotivated?FurtherdatacollectionaspartoftheTDPendlinesurveywillmakeitpossibletoanswerquestionsofcausationwithgreaterconfidencebytesting,forexample,whetherpupiloutcomesimprovefasteriftheyaretaughtbymoremotivatedteachers.

470 480 490 500 510 520 530

noNCENCE

malefemale

JigawaKatsinaZamfara

treatmentcontrol

Teachermotivationscore(mean=500)

Teache

rcharacteristics


©EDOREN 45

Figure11Teachermotivationscoresandteacherbackgroundcharacteristics

4.3.2 Driversofteachermotivation

The ideathat receivingtrainingwould improvemotivationthroughteachers’ senseof theirowneffectiveness echoes insights from the theory of, and studies of, the psychology of self-efficacy(Bandura,1977,citedinFernetetal.,2008).Self-efficacy,definedastheperceptionofone’sownabilitytohaveaneffectondesiredoutcomes,isthoughttobeimportantinmotivation.Individualswhodonotseethemselvesasabletoaffecttheworldaroundthemareunlikelytoexpendmucheffort in trying todo so. If teachers’pedagogicalmethodsareamajor factorundermining theirself-efficacy,andself-efficacyisinturnimportantformotivation,thenimprovementsinpedagogywouldbelikelytoimprovemotivation.If,ontheotherhand,otherexternalfactors–forexample,poorinfrastructureorlackoflearningresources–meanthatteacherswillnotachievevisiblegainsineffectivenessevenafteradoptingnewpedagogicalpractices,thentheinterventionislesslikelytoaffecttheirmotivation.Similarly,iftheirmotivationiskeptlowbyfactorssuchaserraticpayorrulesaroundpromotionandtransfer,thentheeffectsoftheinterventiononmotivationarelikelytobelimited.

Teachersinseveralcasestalkedaboutfeelingdemotivatedwhenpupilsdonotunderstand.


©EDOREN 46

Teachers:Whatdiscouragesmeisthatwhenyouteachachildheclaimsheknowsbutwhenyouaskhimaboutithewilljuststandandsaynothingeventhoughsomemaygiveyouwhatyoutaught,butthisisfrustratingbecauseyoumaysometimestakealongtimetoexplainbutthechildrenarenotunderstanding.Itissoannoyingattimes,especiallywhenyougiveoutyourpossiblebest.

Teachersgavevarieddescriptionsoftheirattitudestowardstheteachingprofession,rangingfromthosewhodescribedteachingasanobleprofessionthatadvancedsociety,tothosewhoacceptedteaching jobs because theywere unable to find anything else.One teacher explicitly became ateacherbecauseitallowedhimtocarryonasidebusiness.

Idecidedtobecomeateacherbecausethejobissimpleandisnottimeconsuming.Igottwoappointmentsfromthebeginning,thatisNigeriaCustomServiceandthisteachingjob,IdecidedtoaccepttheteachingjobbecauseIhaveashopanditisnotpossibleformetomixthecustomsjobandmyshopbusiness,butteachingwillgivemeampletimetomanagemybusiness.Forexample,weopenthisschoolby7:00amandcloseafter1:00pm,thisshowsthatafterclosinghoursIwillhavetimeformyfamilyandmybusinessaswell.Wehaveholidaysunlikeotherjobs…[TDPteacher]

Some teachers cited respect towards teachers, the idea that teaching is an inherently valuableoccupation,andthesatisfactionofseeingchildrenlearn,asmotivatingfactors–particularlyintheabsenceofagoodorregularlypaidsalary.Teachingwasalsoseenasrespectableonthegroundsthatitisrelativelyfreeofcorruptioncomparedtootherjobsingovernment,‘awayofgettingpureandcleanmoney’ [TDPteacher].However,othersrefutedaltogetherthe ideathatteachingwasrespectedbyothers inthecommunity,placingtheblameonthegovernment’sattitudetowardsteachers or on the fact that they are not well paid. One argued that the teacher was still arespectedfigureinruralcommunities,butthatinurbanareasteachingisjusta‘steppingstone’toabettercareerprospect.

Interviewer:Okaydoyouseeyourselfasaprimaryschoolteacherlessimportantinsociety?Teacher:No,Iamveryimportanttosociety,eventhoughsomepeoplemightthinkwearenotimportant,butasfarasIamconcerned,Iamimportant,forexamplewearethegrassroots,everythingdependsonus….[TDPteacher]

Ihateteachingbecausepeoplenolongerrespectteachers;sometimessecuritypersonnelrespectstudentsmorethanteachers.Iwishthenewgovernmentwoulddosomethingaboutit.Manyservicesthatarenotliketeachinggetmorerespectandahighersalarythantheteachingprofession.Fromthebeginningithasbeenmyfatherwhowasinterestedintheprofession,notme.IamdoingitbecauseIdon’thaveanyotherjob[Casestudyteacher]

Inthefuture,howdoyouseeyourcareerprogressing?Teacher:Well,weareprayingforawayoutregardingtheteachingoccupation,becauseateacherisconsideredasnobody,thisisbecauseateachercanspendlikethreemonthswithoutsalaryandstillgoingtowork.[TDPteacher]

Teachersdescribedacareerpathforaprimaryschoolteacherthatwouldtakethemfirsttothepositionofheadteacher,andthentoapositioninlocalgovernmentandeventuallytotheSUBEB.Even among those who described teaching as an important and respected profession, many


©EDOREN 47

seemedtowanttoleaveteachingitselftoreachthesesupervisoryroles,whichclearlyheldhigherstatus.Asnotedabove,somewerepursuingfurtherstudies,sometimeswithaviewtoadvancingoutofprimaryschoolteaching.

When asked about demotivating factors, teachers predominantly talked about salaries,workingconditions, the lackof learning resourcesandpoor infrastructure.Salarycomplaintswereneverfarfromthesurface,althoughincreasesinsalaryinKatsinahad,reportedly,ledtoanincreaseinapplicantstobecometeachers,andanincreaseintheproportionwithNCEqualifications.Aswellas salary payment problems, teachers said theywere demotivated by the lack of promotion orperceived unfairness in promotions; arbitrary transfers; the poor state of infrastructure andteaching resources;over-crowdedclassrooms; irregularattendanceofchildren;and ‘untalented’pupils.Teacherscouldbediscouragedbyinspectionthatwastooharsh,butpositivelymotivatedby inspectionthatwasencouragingorconstructive.Lackof textbookswassometimescitedasademotivating factor, although researchers also saw schools with large numbers of unusedtextbookslockedupintheheadteachers’office.

Insomecasesanexplicitcausallinkwasdrawnbyteachersbetweendemotivatingexternalfactors– family problems or delayed salary – and teachers’ inability to ensure that pupils learn.Unmotivated teachers, it was suggested, would turn up and teach, but not ‘effectively orzealously’,andwithoutbotheringaboutwhetherpupilsareactuallylearningornot.

Teacher:Ifateacherhasalotofissueslikefamilyproblemsthiswillaffecthisperformanceandwhenhecomesintotheschoolhewillnotbeveryactivesohewillonlyteachasadutynotmindingwhethertheyunderstanditornot.[TDPteacher]

Teacher:Honestlyitisnotalways,especiallywhenthereisnosalaryandyoudon’thaveanythingthatwillhelpyoutofuelyourbikeandwhattogivetoyourfamily,whenyoucometoschoolwithoutapennyinyourpocketyouevenhatethejobandgototheofficethinking,youwillnotevenhavethezealtoworkwellevenwhenthepupilsarepresent.[TDPteacher]

4.3.3 Currentlevelsofteacherabsenteeism

Forpupil learning to takeplacemeaningfullyandeffectively teachershave tobepresent in theschoolandclassroom,andtheyhavetoteach for the intendeddurationof time. If teachersareabsent, instructional time is reduced –with adverse effects on pupil learning (Das et al. 2004;Suryadarma et al. 2006; Wobmann 2000), and the potential impact of in-service training onteachereffectivenessisweakened.TDPwillseektoreduceabsenteeismbytrainingheadteachersto strengthen SLM. As noted above, the programme also hopes that teacher motivation willimprove as increases in pupils’ learning outcomes become apparent, and that this in turn willimprove teacher attendance (EDOREN 2014). This section examines quantitative data from thebaselinesurveyontheextentofteacherattendancebeforethestartofTDP.34

The average level of daily teacher absenteeism over the previous five days, according to theschools’ records, was 14% (Table 19). This average conceals large variations in teacherabsenteeism across schools. At one extreme, for the 10% of schools with the lowest level ofteacherabsenteeism,averagedailyabsenteeismduringthepreviousfivedayswaszero,whileattheotherextreme,forthe10%ofschoolswiththehighestlevelofabsenteeism,morethan33%of

34SeeVolumeIIforfurtherexplanationofindicators.


©EDOREN 48

teacherswereabsentonaverage(seesupplementaryanalysisinAnnexMofVolumeIIfordetails).Astheserecordswereobservedonannouncedvisits,itisalsopossiblethatabsenteeismishigheratother times.There isa twopercentagepointdifference in teacherabsenteeismbetween thetreatmentandcontrolgroups,ata10%levelofsignificance.35

Table19Teacherabsenteeism(meanestimates)

Indicator Overall Treatment Control N

Teacherabsenteeismfromschoolrecords(logframeOutcomeIndicator1.2)

Averagedailyteacherabsenteeism(%ofteachersabsent)

13.8 12.9* 14.8 330

Source:Quantitativebaselinesurvey(October2014),schoolrecords.Note:(1)Theaveragedailyabsencerateiscomputedoverthepreviousfiveworkingdays;(2)statisticallysignificantdifferencesbetweengroupsaremarkedwithasterisks:*significantatthe10%level;**significantatthe5%level;***significantatthe1%level.

Theaveragedailyabsenteeisminthethreestatescanbebroadlycomparedtoabsenteeismresultsfromotherstudies inNigeriaand intheregion,althoughtherearecaveats.Thestudiesdiffer intermsofhowabsenteeismisdefined,howthedataarecollected(includingwhetherschoolvisitswereannouncedorunannounced),andgeographicalscope.Nonetheless,thesestudieshelpplaceabsenteeisminthethreeTDPstatesintocontext.TheTDPdesignmissioninNigerstatefoundthatteacher absenteeism ranged from 16% to 29% in three different LGAs (DFID Nigeria 2012).Anotherstudythatusedunannouncedschoolvisitsfoundmuchlowerteacherabsenteeism–3.4%ofteachersinEnuguand2.4%inKadunawereabsentonthedayofthesurvey(Guerrero2012).This suggests that teacherabsenteeismvariesmarkedlyacross states inNigeria.Similar rates toourTDPestimateof14%havebeenfoundinothercountriesinsub-SaharanAfrica(seeOPM2015onTanzania;WorldBank2012bonKenya;andChaudhuryetal.2006onUganda).

State-level baseline results for teacher and head teacher absenteeism in Jigawa, Katsina andZamfaraaresetoutinBox11below.

Box11BaselineresultsforteacherandheadteacherschoolabsenteeisminJigawa,KatsinaandZamfara

• Average daily teacher absenteeism is lower in Katsina (13%) than in Zamfara (14%) and Jigawa(15%).

• Themostcommonly reported reason forabsenteeismgivenby teachers isownor family illness(51%–64%)inallthreestates.

• Social/religiousobligationsaremorecommonlyreportedasareasonforabsenteeisminZamfara(15%) compared toKatsina (11%) and Jigawa (7%), and lateor non-paymentof salaries is citedmoreofteninJigawa(10%)andZamfara(5%)thaninKatsina(1%).


35Theeffectsizeisverysmall(0.05)andassuchthisdifferenceislikelytobenegligibleinpractice.ThesedifferencesbetweentreatmentandcontrolgroupsisdiscussedinmoredetailinSection3(VolumeII).


©EDOREN 49

4.3.4 Driversofteacherabsenteeism

The baseline survey asked teachers who had been absent in the previous five days what thereasons for their absence were.36 The most commonly cited reason was own or family illness(58%), followed by: collecting salary and family reasons (20%); social/religious obligationsincluding attending funerals (10%); late or non-payment of salary (6%); transport (6%);epidemic/disease outbreak (2%);37 training (1%); and ameeting or event at the LGA or SUBEB(1%).

Inthequalitativestudy,however,researchersfoundasomewhatdifferentsetofreasons.Severalteacherscombinedtheirteachingworkwithfarmingorsmallbusinessesoutsidetheschool.Mostclaimed that this did not affect their teaching, because the school day ends around 1pm andteacherscanthenattendtohouseholdchores,farmsorbusinesses.Insomecases,suchasthosequoted below, however, teachers were frank about their jobs or other home commitments orfinancial problems keeping them out of school even during lesson times. Some teachers hadseveralbusinessestosupplementtheir incomefromteaching.Theseresponsesalsosuggestthatteachersmaynotgivemuchtimetotheirworkoutsideofteachinghours:forexample,planningthenextday’slessons.

Interviewer:Howdoyoubalancethefarming,yourmanualworkandtheteachingjob?Teacher:Iamtryingmybesttogivetheteachingjobmybest,thesetwobusinessesofmanualworkandfarmingarejusttheretosupportme,becausetheteachingsalaryisnotgoodenoughtotakeofmeandmyfamily,mysalaryisjustNGN11,000…andrememberIhaverelationsandfamilymembersthatdependonmeandsometimesweusedtospendabout40dayswithoutsalarysowiththatmanualworkandfarmingIwillbeabletotakecareofmyselfandmyfamily.Interviewer:Sodoyouthinkyourmanualworkwillnotaffectyourteachingjob?Teacher:Maybefarmingmightaffectmyteachingjob,becauseduringtherainyseasonsometimesIamabsentforoneortwodaystotakecareofmyfarm,plantseedssothatIwillnotbeleftbehind.[Casestudyteacher]

Teacher:Honestly,sometimesaftereverytwoweeksIusedtomisslikeoneortwodays,thisisbecauseoffinancialchallenges,IamnotfromthiscommunityandsometimesIwillwakeupwithonlyNGN200,Iamafamilyman,Ineedtoprovideforthem,soinsteadoffuellingmymotorcycleandcomingtoschoolwillpreparetostayathomeandassistmyfamilywithit[non-TDPteacher]

Teacher:Iwasinschoolthroughoutlastweek,justthatIusedtocomelate.Ihavetohustleforwhatwillkeepmeforthedaybeforecomingtoschool.Thereasonbehindallthisisthefactthatwedon’tgetoursalaryontime[TDPteacher]

Someteacherswereundergoingfurthertrainingoreducationatthesametimeasteaching,takingthemaway fromtheir regular jobs. Inmost schools, researchersalso sawteachersarriving late.Lateattendanceby female teacherswasexplained in termsof theexpectation that theywouldprioritisefamilyissuessuchasillnessandchildcareovertheirjobs,howeverresearchersoftensawmaleteachersarrivelatetoo.

36Teacherscouldgiveuptothreereasonsandthereforethecategoriesmaynotsumto100%.37Thispossiblyreferstothe2014outbreakofEbolainWestAfrica,includinginNigeriafromJulytoOctober2014,whichisroughlyaroundthetimewhenthisbaselinesurveywasbeingconducted.


©EDOREN 50

The quantitative survey only measured teacher absenteeism from school, but during thequalitative study researchers also observed teachers present in school but not teaching,sometimesat the same timeasoneormore classeswerewithout a teacher. This accordswithprevious evidence from Enugu and Kaduna in Nigeria, and from other sub-Saharan Africancountries, suggesting that, inaddition to schoolabsenteeism, there is also typicallya classroomabsenteeism problem. In many cases it is substantially higher than school absenteeism andaccountsforlargelossesininstructionaltime.InEnugu,classroomabsenteeismwasfoundtobeat13%andinKadunaat6%(WorldBank,2008).TwostudiesinTanzaniafoundthat50%–67%ofteachers in rural schools,althoughpresentat school,wereabsent fromclassrooms (OPM2015;World Bank 2012b),38 while in Kenya classroom absenteeism was at 40% (World Bank 2012a).Watts and Allsop, 2015, also noted that, until recently, teachers were trained in the NCEprogrammetoseethemselvesassubjectspecialists,andtheyarguethatthismakesthempoorlyprepared for teaching in the primary classroom. Teachers’ specialised training often does notmatchthesubjectthatisneededwhentheyarepostedtoaschool,andshortagesofteacherscanarise inspecificsubjectareas,evenwhenthereisnotanoverallshortageofteachers.Oneheadteacherdirectly linkedabsence fromclassroomstothe latepaymentofsalaries, suggestingthatteacherswereonakindofunofficialstrikeorwereseverelydemotivated.

Table20Summaryofstrengthofevidencerelatingtoteachers’motivationandattendance


Strengthofevidence

Teachersaremotivated–whetherthroughexternalincentiveslikesalaries,promotions,etc.ortheirowncommitment–toattendschoolandlessonsregularly,totrytoimprovetheirteaching,totakepartinlearningopportunitiesandtoapplynewknowledgewhentheygetit

• Teachersareintrinsicallymotivated,totrytoimprovetheirteaching,totakepartinlearningopportunitiesandtoapplynewknowledgewhentheygetit

Mixed

• Teachersareextrinsicallymotivatedtodothesame No Strong

• Teachersattendschoolregularly,andareintheclassroomswheninschool

No Strong

4.4 Contextualfactorsthatinfluencethequalityofteachingandlearning

It iswidely acknowledged that teachers in someparts ofNigeria face a particularly challengingenvironmentintermsofclasssizes,schoolinfrastructure,recruitmentandtransfers,andthesocialand political context. Past studies have suggested that school characteristics may influenceteachereffectiveness(BennellandAkyeampong2007;Robinson,Lloyd,andRowe2008;UNESCO2004),and,conversely,thattheimpactofother investments ineducationaredependentonthequalityofteachers(HanushekandRivkin2006;Yoshikawaetal.2007).

The theory underlying TDP’s in-service training component implicitly suggests that thesecontextualfactorswillnotbesevereenoughtohinderthequalityofteachingandlearning.Totestthisassumption,thissectionexaminesteachers’physical,social,andpoliticalcontext,andhowitaffectstheirteaching(Table21).Thebaselinequantitativesurveygatheredinformationaboutthecharacteristics of the surveyed schools to help understand the environment in which teachers38Thisignoresifateacherwasscheduledtoteachornot.Accountingforwhetherteacherswerescheduledtoteach,oneofthestudiesfoundthatclassroomabsenteeismwas66.8%(OPM2015).


©EDOREN 51

workandhowitmaylimittheireffectiveness.Thequalitativestudyalsoaskedteachersabouthowthey dealwith aspects such as poor infrastructure, limited resources, and large class sizes, andhow they relate to theirpupilsand the familyandcommunity that theirpupils come from.Thefollowingsub-sectionsexamineeachofthesequestionsinturn.

Table21Evaluationquestionsonthecontextthatinfluencesthequalityofteachingandlearni

Question Keydatasources

Whatclasssizesdoteachershavetoteach? Quantitative (interviews and schoolrecords)

Howdoteacherscopewithlargeandmulti-gradeclasses? Qualitative (lesson observation andinterviews)

Howareteachersaffectedbythepoorstateofinfrastructureandresourcesinsomeschools?

Qualitative (lesson observation andinterviews)

Howareteachersaffectedbyrecruitmentandtransferprocesses? Qualitative(interviews)

Howareteachersaffectedbyotheraspectsofthesocialandpoliticalcontext?

Qualitative(interviews)

4.4.1 Classsizesandmulti-gradeclasses

AsshowninTable22below,theaveragePTRis59:1.ThiscomparestothenationalPTRof36:1in2011(UNESCO2014)andtheofficialpolicyof35:1(FederalRepublicofNigeria2004).The10%ofschoolswiththelowestPTRshave23pupilsperteacherbutattheotherendofthedistributionthe10%ofschoolswiththehighestPTRshave106pupilsperteacherormore.This impliesthatthese latter schools have a severe shortage of teachers, with likely consequences for teachereffectivenessintheclassroom.

Theaverageclasssizeis42pupilsperclassroom;earlygrades(Grades1–3)constituted60%oftheclassrooms observed. The fact that the average class size is lower than the average PTR (59:1)could imply that early grade primary class sizes are typically smaller than upper primary gradeclasssizesand/orallpupilsarenot taughtat thesametime(e.g.doubleshifts), leadingtoclasssizes being less than the PTR. Finally, pupil absenteeism may also be driving the differencebetween class size and PTR since class size calculations were based on the number of pupilspresentintheobservedclassesonthedayofthesurvey,whilePTRwasbasedonthetotalnumberofpupilsregisteredintheschool.PupilabsenteeismwouldthuslowertheclasssizeonthedayofthesurveywhilenotaffectingthePTR.Treatmentschoolsonaveragehavesignificantlysmaller39classsizesthancontrolschools.40

39Ata5%levelofsignificance.40Theeffectsizeisverysmall(0.03)andthisdifferencemaybeconsiderednegligibleinpractice.ThesedifferencesbetweentreatmentandcontrolgroupsisdiscussedinmoredetailinSection3(VolumeII).


©EDOREN 52

Table22Schoolcharacteristics:Size,PTRandclasssize

Indicator Overall Treatment Control N

Numberofteachersemployed(Grades1–6) 12.2 11.9 12.6 330

Numberofpupilsregistered(Grades1–6) 654.8 614.9 696.6 328

PTR 58.7 58.2 59.2 328Numberofpupilsperobservedclassroom(classsize) 41.9 39.6** 44.3 1053

Source:Quantitativebaselinesurvey(October2014),schoolrecords,classroomobservationandheadteacherinterview.Note:(1)Classsizedatacomefromtheclassroomobservationinstrument;(2)statisticallysignificantdifferencesbetweengroupsaremarkedwithasterisks:*significantatthe10%level;**significantatthe5%level;***significantatthe1%level.

While it is clear that not all classes are over-crowded, some teachers in the qualitative studydescribed large classes, sometimes in excess of 200 pupils, and talked about the difficulties ofteachingintheseclasses.Teacherssometimesadmittedtohavinglostcontrolofalargeclass,andcomplainedthat it tooka longer timeto finisha lessonwitha largeclass.Largeclassesmade itdifficult for teachers toget toknow theirpupils andmonitorattitudesorbehaviouramong thepupilsthatmightaffecttheirlearning.Theyalsomadeitdifficulttoapplychild-centredmethodsthatteachershavebeentrainedon.Itwasnotclearthattheyhadanystrategiesforhowtodealwiththisissue.

INTERVIEWER:Howdoyouknowiftheyunderstandornot?TEACHER:Wedoknowwhensomefewpupilsintheclassunderstandbecause…theyaretoomanyinaclass,asyouareworkingapupilmaybebeatinganotherwhileanotheroneisdragginganotherandyouareallalonetocontroltheclass.[Non-TDPteacher]

Evenwhenyoutryusingthechildcentredapproach,duetotheoverpopulationyoufinditdifficultbecausetheykeepmakingnoise.[Non-TDPteacher]

Someschoolswereoperatingdefactomulti-gradeclassesbecauseofashortageofclassroomsorteachers,butteachershadlittletosayabouthowtoteachtheseclasses.Theyappeartohavehadnosupportonhowtodealwithpupilsfrommorethanonegradesittinginthesameclassroom.Teachersdetailedusinginformalmethodsformulti-gradeteaching,suchasrotatingthegradessothatdifferentpupilsweresittingatthefrontduringdifferentperiods,butresearchersdidnotseethesemethodseffectivelyputintopractice.

Although large class sizeswas often cited as a problem, it is alsoworth noting that during thesurveyvisittherewereoftenfarfewerpupilsinclassthanthenumberregistered.Teachersusuallyattributedthistopupilsbeingkeptathometoworkonfarmsduringtheplanting/harvestseason.Inurbanschools,therewerealsosometimesunusedorbarelyusedroomsforITorlibraries,whichcouldhavebeenpressedintoserviceastemporaryclassroomsifnecessary.


©EDOREN 53

4.4.2 Infrastructureandresources

In the quantitative survey, head teachers at 88% of schools reported that their school neededmajorrepairs.Inteacherinterviews,70%ofteachersagreedwiththestatementthattheirschool’sbuildingwasina‘poorcondition’,and33%saidtheydidnothaveadequatematerialstodotheirjobproperly.Thetypesofrepairsmostfrequentlyreportedasbeingneededwere:repairstotheschool rooforceiling (68%); repairsofbrokenwindows (65%); repairsofclassroomwalls (53%);repairsoftoilets(52%);repairsofoutsidewalls (30%);andrepairsoftheplayground(21%). Inaqualitativeconstraint rankingexercise, ‘inadequateclassroomresources’and ‘inadequateschoolinfrastructure’ were fairly consistently ranked among the worst constraints faced by teachers.Severalheadteachersalsoreportedthattheyneededadditionalresources,inparticularadditionalclassrooms, furniture and blackboards. Even though all respondents were told that data werebeing collected for research purposes only, some degree of ‘need signalling’ (in anticipation ofreceivingschoolresources)maybeinfluencingtheseresponses.

Teachers’ own accounts during qualitative interviews suggested that the poor state ofinfrastructure, shortage of classrooms, and lack of textbooks made teaching difficult anddemotivatedthem.Teachersalsoclaimedtoarrivelatesometimesbecauseofrain,poorroads,orbecause they do not have vehicles. Teachers complained about the lack of a teachers’ room,forcingthemtositoutsideduringoffperiods,althoughinschoolsthatdidhaveteachers’roomsresearchersobservedthattheywereoftenuncomfortable,andwereusedforstorageoffurnitureortextbooks,andthattheywerelittleusedbytheteachers.

Textbooks were not routinely given to all pupils, but sometimes pupils sat in groups and onetextbookwasgiventoeachgrouptoshare.Thiswassometimesbecausetherewere insufficienttextbooks, although in some cases there were textbooks that were not used. During oneobservationteachershandedouttextbooksthatlookedvirtuallyunused,andpupilswereunableto find the correct page number or recognise which part of the page the teacher was talkingabout.Thelackoftextbookswassometimeslinkedtothestyleofteaching,withteacherscopyingextensivenotesontheboardandleavingpupilstocopythese.

4.4.3 Recruitmentandtransfers

Several teachers described how they became teachers through having relatives or otherconnectionsthatwereabletogetthemthejob,evenwhensomeofthemlackedtheappropriatequalifications. There was evidence that the least competent or least motivated teachers werebeingposted to remote rural areasasapunishmentby LGEAs thatwereunableorunwilling todiscipline them in other ways, or to remove them from service. Teachers suggested that well-connectedheadteacherscouldfindawaytohaveateachertransferrediftheytookadisliketothatteacher,orinanattempttosettleaconflictbetweenteachers.

Teacher:Somefewteachersarelazyandtheydon’twanttoteach,theyareafterthesalarytheywillreceive.Interviewer:Doyoureportthosekindsofteachers?Teacher:Yes,weusedtoreportthemtotheEO(educationofficer).Interviewer:Whatactiondoeshetakeusually?Teacher:HeusuallytalkstothemandifallthingsproveabortivethenhewilltaketheissuetotheEducationSecretary(ES),ifthatteacherisnotcarefultheESwillposthimouttoaveryremotearea.[Casestudyteacher]


©EDOREN 54

…onceyouhaveaproblemheretheywilltransferyouwithoutinvestigatingintothematter,theyjusttransferatanytime,Iwaseventransferredtoavillagecalled---beforeIwaslatertransferredbacktothisschoolwereIthinkIwillretirefrom,evenhereIwastransferredlastmonthbutIrefusedtogo[laughs]yes,becausemytimeisalmostoverwhyshouldtheyplaymelikeaball.Toomuchtransferistheonlyproblemwearefacing,oncetheheadmasterdoesnotlikeyouhewillmakeawaytotransferyou.[Non-TDPteacher]

4.4.4 Socialandpoliticalcontext

ThissectionconsidersthesocialandpoliticalrelationshipsbetweenteachersandLGEAandSUBEBofficers,communities,parents,andpupils,inordertounderstandhowtheseaffecttheirabilitytodotheirjobs,theirmotivation,andthelikelihoodofthemputtingnewinsightsfromtrainingintopractice.

Some teachers complained that LGEA and SUBEB inspection officers victimised and intimidatedthem,ratherthansupportingthem.Ontheotherhand,althoughtheyfacedtheriskoftransfer,someteacherswerereportedlyunfazedbythethreatsofpunishmentfromtheLGEA,andoftendidnotappeartobeparticularlysubordinatetoheadteachersorLGEAofficers.LGEAofficershadthepowertocautionteachersortransferthem,butnottoremovethemfromserviceordeductpay.Inspectorsreportedly‘don’tcareaboutwhatweteachasteachers,theyareconcernedwiththerecordofwork,theywilljustcheckourdiaryandpraiseusandmoveon’[TDPteacher].

Teachers alleged that promotions were given unfairly, based on a teacher’s political or socialconnections, rather than seniority. Similarly delays in payment and transfers were sometimesthoughttobeduetopoliticalconnections.

…InfactinNigeriayoucanhardlyseeanareawherepoliticshasnotinterfered,butwhatweexperiencedforthepasttwotothreeweeksisalarminginZamfarastate,ourfriendsandbrothersweretransferredillegallytodifferentareas,takeforexampleinB---,andyouarefromB---andyouteachtherebutbecauseyoudon’tsupporttherulingpartytheywillpostyoutoavillage.[TDPteacher]

Teachersclaimedtohaveacordialrelationshipwithparentsandthecommunity.Inpracticethisseemedtoinvolvevaryinglevelsofmutualrespectdependingontherelativesocialstandingoftheteachersandcommunitymembers.Someteachers lived inthesamecommunityasthepupils intheirschool,whileothers,particularlyinremoteareas,travelledfromnearbytowns.Livinginthesame area did not guarantee close relationships with parents, but may have played a role. Inurbanareas,someteacherssendtheirownchildrentoprivateschools.Overall,mostteachersdidnotalwaysfeeltheywererespectedbythecommunity.Despitethis,someresponsessuggestthatteachersinruralareashadahighersocialstandingthanmostparents,althoughtheycouldbeheldtoaccountbymorerespectedorpoliticallyconnectedmembersofthecommunity,suchasvillageheadsandSBMCheads(oftenthesameperson).PTAmeetingswererarelymentioned,andinonecaseitwasclearthatparentswerechargedPTAfeeseventhoughnomeetingstookplace.SBMCswere often mentioned as a place for teachers to interact with parents and the broadercommunity,butinsomecasestheywereinactive.

Whenaskedhowtheywouldrespondtoafatherwhocomplainedthathischildwasnotlearningenough inschool, teachers tendedto raisepointsabout the importanceofparents’ownrole inensuring their children attend school regularly and supporting their learning. Similarly, teachers


©EDOREN 55

oftenblamedpupilabsenteeismonfarmingorotherhouseholdwork,discountingproblemssuchasteacherabsenceorpoorinfrastructurethatmighthavedeterredpupilsfromattendingschool.Whilevalid,thesepointstendtoshiftresponsibilityawayfromtheteacherandontotheparents–who inmanycasesare likely tobe illiterateandhave little timetosupervise theirchildren– toensurechildrenlearn.

Therelationshipsofteacherstothepupilstheyteacharealsolikelytobeimportantforthewaythattheyteachandadoptnewtraining,especiallywhentheaimofthetrainingistomovetowardsmore ‘child-centred’ forms of teaching. Teachers stressed the need to control a classroom andmaintain authority over pupils, by being friendly, telling jokes, using songs, and so on, andsometimesbygivingsmallamountsofmoneytopupilswhoanswerquestionsorperformwell,butif necessary alsobybeatingor otherwisepunishing them. Teachers talked about themselves asbeing‘afather’tothecommunity,orevenas‘parentsofthenation’,aboutbeingapersonwhosteps intoaparental role in relation to thepupils in their care,andashavinga role in instillingmorality,whichisvaluedatleastasmuchastheirroleinimpartingotherkindsofknowledge.

Theadvice[forahypotheticalnewteacher]isthatheneedstoshowthepupilsheisateacher,heisabovethembecauseattimesyouneedtoforceachildtodosomethingandsometimesyouhavetobeatthatchild,ifhedoesn’tdothatthepupilsmaydecidetodisrespecthimandturnhimintoaplaytoyandthatwillmakethemunseriouswiththeirstudies.[Non-TDPteacher]

Ilovediscipline,itgoeshandinhandwithteaching,educationgoeswithmoralityandsoifyoueducateachildwithoutmoralvaluehiseducationisvaluedlessbecausemoralityisveryimportantinanysocietyyoulive,thismademechoosetobeateacher.[Non-TDPteacher]

Teachers’ role in ensuring good behaviour could sometimes result in the exclusion orembarrassment of pupils frompoorer or less supportive homebackgrounds, for instance thosewhoarriveat schoolwithdirtyuniforms,orwithoutnotebooks. Inonecase teachers’authorityapparentlyextendedtobeingabletousepupilsforfarmwork.

Teacher:Sometimeswecallthedirtylearnerstocomeforwardandwarnothersnottocometoschoollikethatagain,bybringingthedirtypupilsout,theyfeelembarrassedandthenyouwillfindoutthatmanywillstrivehardtowashtheiruniformsbecauseofthefearofbeingcalledinfrontoftheassembly.Anotherweconductinspection,checktheirnailsandtheirbodytomakesuretheyareclean,weconductinspectioneveryweek.Interviewer:Don’tyouthinkbyembarrassingthedirtyoneswillaffecttheirmoodthatday?Teacher:Wewanttodisciplinethem,wedon’twantthemtocopythatbehaviour,thoughtheywillfeelrejectedandsometimesevencryalot,butthatreallyhelpsusalot,andtellthemtowatchhimasadirtyboy,manyofthemwillnottryitagain.Interviewer:Wenoticehowteacherswhippupilsinthisschool,howoftendoyoudothatandwhy?Teacher:Yes,wearejusttryingtomakethemstopcominglateandsometimesitisthewayofsettlingthemwhentheyfighteachother[Casestudyteacher]

Interviewer:Whatabouttheschoolland,doyouplantsomethingsforyouruseasateacher?


©EDOREN 56

Teachers:Yes,teachersutilisetheschoollandandalsousepupilstoworkfortheminthefarm...

Oneteacherrecognisedthatpupils’attendanceintheschool is ineffectvoluntary,andthat ‘wecannotcontinuetobeatthe learnersandpunishthem’,becausetheirparentswillnot forcethechild togotoschool ifheorshedoesnotwant to. Inothercases teachersweredeterred frompunishingpupilsbecauseparentswouldcomplain.Thoughtheywereexpectedtotakeonasemi-parentalrolebyparentsandthecommunity,teacherswerenotalwaysabletodothisbecauseofthenumberofstudentstheywereinchargeof.

TeachersencounterproblematicpoliticalrelationshipswithLGEAandSUBEBofficersaspartoftheprocessesofinspection,recruitment,promotionandtransfer.Thistendstoaffecttheirmotivationnegatively and may have resulted in less competent teachers being placed in certain schools,particularly remote ruralones.Their relationshipswithcommunitiesandparentsareapparentlycordial, but generally not on equal terms; other than through powerful members of thecommunity, such as village heads, there appears to be little accountability of teachers to thecommunity. Their relationships with pupils reflect their perceived traditional role as moralguardiansandnotionsofcontrollingaclassroomthatarepotentiallydifficulttoreconcilewiththeputative move towards child-centred learning, and may mean that training on child-centredmethods is taken up in a superficial way, rather than as a fundamental adjustment in therelationshipbetweenpupilsandteachers.

Table23Summaryofstrengthofevidencerelatingtocontextualfactorsinfluencingteachingandlearning


Strengthofevidence

Context:Teachershaveaccesstosufficientmaterials,theirclasssizesarenottoolarge,andthesocialandpoliticalcontextdoesnotpreventtheadoptionofnewteachingpractices

• Classsizesaresmallenoughforteacherstoteacheffectively No Strong

• Infrastructureandmaterialsareadequateforteacherstoteacheffectively

No Strong

• Thesocialandpoliticalcontextinwhichteachersworkpermitsengagementwithparents,communitiesandpupils

Mixed

4.4.5 Linkagesbetweenvariousaspectsofteachereffectiveness

Additional analysis was conducted to examine linkages between three key aspects of teachereffectiveness(subjectknowledge,pedagogyandabsenteeism).ThedetailedanalysisisreportedinVolumeIIbutsomenoteworthyresultsarediscussedbelow.First, correlations between TDNA scores across the three subjects shows that teacherswho dorelativelywellononesubjectalsodorelativelywellontheremainingtwosubjects,andviceversa.Second, teachers with higher TDNA scores also demonstrated better skills at assessing andmonitoring pupils’ academic progress. However, by contrast, teachers demonstrated an almostuniformly low proportion of total lesson time (22%–25%) in positive interaction with pupils,irrespective of their subject knowledge scores, thus strongly suggesting that all teachers,irrespective of their achievement level in the TDNA, would benefit from in-service training inpedagogy. Teachers with higher subject knowledge scores also demonstratedmarginally lower


©EDOREN 57

rates of absenteeism. Finally, the proportion of teachers who hold an NCE is larger amongteachers with sufficient (maths)41 subject knowledge (85%) than among teachers with near-sufficientsubjectknowledge(78%)orlimitedsubjectknowledge(72%).Overall, the available evidence suggests that levels of subject knowledge and knowledge ofassessmentandmonitoringofpupil progress are very low.However, thereexists a small group(8%) of relatively more effective teachers who on average have relatively stronger (albeit stillweak) maths, English and science subject knowledge, and ability to assess and monitor pupilprogress, and somewhat lower absenteeism. However, this relatively more effective group ofteachers still displays a limited use of effective teaching practices, similarly to the other threegroupsofteachers.

4.4.6 Relationshipsbetweenteachereffectivenessandpupillearninglevels

The theory underlying the outcome-to-impact link in the TDP in-service training model is thatpupils learn more when they are taught by effective teachers. This section discusses therelationshipbetweenpupil learning levelsandaspectsofteachereffectivenessmeasuredbythebaseline survey, namely subject knowledge, ability to assess and monitor pupils’ academicprogress, pedagogy and teacher motivation (the graphs presented in Figure 12 show some oftheseresults).42

Basedona simplebivariate andmultivariate regressionanalysis, no significant relationshipwasfoundbetweenpupils’learningscoresinEnglishliteracyandteachers’TDNAEnglishscores;pupils’scores innumeracyand teachers’ TDNAmaths scores;orpupils’ scores in scientific literacyandteachers’ TDNA science scores. Similarly, pupils’ learning levels in the three subjects were notsignificantlyassociatedwithteachingpracticesintheclassroom,asmeasuredbytheproportionoflessontimeteachersspendinpositiveinteractionwithpupils.Incontrast,pupillearningscoresinallthreesubjectswerepositivelyandsignificantlycorrelated43withteachers’abilitytoassessandmonitor their pupils’ academic performance. Overall,more than half of the variation in schoolperformance in pupil knowledge cannot be explained by observed teacher-level characteristicsmeasuredinthissurvey.

However,itisnoteworthythatasignificantamountofthevariationinpupillearningcomesfromdifferences inpupils’own familybackgrounds (asmeasuredbyanasset index)andcommunity-levelcharacteristics(aspickedupbyLGAfixed-effects),bothofwhichrelatetoaspectsofschoolperformancethatthequantitativesurveyisnotmeasuring.

41Thisanalysiswasonlycarriedoutforperformancebandsinmathsbecausetheothertwosubjects(Englishliteracyandscience)didnothaveadequatesamplessizes(especiallyinthetopperformanceband)topermitreliabletestingofdifferencesinmeans.42 School-level pupil and teacher variableswere constructed to examine the direction and significance of correlations betweenselectedaspectsofteachereffectivenessandpupillearning.43Ata1%levelofsignificance.


©EDOREN 58

Figure12Teachereffectivenessandpupillearninglevels:Simplebivariateregressionanalysis

The fact that teacher knowledge, pedagogy and ability to assess pupils has very low (ormaybeevenzero)predictivepowerdoesnotnecessarilyinvalidatetheimportanceofteachingqualityforimprovedpupillearning.Thelowcorrelationsandexplanatorypowercouldmerelyreflectthefactthatitisverydifficulttoaccuratelymeasureteacherknowledgeandability–aperfectlyaccuratemeasure of teacher knowledge, pedagogy and ability might have stronger predictive power.However, characteristics like teacher knowledge, pedagogy and ability are inherently noisy andoften unobservablemeasures, and consequently as the level of noise increases, the coefficientvalue tends towards zero. In sum, even though this analysis found no significant relationshipbetween pupils’ learning scores and some characteristics of teacher effectiveness, it does notconclusivelyinvalidatetheimportanceofteachingqualityforimprovedpupillearning,and,rather,highlightsthechallenges involvedinmeasuringcharacteristics liketeacherknowledge,pedagogyandability.


©EDOREN 59

5 Schoolleadershipandmanagement(SLM)

Manyfactorsinfluenceteachereffectiveness.ThesefactorsincludeSLMandschoolcharacteristics(BennellandAkyeampong2007;Robinson,Lloyd,andRowe2008;UNESCO2004).Thereisalsoagrowing body of research examining the ‘school system’, which focuses on the complexrelationship between school-level dynamics and the accountability, political economy, andbroader socio-economic context within which a school is embedded (Moore 2015; Lemke andSabelli2008).Thefocusof thissectionof thereport isguidedbytheTDPTOC(Figure2above),and draws on data collected in both quantitative and qualitative surveys. It examines the SLMpracticesandotherbehavioursofheadteachers,andthefactorsthatexplainthese.44

5.1 AssumptionsrelatingtoSLMintheTDPTOC

EffectiveSLM is crucial in raising schoolperformance. Literature shows that school functionalitybefore the start of education quality improvement programmes is an important factor indeterminingtheabilityofschoolstotakeadvantageofandbenefit frominterventions(Hopkins,Harris, and Jackson 1997). Some important aspects of SLM and the creation of a schoolenvironment that is conducive to learning are captured by this baseline survey. These includesupportandsupervisionprovidedbyheadteacherstoteachers;whetherandhowheadteachersmanagepupilandteacherabsenteeism;andifthereisanactiveSBMC.Thisquantitativeanalysisiscomplementedbyarichpictureofwhatitisliketobeaheadteacherintheschoolsvisitedaspartofthequalitativeresearch.

SLMisrelevanttotheevaluationoftheTDPintwoways.First,teacherscannotbeassumedtobemotivated to invest the time and energy required to learn and adopt new teaching practiceswithoutappropriate leadershipandmanagementfromheadteachers.EffectiveSLMisthereforeanimportantfactorindeterminingwhetherTDPtraining(output)leadstoimprovedsubjectandpedagogical knowledge (outcome), and whether this knowledge is applied through improvedteachingpractices(intermediateimpact).Second,theTDPplanstotrainheadteachersonissuesrelatingtoSLM.

EffectiveSLMisthereforebothanimplicitassumptionin,andanexplicitoutcomeof,theTDPTOC.Threeheadlineassumptions relating toSLMwere thus identifiedon thebasisof thequalitativeevaluationmatrix (AnnexA,VolumeII)andthematicanalysisofthedata.Thefollowingsectionsexamine whether each of these assumptions hold, assess the strength of evidence, and breakdowntheassumptionsintotheirmainconstituentpartstoallowformorenuancedinterpretation.

44Thescopeofthisreportdoesnotextendtoanassessmentofeducationsystemgovernanceandaccountabilityatthestateorfederallevel,ortoanin-depthanalysisofparentalperceptionsthatmayinfluenceSLM.


©EDOREN 60

Table24SummaryofassumptionstestedregardingSLM

Descriptionofassumptions Toolsusedtocollectdatatotestassumptions

Assumption1

Headteachersareeithermotivatedorincentivisedtoidentify,incentiviseandinfluencethepositiveteachingandmanagementpracticesthatTDPpromotes

Quantitative(headteacherinterviewsoninspectors,SBMCs);qualitative(mostlyKIIsandFGDswithheadteachers,TDP/non-TDPteachers,LGEA,SUBEB)

Assumption2Headteachershavetheabilitytoidentify,incentiviseandinfluencethepositiveteachingandmanagementpracticesthatTDPpromotes

Quantitative(lessonobservations,teacherinterviews);qualitative(mostlyFGDsandKIIswithheadteachers,TDP/non-TDPteachers,LGEA,SUBEB)

Assumption3

HeadteachersareabletoexerciseeffectivemanagementofschoolinfrastructureandresourcestofacilitatetheadoptionofthepositiveteachingpracticesthatTDPpromotes

Quantitative(interviewsoninfrastructure,repairs);qualitative(mostlyheadteachers,LGEA,SUBEBtools)

5.2 Headteachers’motivationandincentivestoidentify,incentiviseandinfluenceteacherbehaviour

Head teachersneed tobeeithermotivatedor incentivised to identify, incentiviseand influencethe positive teaching and management practices that TDP promotes. At best, without suchmotivationorincentivesheadteachershavenoreasontoaligntheirleadershipandmanagementof their teachers and schools with the objectives of the TDP. At worst, head teachers mayencounter countervailing incentives that actively undermine the adoption of TDP. This sectiondiscusses twogroupsofaccountabilityactors (SUBEB,LGEAand inspectors;andcommunityandparents)thatmaybeexpectedtoinfluenceheadteachers’incentives,and,finally,itaddressestheissueofheadteachermotivation.

5.2.1 SUBEB,LGEAandinspectorsasaccountabilityactors

SUBEBsandLGEAsarethegovernmentinstitutionsthatheadteachersreportto.Thus,theymightbe expected to be influential actors in incentivising head teachers. Most contact between theSUBEB/LGEAandheadteacherstakesplacethrough inspectionvisits. Inthequantitativesurvey,some 28% of head teachers reported that their school was visitedmore than three times permonthbyschoolinspectors,58%thattheirschoolwasvisitedtwoorthreetimespermonth,and14% that school visits occurred once permonth or less (Figure 13).45 Therewere no significantdifferences in regard to anyof these SLMor external support results across the treatment andcontrol schools. These findings broadlymatch the experience of the nine head teachers in the

45Thesefindingsregardingthefrequencyofschoolinspectionsstandinsharpcontrasttointernationalpractice.Forexample,theOECDTALISsurvey,whichcollatesdatafrom34countries,foundthatonaverageonly9.3%oflowersecondaryeducationschoolhead teachers reported having their teachers formally appraised by external individuals or bodies twice ormore a year (OECD2014).Thefrequencyofformalappraisalofschools(ratherthanteachers)byexternalindividualsorbodiesmaybehigherthanthisifsuchvisitsaremadewithoutinvolvingtheappraisalofteachers.Moreover,acomparisonoffiguresregardingfrequencyofvisitsinhigh-andlow-incomecountriesmaybemisleading,sincehigh-incomecountriesaremorelikelytohavedatainfrastructurethatenablesremotemonitoring/ongoingappraisalthatreducesdependenceonin-personvisitsbyexternalbodies.


©EDOREN 61

qualitativestudy:theoneheadteacherwhohadnotreceivedaninspectioninthecurrentschoolterm(i.e.inmorethantwomonths)wasbasedinaruralandrelativelypoorlyperformingschool:

Thistermwedidn’thaveanyinspectionteam,thelastonewasdonelastterm…Youfindout that nearby schools lack class rooms resources, no good infrastructures, the officialsdoesn’tvisittheschoolandtheteachersregularly.[Headteacher]

Figure13Frequencyofschoolinspections

However, in most cases there was limited evidence that these inspection visits incentivisedimprovedperformanceoftheheadteachersinamajorway,forseveralpossiblereasons.

Theinformationcollectedbyinspectorswasoftentoo‘shallow’toprovideaninformedbasisforassessingqualityofteaching(routineinspections,unliketherelativelyrarerfull-schoolinspections,focus on checking records, such as the teacher and pupil school attendance register and thevisitorsbookratherthanlessonobservations).Inmostcases,headteachers’examplesoffeedbackfrom LGEA inspectors included raising issuesof lowattendance and cleanliness, and to a lesserextent corrections of subject knowledge rather than pedagogy. Records checked by inspectorswere occasionally misreported or deliberatelymanipulated to give a misleading impression ofschoolperformance(forexample,entries inoneteacherattendanceregisterwereforgedandinanotherwerefilledinatmid-morning).

Asignificantmajorityofheadteachers interviewedduringthequalitativestudycomplainedthatLGEAs did not follow up on inspection through the provision of feedback to head teachers orrespondingtoheadteachers’requestsforadviceorsupport.Thistendedtobemorethecaseinruralschools.Insuchaschooloneheadteachernotedthatinspectorsprovide‘nofeedback.Theyonlyreportthattheycametosuch-and-suchaschoolandthisiswhattheysawandwriteitonanotebook.Thatisall’.Inanotherschoolaheadteacherstatedthatinspectors‘wroteareportbutwehavenotheardanyfeedbackfromthem’.

In a minority of cases, head teachers shared teachers’ complaints that the critical and publicmanner in which inspectors give feedback damages head teacher motivation, thereby

14%

58%

28%

Frequencyofschoolinspectionslastacademicyear(%ofschools)

Onceamonthorless

Twoorthreetimesamonth

Morethanthreetimesamonth


©EDOREN 62

undermining any positive impact. A head teacher described how the inspectors ‘decided tovictimiseus:theseminorthingsarewhatreallydiscouragesmanyofus.Insteadoftheinspectorsmaking corrections for us, some of them end up intimidating us’. Some well-connected headteachers allegedly use their political connections to resist LGEA officials’ attempts to disciplinethem. In one case, the reverse happened: politically active LGEA officials allegedly transferredteacherstoremoteruralpostingsduetotheirmembershipofanopposingpoliticalparty.

In one instance, inspectors who were unfamiliar with child-centred learning practices advisedagainst the adoption of related teaching techniques, such as the use of multimedia, thusdisincentivisingtheheadteacher’seffortstosupportTDP.AheadteachernotedthatwhilstSUBEBinspectors ‘alwaysdiscouragedus touse TDPmaterials…because they arenot trained, on theother hand whenever some LGEA officials come they will encourage us to make use of TDPmaterials,becausetheyaretrainedandareinvolved’.

Headteachersinaminorityofschools–whichtendedtobebasedinurbanareas–reportedmoreconstructiverelationshipswiththeLGEA.Oneheadteachernoted:

Whatmotivatesmeishowsupervisorsvisittoseehowtheworkisdone…weseethatallwhatwearedoingisbeingappreciated.…Ifyouarefacinganyproblemyouwriteandsendittothemandtheywillcometoseehowtheproblemcanbesolved.–Headteacher

Anotherheadteachernoted:

IamjustveryluckywhateverIreportedtheywillfollowitupandtakeactionagainstit.–Headteacher

Ina ‘low-performingschool’, theheadteachernotedthatdrasticactionwasnot takenuntil theSUBEBchairman, rather thangeneral inspectors, visited theschool.Thehead teacherdescribedthatwhen the chairman foundonly sevenout of 15 teacherswere at the school, ‘hewas veryangry, very angry thenhe gave out directives towithhold their salaries, and they succeeded inwithholdingtheirsalaryforthreemonths,aftersometimehepostedthemouttoanotherschoolandreplacedthemwithotherteachers fromanotherschool’.The lackofpreviousactionbytheLGEA may be explained by a perception of the disempowerment of senior LGEA officials. Forexample,oneofficialinanLGEAmentionedthatitwaspreviously‘theresponsibilityoftheLGEAtoemploy,todiscipline,topromotealltheLevel1–6teachers,butitisnowtheresponsibilityoftheSUBEB’.TherewassomevariationinLGEAofficials’perceptionsoftheirformalresponsibilitiesandpowers,evenwithinthesamestates,whichsuggeststhatconfusionmayundermineLGEAs’attemptstoincentivisepositiveheadteacherbehaviours.

Despitethesecausesforscepticism,mostLGEAofficialsmaintainthattheyarewillingandabletotake appropriate action to punish head teachers for poor performance. The ‘sticks’ that LGEAofficialsclaimtheycanuseto influenceheadteacherbehaviour includedisciplinarycommittees,demotions,andsuspensionofsalaries,thoughtheyadmitthatthesearerarelyusedandtherewaslittleevidencefromheadteachersorteachersthatsuchmeasureshadbeentaken.

There is therefore strong evidence that, despite regular inspection visits, inmost cases SUBEBsandLGEAsdonotsignificantlyincentiviseheadteacherstoidentify,incentiviseandinfluencethepositiveteachingandmanagementpracticesthatTDPpromotes.


©EDOREN 63

5.2.2 Communityandparentsasaccountabilityactors

Communitiesandparentsarethesecondgroupofaccountabilityactorsthatmaybeexpectedtoinfluence head teachers’ incentives. Communities generally have inputs into SLM throughrepresentationonSBMCs.Nearlyallschools(99%)hadanSBMC,and92%ofschoolsreportedanSBMCmeetinginthecurrenttermorovertheprecedingvacation.CommunityorSBMCmemberswerenotinterviewedaspartofthequalitativestudy,howeversomecommonthemesrelatingtoSBMCsdidemergefrominterviewswithheadteachers,teachersandofficials.

SBMCs and PTAswere generally spoken about by head teachers and government officials as ameanstoaddresslowpupilenrolmentandattendance.46Thequantitativestudyfoundthat75%ofheadteachers reportedpupilabsenteeismproblemsto theSBMCs,whichwerepraisedbyheadteachersfortheirefforts:

Duringbreak,somepupilsmaydecidenottocomebackontimeandstayintowntoloiterround,theSBMCcreatedacommitteethatwillensurethatthepupilsarebacktoschoolontime–Headteacher

DuringSBMCmeetingswenormallydiscuss issues related topupils’absenteeismandwenormallyadvisetheparentstostopsendingtheirchildrentoalmajirischool–Headteacher

TheSBMCcreatedateamoftheyouthinthiscommunitytohelpthepupilscrossthemainroadeverymorningandafterclosinghours–Headteacher

TeachersandheadteachersdidnotreportSBMCstakinganinterestinthequalityofteachingorlearning outcomes of students,which suggests SBMCs do not effectively hold head teachers toaccountonthebasisofschoolperformance.ThisisnoteworthygiventhatoneofthekeytasksforSBMCs is to ‘challengebadmanagementandmakehead teachersand teachersaccountable forpoorperformance’ (ESSPIN,n.d.).ThismaybebecauseSBMCmeetings tendtobeorganisedbyhead teachers, who therefore see SBMCs as a resource to be drawn upon rather than as amechanismformakingthemaccountabletothelocalcommunity.Indeed,oneheadteachernotedthat SBMCs were ‘responsible’ for important elements of SLM, rather than for ensuring hispositiveperformance:

DuringtheSBMCtraining,parentsaretoldthatitistheirresponsibilitytotakecareoftheschool,itistheparents’responsibilitytosendtheirchildrentoschool,itistheresponsibilityofthevillageheadandwhoever isresponsibleforthetowntosupervise itsdevelopment.SBMCisanorganisationresponsibleforcommunityandschool.–Headteacher

Parentsinoneruralcommunityhaddecidedtostopsendingtheirchildrentoschoolbecausetheywere frustratedwith the lackofprogress their childrenhadmade inEnglish.The fact that theytooksuchdrasticactionsuggeststhatthiscommunityinparticulardidnotfeeltheywereabletoholdtheschooltoaccount–whetherthroughSBMCorothermeans–toimprovestandards.

46SBMCswerealsospokenaboutintermsofmobilisingfinancesorin-kindsupporttomaintainschoolinfrastructureorresources.ThisfunctionisdiscussedinmoredetailinSection5.4on‘Managementofschoolinfrastructureandresources’.


©EDOREN 64

5.2.3 Headteachers’internalmotivation

The assumption described above outlines two broad ways in which head teachers can bemotivated: either through external incentivises to influence teacher behaviour in the way TDPpromotes, or through their intrinsic motivation to do so.47Whereas the previous two sectionsdealtwithexternalincentives,thissectionaddressesinternalmotivation.

Head teachers generally reported the same sources of lowmotivation as teachers (see Section4.3). Common complaints included dilapidated school infrastructure, inadequate teachingresources,parentsnotadequatelysupportingtheirchildrenwithfoodandresources,andexternalpressures such as farming andhawkingpreventingpupils fromattending school. Additional butrarer complaints unique to head teachers included: teachers not being responsive to the headteacher’s feedback (particularly relating to teacher absenteeism); their lack of control over therecruitmentofteachers,whotendtolackappropriatesubjectandpedagogicalknowledgedespitehavingtherelevantformalqualifications(SSCE,NCE,B.Ed.);anddelayedor lowsalarypaymentsthatdemotivatetheteacherworkforce.

However, a minority of head teachers reported being motivated by support given by thecommunity,eithertotheschoolorpersonallytotheheadteacher(suchasgiftsofclothes,foodormedicines).Twoheadteachers,bothofwhomhappenedtobebased inbetterresourcedurbanschools, also reported being motivated by contact with the LGEA, such as through schoolinspectionvisits.Theseheadteacherswerenoticeablymoreproactivewhenitcametomanagingteacherandpupilbehaviour.

There is thereforemixed evidence onwhether head teachers’ intrinsicmotivation to influenceteacher behaviour compensates for the lack of accountability mechanisms or appropriateincentivestructure.

Table25Summaryofstrengthofevidenceunderpinningassumptionsrelatingtoheadteachers’motivationandincentives


Strengthofevidence

Headteachersareeithermotivatedorincentivisedtoidentify,incentiviseandinfluencethepositiveteachingandmanagementpracticesthatTDPpromotes

• SUBEBandLGEAinspectorsincentiviseheadteacherstoidentify,incentiviseandinfluencethepositiveteachingandmanagementpracticesthatTDPpromotes

No Strong

• SBMCsandothercommunityactorsincentiviseheadteacherstoidentify,incentiviseandinfluencethepositiveteachingandmanagementpracticesthatTDPpromotes

No Moderate

• Headteachersareintrinsicallymotivatedtoidentify,incentiviseandinfluencethepositiveteachingandmanagementpracticesthatTDPpromotes

Mixed

47 In practice, ‘external incentives’ and ‘internalmotivations’mayoverlap and interact. For example, certain individualsmay beintrinsicallymotivatedbysocialstatusorrespect.However,socialstatus isdeterminedbyasetofexternalfactors. Inthissense,external incentives and internal sources of motivation may be considered more a spectrum of influences on head teacherperformance,ratherthantwodistinctcategories.Forthepurposesofthisevaluation,externalincentivescoverproximatefactorsthat are consciously designed and implemented by others in order to influence head teacher behaviour. Internal sources ofmotivationrefertothesetoffactorsthatheadteachersthemselvesreportasinfluencingtheirownbehaviour.


©EDOREN 65

5.3 Headteachers’abilitytoidentify,incentiviseandinfluenceteacherbehaviour

FortheTDPTOCtohold,itisnotsufficientforheadteachersonlytobeincentivisedormotivatedto identify, incentivise and influence teachers to adopt thepositive teachingpractices that TDPpromotes.Headteachersalsoneedtohavetheabilitytoidentify, incentiviseandinfluencesuchchange. This section examines three ways head teachers may have this ability: throughmonitoring,andprovidingadviceto,teachers;throughauthoritydelegatedfromtheLGEA/SUBEB;and through innovative ways of rewarding and disciplining teachers that do not rely onLGEA/SUBEB’sdelegationofauthority.

5.3.1 Headteachers’monitoringof,andadviceto,teachersoncontent,pedagogyandbehaviour

Headteachersmustbeabletoidentifyteacherbehaviourandperformanceinordertoincentiviseand influence the positive teaching practices that TDP promotes. Without this ability, headteacherscannotdistinguishbetween‘positive’and‘negative’behaviourandapplytheappropriateSLMapproaches toencourageanddiscourage these respectively. Information regarding teacherbehaviour and performance is mostly collected through the attendance register and lessonobservations.

Inthequantitativesurvey,thelargemajorityofheadteachers(80%)reportedcarryingoutlessonobservations during the previous two weeks (Figure 14), suggesting that head teachers aresupervising teaching.48 Teachers also reported having their lessons observed on a fairly regularbasisandspokepositivelyofthefeedbacktheyreceived.Examplesoffeedbackdiscussedduringthequalitativeresearchincludedcorrectingspellingsorsubjectknowledgemistakes,encouragingteachers to praise pupils who answer questions correctly, encouraging silent pupils to answerquestions,increaseduseofteachingaids,andcomparinglessonexecutionagainstthelessonplan(ifoneexists).Inonlyoneinstancewasthereagreementbetweentheheadteacherandteachersthattheheadteacher’sadvicewasregularlyignored.

However, such feedback on lesson observations will only result in positive change where thecontent of the advice is appropriate.While the quantitative survey foundhead teachers’ TDNAscores tobestatistically significantlyhigher49 than teachers’ scores, thehead teachers’absoluteTDNA scores (25% items correctly answered) could still be deemed as ‘emerging professionalknowledge’.Therewasnosignificantdifferenceinpedagogicalscoresbetweenheadteachersandteachers.Giventhisfindingofthequantitativebaselineandqualitativeobservations,thereislittlereason tobelieve thathead teachershavesignificantlybetterpedagogicalor subjectknowledgethantheteachersthattheyaremeanttobeprovidingfeedbackon.

48TherewerenosignificantdifferencesinanyoftheindicatorsshowninFigure14betweenTDPtreatmentandcontrolschools.49Ata5%levelofsignificance.


©EDOREN 66

Figure14SLMpractices,basedonheadteacherself-reporteddata

Inthethreestates,30%ofheadteachersreportedholdingformalmeetingswithalloragroupofteachersonceaweekormorefrequently(casualmeetingsarenotincluded).Themostcommontopics discussed in these meetings were: teacher absenteeism and punctuality (70%); pupilabsenteeism (63%); teaching practice and pedagogy (45%); teachers’ professional development(26%); parents and the community (15%); individual pupil needs (13%); materials (9%); schoolbuildingandrepairs(5%);teachers’payandsalaries(3%);andtraining(2%).

Almostallheadteachersreportedtakingactiontoreduceteacherabsenteeism(95%) inthe lastacademic year (no evidence was sought by data collectors to substantiate head teachers’responses).Thereportedactionstakentoreduceteacherabsenteeismwere:discussingwiththeteacher(s)concerned(69%);monitoringtheattendancebookandfollowinguponabsences(47%);insistingonwrittenabsence requests (41%); thecompletionofamovementbookduringschoolhours (30%); addressing pay-related grievances (10%); addressing childcare/maternity relatedissues(5%);addressingalackofteachingmaterials(2%);andaddressingissuesrelatedtoschoolinfrastructureandconditions(1%).

However, teacher behaviour observed in every school visited during the qualitative researchrevealedthat therestillexistsasignificantnumberof teacherswhoareabsent fromclassroomsforlargepartsoftheschoolday,despitetheseteachersbeingpresentattheschool.Thefactthatthis remains the case suggests that the above actions taken to reduce teacher classroomabsenteeism are largely ineffective and therefore raises doubts about head teachers’ ability toinfluenceteacherbehaviourwithoutrecoursetostronger‘carrots’and‘sticks’.

5.3.2 Rewardanddiscipline:Headteachers’formalauthority

Whereinformalfeedbackandadviceisinsufficienttoinfluenceteacherbehaviour,headteachersmay need to resort to using more concrete ‘carrots’ and ‘sticks’ to reward successful, anddisciplineunder-performing,teachers.However,amajorityofheadteachersfeelthattheyarenotempowered by government to do so, especially in regard to their ability to influence teachers’careerprogression.

29.8

79.6

94.5

98.8

70.2

20.4

5.5

1.2

0 20 40 60 80 100

Formalmeetingswithteachers(1+perweek)

HTobservedlessons

HTtookactiontoimproveteacherabsenteeism

HTtookactiontoimprovepupilabsenteeism

Percentageofschools(%)

SLMpracticesbasedonheadteachrself-reporteddata

Yes No


©EDOREN 67

Head teachers frequently complained about the lack of input into the SUBEB recruitment andLGEApostingandtransferprocesses.Inpracticethismeansthatteacherspostedtoschoolsoftenlack the specific subject knowledge required to fill gaps identified by head teachers, and headteachershavenoopportunitytoassessteachersforpersonalitytraitspriortorecruitment,ortoretain high-performing staff. The qualitative research encountered head teachers in all threestateswhowerefrustratedwithteacherswhohadbeenappointedtotheirschoolsonthebasisofpersonalconnectionsratherthanability:

[The LGEA] ‘recruit[s] teachers that are lazy, based on personal connection … The[recruitment]processisnoteventhere,theycannotemployevenacommongatemanforus.Eventheteachersthatteachvoluntarilyhavestoppedcomingbecausewecannotpaythem…Ifinditverydifficulttoretainorsecuretalentedteacherstoteachinthisschool.’–Headteacher

In one case, a teacher admitted that he had secured employment through a family member,despitenothavinganNCE.Onestate-levelgovernmentofficialnotedthatheadteacherscouldnotcrediblypunishthesewell-connectedteachers:

Thereisapoliticalissue.Youcanseeateachernotattendingschoolbutifyoutalktohimhewilldirectlygoand tell eithera councilloror somebodyelsewithahigher rank in thepolitical ladder. So insteadof beingpunished, hewill just lobby [the LGEA] and theywillallowhimtogo.

Thiswasafrustrationthatwassharedexplicitlybyatleastoneheadteacher:

Somepeoplehavegodfatherswhostandforthem,soevenifyouforward[acomplainttotheLGEA],nothingcanbedonetothem–Headteacher

Headteachershavelittleopportunitytoinfluenceteacherremunerationthroughprovidingadviceonpromotionsorthroughrequestingsalarydeductionsasaresultofpoorattendance.

Iwantheadmasters to [beable to]deduct from the teacher’s salary thenumberofdaystheyareabsentfromschoolandthiswillmaketeachersbeabsentonlywhentheyhaveatangiblereason–Headteacher

Inaroundhalfof schools, thequalitative survey found thathead teachersexpressed frustrationthattheLGEAwasunresponsivetofeedbackevenwhenitwasprovided.Aheadteacherina‘low-performingschool’explainedthat‘thereasonwhywedon’treportthemtotheLGEAisbecausethey don’t take any action… I personally never reported any teacher’. In such cases there is adangerthataviciouscycleemergesinwhichheadteachers’lowexpectationsofLGEAsmakethemlessdemandingand,inturn,thisresultsinlowerlevelsofLGEAsupport,thoughthisdynamicwasnotexplicitlyraisedbyinterviewees.However,therewasaperceptionamongasmallnumberofhead teachers that they were not entitled to support from the LGEA on some issues, such asstaffingorresources.Forexample,oneheadteachercommented:‘Idon’tevenfollowuporcalltoaskiftheydon’tdoitbecauseIdon’thavetheright.’

Thislackofformalleverswithwhichtoinfluenceteacherbehaviourwasreflectedinacommonlyobserveddistinctionbetweenthe languageof ‘reward’andthe languageof ‘appreciation’.Headteachers and government officials often noted that in a context in which they lack options to


©EDOREN 68

rewardteachers,thebesttheycandoistoprovidewordsofencouragementandappreciationforteachers’efforts.Aheadteachernoted:

‘[Ionlye]ncourageandappreciatetheteacherandmakecorrectionsfortheteacherwherepossible,thatistheonlyauthorityIhave…[It]istheLGEAthathastheauthoritytorewardanddisciplineteachers.’

ThesenseofdisempowermentwasalsomentionedbyLGEAofficials,someofwhomnotedthatwhereas theyused tobe responsible for takingdecisions relating to recruitment, discipline andpromotion, their role had shifted over time to one ofmerelymaking recommendations to theSUBEB. It was not clear whether this was the result of genuine confusion over the division ofresponsibilitiesor a tactical ‘shifting theblame’ for failing to supporthead teachers inhandlingteacherunderperformance.

Overall, there is strong evidence that head teachers lack the formal authority to administer‘carrots’and‘sticks’thatwouldenablethemtoincentiviseandinfluenceteacherbehaviourintheway that TDP seeks to promote. This findingbroadly corroborates thoseof a previous EDORENteachermanagementstudy(WattsandAllsop,2015).

5.3.3 Rewardsanddiscipline:Headteachers’informalinnovation

Despitetheseconstraints,asignificantminorityofheadteachersappeartothinkcreativelyabouthowtheycan influencebehaviour indirectlywithinanotherwiseconstrainingcontext.Themostcommonpracticewasthegivingoffinancialorin-kindrewardsfinancedoutofheadteachers’ownsalaries.Suchgiftsweregenerallysmallcashrewards(NGN100–200),50food,ortransportcosts,but inone instance includedtextbooksandteachingaids.Sincesuchrewardsaredependentonthe financial resources available to thehead teacher, not all head teachers areable touse thismethod to influence behaviour. As one of the head teachers lamented, ‘Since I don’t haveanythingtogive,Iusemymouthtothankthem’.

Perhaps thegreatestpotential for influencingbehaviour lies inhead teachers’ability toharnessthe formal powers of LGEAs and SUBEBs. A key mechanism for doing this is the teachers’attendance register, which inmost cases is checked by inspectors from the LGEA on a regularbasis.Aheadteachernotedthat‘ifateacherisabsentwithoutareason,todisciplinehimwestophimfromsigningthetimebookandthatreallyhurtsthem’.Althoughinsomecasesheadteacherswereunder socialpressure tomanipulateentries in theattendancebook (generallybymarkinglate-attending teachers as arriving on time), they can in theory signal attendance problems toLGEAinspectors,whocanthentakemattersintotheirownhands.

HeadteacherscanalsoexerciseleadershipbymobilisingSBMCsupportinadisputewithLGEAs,ashappened successfully in a school regardinga teacherdeemedby thehead teacher tobehigh-performingwhowastransferredawayfromtheschool:

‘Iwas posted out from this school to [another] school in 2008, but the SBMC and othermembersof thecommunityprotestedbymeetingwiththeLGEAofficialsandbeggedthemanagementtobringmebacktothisschoolandthatishowIcameback.’

50Enoughforasoftdrinkorsnack.


©EDOREN 69

In a ‘high-performing’ urban school one head teacher took an interest in the regular continualassessmentofpupilstoidentifyhigh-performingteachers,andalsothreatenedtostripteachersofpositionsofauthority(perhapsasheadofsubjectormemberofschoolcommittees)thathehadgiventheminthefirstplacetopunishpoorlyperformingones

These techniques highlight the importance of strong SLM skills to overcome the limitations ofheadteachers’formalpowers.However,inmostschoolsitappearedthatheadteachersweresodeeply involved in the day-to-day administration that they had little time for critical reflection,creativityandacademicleadership.

Table26Summaryofstrengthofevidenceunderpinningassumptionsrelatingtoheadteachers’ability


Strengthofevidence

Headteachershavetheabilitytoidentify,incentiviseandinfluencethepositiveteachingandmanagementpracticesthatTDPpromotes

• HeadteachersareabletoidentifyanddifferentiatepositiveandnegativeteachingpracticesandprovidefeedbacktoteacherstoinfluencethepositiveteachingpracticesthatTDPpromotes

Mixed

• HeadteachersareformallyempoweredbyLGEAs/SUBEBstouse‘carrots’and‘sticks’effectivelytoincentiviseandinfluencethepositiveteachingpracticesthatTDPpromotes

No Strong

• Headteachersareabletofindinformal(i.e.non-governmentmandated)waysofusing‘carrots’and‘sticks’effectivelytoincentiviseandinfluencethepositiveteachingandmanagementpracticesthatTDPpromotes

Mixed

5.4 Managementofschoolinfrastructureandresources

SLM include tasks relating to the school’s physical as well as human resources. Section 4.4discussed the importance of adequate school infrastructure and resources in providing asupportiveenvironmentforpractisingteachingpracticesthatTDPpromotes.Inadequatephysicalinfrastructure in schools and lack of teaching materials can reduce teacher motivation, andtherefore teacher effectiveness. This section explores the issue of inadequate schoolinfrastructure and resources, and highlights theways inwhich head teachers are supported byotheractorsinmaintainingandrepairingthese.

5.4.1 Governmentandcommunityrolesinmaintenanceandrepairs

Brokenceilings,furniture,andwindowswereallfrequentlyobservedinthequalitativestudy.Withtheexceptionofafewurbanschools,headteachersalsocomplainedaboutthelackoftextbooksand one ‘low-performing school’ had stopped being provided with chalk by the LGEA, andthereforehadrunoutatthetimeofthesurveyteam’svisit.

The division of responsibilities between government and non-government organisations formaintainingschoolinfrastructureandresourcesisnotclear.Headteachersfrequentlynotedthattheirrequestforadditionalinfrastructure,repairs,andresourcesfromtheLGEA/SUBEBhadgoneunanswered. About 72% of schools reported receiving in-cash or in-kind support from non-


©EDOREN 70

governmentorganisations (NGOs)orprogrammes.51Of these, about70%of sampled schools inJigawa reported being in receipt of support from the DFID-funded ESSPIN, while a similarpercentage of schools in Katsina and Zamfara reported being in receipt of support from GEP,anotherDFID-fundedprogramme.

Headteachersspokefavourablyofthefinancialandin-kindsupportprovidedbySBMCsandPTAs.Innearlyallschools,localcommunitiesplayasignificantroleincontributingmodestfundstobuyteaching resources52 and mobilising expertise to fix broken school infrastructure. It was notuncommontofindthatparentsorothercommunitymembersarerequiredtocontributeasmallfeetoraisefundsforrepairsandthepurchaseofnewmaterials.ItisunclearwhetherSBMCsareofficially responsible for such a role, or whether they step in to fill the void left by theLGEA/SUBEB’s failure to discharge their responsibility, and in doing so inadvertently undermineincentives for LGEA/SUBEBs to perform this role effectively. Overall, this reflects the richconfluenceofcommunity-based,nationalandinternationaldonorswithwhichTDPwouldneedtointeract during its years of implementation, especially with other large education programmesrunning similar teacher training activities such as ESSPIN and GEP. However, the frequency ofrepairsreportedasbeingneeded,combinedwithobservationsmadeduringthequalitativestudyandtherelativelysmallscaleofcommunitysupport,suggeststhat inthemajorityofcasesheadteachersdonothaveaccesstoadequatesupporttomaintainandrepairtheschoolinfrastructureandresourcesthatfacilitatetheadoptionofthepositiveteachingpracticesthatTDPpromotes.

5.4.2 Relativelywellresourcedschools

Twoofthenineschoolsvisitedaspartofthequalitativeresearchstoodoutashavingvastlybetterinfrastructure and resources thanother schools, bothofwhichhad received significant supportfromtheSUBEB.Itislikelythat,assourcesintheschoolclaimed,theserelative‘anomalies’aroseduetotheirconnectionstoinfluentialpoliticiansatthestateandfederallevelwhowereabletodirectresourcesintheirdirection.Inoneoftheseschools,theheadteachernotedthatwhereas:

‘...nearbyschoolslackclassroomsresources,nogoodinfrastructure,theofficialsdon’tvisitthoseschoolandtheteachersthereregularly,inourowncasewehaveeverything,enoughclassroomresources,eventhesetwoadditionalblocks,wereprovidedtousby[nameofinfluentialpolitician].Wehaveenoughdesksandenoughteachingmaterialsaswellasqualifiedteachers,allasaresultofhisassistance.’–Headteacher

At amore general level, urban or semi-urban schoolswere generally found to bemuch betterequippedthanruralschools,withoneschoolevenhavingalibrarywheretextbookswerestored(thoughnotused).However,onegovernmentofficialcomplainedthat

‘Whenyouvisitsomeschoolsyoufindalotoftextbooksintheirstores.Thisiswhywhenwegoforsupervisionsweaskthemtoshareoutthewholebooks,buttheycomplainthattheyareafraidofsharingthemoutandnotgettinganyotherones…Buttheyweretoldtogiveoutthosebookstothepupils!Thisisachallenge.’

51Ofthe72%schoolswhichreportedreceivingnon-governmentsupport,about24%receivedsupportfromprivateindividualsorfirms;16%of schools received support fromanNGO;7% from internationaldonorsother thanDFIDorUNICEF;and2% fromareligious institution (e.g.churchormosque)orprivate individuals.Furthermore,25%ofschools reportedreceiving in-cashor in-kindsupportfrom‘othersources’,andtheseweremostlyspecifiedasbeingfromSBMCs.52PurchasesfinancedbySBMCs/PTAsmentionedinthequalitativeresearchincludedwritingmaterials,textbooks,brooms,kettles,mats,andfootballs.


©EDOREN 71

This pattern of unused teaching resources suggests that even in the rare caseswhere they aresufficientschoolresourcesarenotalwaysmanagedandallocatedefficientlyundertheguidanceofhead teachers.53 On the one hand, this will limit the effectiveness of TDP teacher training byreinforcingthechallengesdiscussedbyteachers inSection4.4.Ontheotherhand,thissuggeststhatthere issignificantscope insomecasesforTDP’splannedheadteachertrainingto improveapproachestoresourcemanagement.

Table27SummaryofstrengthofevidenceunderpinningassumptionsrelatingtoSLMofschoolinfrastructureandresources


Strengthofevidence

HeadteachersareabletoexerciseeffectivemanagementofschoolinfrastructureandresourcestofacilitatetheadoptionofthepositiveteachingpracticesthatTDPpromotes

• HeadteachershaveaccesstoadequatesupporttomaintainandrepairtheschoolinfrastructureandresourcesthatfacilitatetheadoptionofthepositiveteachingpracticesthatTDPpromotes

Mixed

• HeadteachersmanageandallocateschoolresourcesefficientlytofacilitatetheadoptionofthepositiveteachingpracticesthatTDPpromotes

Mixed

53TheWorldBankfoundasimilardynamicinarandomisedevaluationofatextbookprovisionprogrammeinSierraLeone.Inmanytreatment schools, student access to textbooksdidnot increasebecausea largemajorityof thebookswere stored rather thandistributedtostudents.Theyhypothesisethatschoolswithahighuncertaintywithrespecttofuturetransfersaremore likelytostoreaproportionofcurrenttransfersasaformof‘smoothingconsumption’(Marshak,Evans,andSabarwal2014).


©EDOREN 72

6 TDP’sdeliverymodelforin-servicetraining

6.1 Scopeandrationale

ThispenultimatesectionsummarisespreliminaryobservationsregardingTDP’simplementationofin-servicetrainingactivities.Itisbasedona‘lighttouch’processevaluationundertakenaspartofthebaselinequalitativestudy,whichtookplaceroughlysixmonthsaftercommencementof theimplementation of in-service training activities. This gave the authors a nascent opportunity tostudy various implementation processes and practical dynamics, and in turn to provide initialimpressionsofhowimplementationofthein-serviceoutputisprogressing.

TDP’s in-servicedeliverymodel ismulti-pronged. It involvesscheduledmonthlyclassroom-basedtraining sessions (‘cluster meetings’) where pairs of teachers within a school are ‘coupled’ toattendthetrainingtogether(peer-to-peerlearning),alongwithotherparticipatingteachersfromtheLGEA.Trainingmaterialsreceivedbyteachersarealsonumerous.Printedmaterials,includingstructuredlessonplansandguides,aredistributed,aswellasaudio-visualresourcesthroughonebasicmobilephoneperteacher(‘atrainerinthepocket’)andtwoamplifier-radiosperschoolforuseinclassrooms.Finally,TDPteachersreceiveongoingguidancethroughschoolsupportvisitsbytheirtrainers,andheadteachersinTDPschoolswillreceivetrainingonSLM.TeachersaretrainedbyaselectedcadreofTFswhoareinturntrainedbyagroupof‘mastertrainers’recruitedfromSUBEBsandCoEs,calledtheTeacherDevelopmentTeam(TDT).Morediscussiononsomeoftheseaspectsof implementation isprovided in thesectionbelow,andcanalsobe found inTDP2014andEDOREN2014.Thetheoryunderlyingthein-servicetrainingcomponentisthatpupilslearnmorewhentheyaretaughtby‘effective’teachers, andthatteachersbecomemoreskilledandknowledgeable(bothintermsofpedagogyandsubjectknowledge)throughtraining.Asdiscussed inthesectionsabove,this theory rests on a number of key assumptions relating to (head) teachers’ ownmotivation,ability,andcontext.Additionally, thistheoryalsorestsonanumberofoperationalandprocess-basedassumptions,which,basedonourtheoreticaldeductionandfieldobservations,wouldneedto hold in order for the various trainingmaterials and activities to result in teachers becomingmoreskilledandknowledgeable.Thissectionprovidesinsightsintosomeofthesekeyoperationalandprocess-basedassumptionsthat underpin the logic of the TDP TOC. To reiterate, this study was undertaken in the initialmonths of implementation, covering nine schools and, as such, does not intend to be acomprehensiveanalysisofwhether theseassumptionsholdornot ineach instance.54 Instead itprovidesapreliminarypictureofhowandwhythedeliverymodelforTDP’sin-servicetrainingwilldetermineteachereffectivenessandultimatelyenhancedpupillearning.

Three overarching assumptions relating to the core features of TDP’s delivery model wereidentified for this (qualitative) process review on the basis of the qualitative evaluationmatrix(AnnexA,VolumeII)andthematicanalysisofthedata.Theseare:

• Assumption 1: Cluster meetings and peer-to-peer learning – Cluster meetings areconducted regularly and effectively, and teachers aremotivated to participate in them.

54Thescopeofthisanalysisdoesnot includeanassessmentofTDP’sorganisationalsetupatthecentralmanagement(Abuja)orstatelevels;neitherdoesitincludeassessmentoftheheadteachertrainingcomponentonSLM(whichhadnotstartedatthetimeofourfieldwork)–noranin-depthappraisalofthepedagogicalcontentcontainedinthetrainingmaterial.


©EDOREN 73

Teachers’learningfromclustermeetingsisfurtherreinforcedbypeer-to-peerinteractionswithothertrainedteachersinschools.

• Assumption 2:Materials – Both printed and audio-visual training materials are fit-for-purposeandappropriateforthelevelsofskillsandknowledgeteachershaveandtheday-to-daychallengestheyencounterinclassrooms.

• Assumption 3: Teacher facilitators – TFs who constitute the core frontline trainingworkforcefortheprogrammeareappropriatelyselectedandtrained,andhavethecorrectworkload,todelivertheirroleeffectively.

Dataforthefollowinganalysiscomefromqualitativeinterviewswithrecipientandnon-recipientteachersandheadteachers,TFs,clustermeetingobservations,observationoftraining-of-trainersessions,andwithstaffatLGEAsandSUBEBs.

6.2 Clustermeetingsandpeer-to-peerlearning

Theresearchteam’sfieldworkscheduleinJune2015allowedforobservationofaroundsixclustermeetingsacrossJigawaandZamfarathatweretakingplaceaspartofthatmonth’sinstalmentoftrainingsessions.Todescribethesemeetingsbriefly:duringclustermeetings,whichlasttwodays,TDPteachers fromschoolswithinanLGAgenerallymeet inacentrally locatedschooltoreceivetraining on the pedagogical material for that month. Each LGA has 12 TDP schools and fourteachersparticipatingfromeveryschool.TeachersareassignedtoeitherEnglishormathstraining,andhalfof them(i.e.around24)attendthe firstday’s trainingononeof thetwosubjects (say,English),andtheremainingattendthesecondday’strainingontheothersubject(e.g.maths).TwoTFsareassignedtoeachLGAandtheydistributethetrainingloadamongstthemselves.

6.2.1 Monetaryincentivesandteachers’motivationtoparticipateinclustermeetings

A recurring theme arising in teachers’ feedback on cluster meetings related to their unmetexpectation of compensation, in addition to the transport costs and meals provided to themduring themeeting. The absence of such compensationwas a ‘complaint’whose frequency farexceeded any other feedback on cluster meetings relating to training activities, materials orquality of teaching. To be specific, teachers voiced that a travel allowance of NGN 2,500 permeeting was inadequate to cover a return trip to the venue (around NGN 500–1,000) andadditional costs of going to the bank to withdraw the allowance (especially for teachers whoresided in ruralpartsof theLGEAandhadto travela longdistance to findabankorautomatictellermachine).Moreover,wheninadequatenumbersofmaterialswerereceivedbyTFs,teacherssometimeshadtopayforphotocopyingthem.Onarelatedpoint,themodeofpaymentoftheperdiem (e-paymentsdirectly intoparticipants’bankaccounts,presumably toensure transparency)was another source of complaint. Teachersmentioned long delays in receiving these payments(oftenup toeightweeks from thedateof themeeting),meaning theyhad tobear theupfrontcostsoftransport.

ATFsuccinctlyexplainedthatwhat, intheend,motivatesteacherstoattendclustermeetingsisthe amount theymanage to save from theirmonthly per diems after deducting transport andother costs of participation in clustermeetings. Coupledwith poverty, and low and late salarypayments, this indicates a severe offsetting of teachers’ intrinsicmotivation to attend training:‘themoney to convey them to the clustermeeting is the greatest problem,what ismotivatingthemisthelittleallowancetheyreceivedfromTDP,buttheydon’tgettheallowanceontimeand


©EDOREN 74

coupledwiththefactthattheyarenotgettingtheirsalaryontime,intermsofparticipationtheyaretryingtheirbest…’.

Thispoints to theneedtounderstandwhatmotivates teachers toparticipate inany formof in-servicetraining.AsmentionedinSection4.3above,therearecertainlyteacherswhotakeprideintheirprofessionandseekoutopportunitiesforlearning.However,giventhelevelofpovertyofthepoorlypaid(ifatall) teachingworkforce,whatappearstobetheoverwhelmingcase is thatanysuchintrinsicmotivationiserodedbyextrinsic,pecuniarymotivation.

6.2.2 Levelsofparticipationandeffectivenessofclustermeetings

ItwasreportedbyTFsandteachersthattheclustermeetingswerebeingconductedregularlyandasperschedule.However,thegenerallevelsofteachers’participationandeffectivenessofclustermeetingswasobservedtobeweakoverall.TFsspentlargeamountsoftimewritingonflipchartsor the blackboard, oftenwith a series of spelling or grammatical errors, against a backdrop ofteacherswhowereclearlydisengagedwithvastamountsofinstruction,andwithlittlediscussion.A handful of teachers (oftenmale, and head teachers) dominated clustermeetings in terms ofresponsestoquestionsandeagernesstoparticipate.WhileTFsoftentriedtoencouragebroaderparticipationandattentionfromacrossthegroup,theireffortswerenotnecessarilysuccessful.

Female teachers often constituted a small minority of those attending the cluster meeting, areflection of the low levels of female participation in teaching and theworkforce in general innorthern Nigeria. Females attending clustermeetings often displayed far less participation andwere more often late, presumably due to family commitments and living far from the venue.Thosethatdidattendhadtotakefrequentbreakstoattendtotheiryoungchildren.

AnadditionalimportantdriverofweakoverallparticipationmayhavebeenthesignificantuseofEnglishinclustermeetings,giventhelowlevelsofcompetencyinthelanguageamongteachers,asevidenced in the quantitative baseline survey. Teachers were often lectured on pedagogicaltechniques inEnglishusingrelativelydifficult termstheywereunfamiliarwith (e.g. ‘segmenting’and ‘blending’ in phonics instruction). Thus, similar to the case in primary school classrooms,clustermeetings also encountered complex language challenges in teaching and learning:whileteachers’ownknowledgeofEnglishwasgenerallyweak,manyofthediscussions(andreadingandwriting)inclustermeetingsstilltookplaceinEnglish,withoutadequateadjustmentandrepetitionbyTFsinHausa–thelanguageavastmajorityofteachersarefamiliarwith.

6.2.3 Peer-to-peerlearning

A key pillar of TDP’s in-service training model is peer-to-peer learning. This mode of learningexpectsthatpairsofteachersfromaschoolundertakingtrainingtogetherwillprovideanongoingsupport network for teachers within their own school to discuss the material taught andchallengesfacedintheclassroom.Accordingtothemodel,thiswouldalsothengraduallyleadtoabroaderdiscussionofmaterialswithintheschoolamongTDPandnon-TDPteachers,leadingtoawholeschoolimprovementmodel.

Inclustermeetings,TFstrytofacilitatepeer-to-peer learningbyseatingteachersfromthesameschool near each other, to encourage discussion among them. This is of course not easy,particularly if there is not a pre-existing cultureof collaboration to attain commonprofessionalgoals,andisfurthercompoundedbythepresenceofageorgenderorothercultural,socialand/or


©EDOREN 75

economicbarriersamongteachers.Thereappearedtobenascentsignsofpeer-to-peer learningbasedonteachers’anecdotesofobservingtheirpeers’lessonsanddiscussingcommonchallenges;however, these were not observed by the research team directly in any of the schools.Absenteeismofteachersfromschool,especiallyonlongperiodsforstudy,couldpotentiallyputatriskanyincipientsignsofpeer-to-peerlearningbecauseofthelimitedtimespentinface-to-facecontactwithone’speer.

AsforsharingofknowledgeandmaterialsbyTDPteacherswithotherteachersintheschool,theevidenceseemstosuggestthatpresentlynosuchexchangeappearstobetakingplace,orisverylimitedand ishappening in rare casesonly. TDP teachersoftenclaimed tobe sharingmaterialsamongcolleagues,whilenon-TDPteachersclaimedthatthiswasnottrueandinsistedontheneedfordirectTDPtrainingforthemselves.Onarelatedpoint,inoneschoolcomplaintsoffavouritismby thehead teacherand local authorities towardsTDP teachers in regard to selecting them forparticipation in the programme, and instances of hostility between TDP and non-TDP teachers,werereported.Inthisparticular instance,thenon-TDPteacherswereaggrievedaboutnotbeingselectedtoparticipate,eventhoughtheyfulfilledtherequiredselectioncriteria.Suchaggravationismorelikelyinlargeschoolswithmanyteachers,lesssoinsmallschools.

Table28Summaryofstrengthofevidenceunderpinningassumptionsrelatingtoclustermeetingsandpeer-to-peerlearning


Strengthofevidence

Clustermeetingsandpeer-to-peerlearning–Clustermeetingsareconductedregularlyandeffectively,andteachersaremotivatedtoparticipateinthem.Teachers’learningfromclustermeetingsisfurtherreinforcedbypeer-to-peerinteractionswithothertrainedteachersinschools.

• Teachers’aremotivationtoparticipateinclustermeetings(andtheirmotivationisnotoffsetbycountervailingincentives)

No Strong

• Generallevelsofteachers’participationandeffectivenessoftrainingactivitiesinclustermeetingsishigh

No Strong

• Teachers’learningfromclustermeetingsisfurtherreinforcedbypeer-to-peerinteractionswithothertrainedteachersinschools

No Strong

6.3 Contentandusageofmaterials

TDPteachersreceiveabundleofmaterials,bothprintedandaudio-visual,aspartoftheirtraining:this includes lesson plans and teachers’ guides (printed) and the phone-based ‘trainer in yourpocket’,whichcontainsuploadedtrainingmaterialsandotheraudio-visualmaterialsforuseintheclassroom.

6.3.1 Relevanceoftrainingmaterials

Asmentioned in Section 4.4 above, teachers face severe day-to-day constraints in the form ofinsufficient infrastructure and materials in classrooms, coupled with their own weak subjectknowledge and pedagogical skills. Any programme aiming to enhance teachers’ knowledge andskills in this difficult context would thus need to help teachers deliver instructional content topupils in large, (often)multi-grade classrooms to thebestof their abilities, given teachers’ownweaknessesinknowledgeandskills.


©EDOREN 76

Do TDP materials address these challenges sufficiently? First, training material and clustermeetings do talk at length about some of these challenges, mainly classroom managementtechniques in largeclasses.However,fieldobservationsofferedlimitedevidence, ifanyatall,ofanyperceptibleshiftinteachers’behaviourandskillsintheclassroomamidstthesechallenges.Itislikelytobetooearlyintheprogramme’slifetoexpectchangesindeeplyentrenchedclassroompractices, and indeed field observations suggest this is the case. However, what was clearlyobserved in classroomswas a certain superficial adoption of some of the techniques taught toteachers in regard to child-centred learning. For instance, teachers displayed a tendency toorganisepupilsintogroupsirrespectiveofwhetherthecontentbeingtaughtrequiredgroupworkornot, leadingtoacertain‘isomorphicmimicry’55ofpracticesthataredeemedtobeassociatedwith superior performance, irrespective of context or need. However, it was not possible toconduct a detailed pedagogical appraisal of the trainingmaterials under this study due to timeconstraintsandthelackofapedagogyexpertintheteam.

Anumberofmore specific issueswere raisedby teachers andTFs aspart of their feedbackontrainingmaterials.Thelessonplans,itwasmentioned,weredesignedforone-hourlessons,whilethestandard lesson length inpracticewastypically35minutes–thus leavingteacherstoeithercramanhour’sworthofmaterialintoapproximatelyhalfthetimeorworkoutforthemselveshowtomodifythelessonplanstotheshorterlessonlengths.Thelessonplans,alsousedbyESSPINinKanoandJigawa(called‘Kan-Jiga’lessonplans),werefairlydetailedandstructuredtotheextentthat page numbers of specific textbooks of specific publishers were used to guide a teacher’slessonthroughaclass,evenwhentheparticularpublisher’stextbookswerenotreadilyavailablein some of the TDP states (Katsina and Zamfara), thus leaving teachers unable to fully followlessonplanswith thecorrespondingcontent in textbooks. Itwasnotpossible todefinitively saywhetherandtowhatextentteacherswereabletoadaptthese(longer)lessonplanstotheirclasslengths.

Usuallyclustermeetingsbeganwithabriefsessionon listingsuchchallengesandfeedback,andtheredidappeartobemechanismsforTFstoroutethesemessagesuptoprogrammestaffinthestatecapitalsandAbuja:forinstance,seniorprogrammemanagersinAbujawerecertainlyawareofthe lesson length issuepertainingto lessonplans.What is lessapparent iswhethertherearemechanismsforsuchfeedbacktotranslateintoactionandrectification.

6.3.2 Usageofmaterials

As mentioned above, aside from printed materials, each teacher also receives audio-visualmaterialuploadedontoasimplemobilephoneforuseintheirowntime(‘trainerinthepocket’);and each school receives twomobile amplifiers to play a variety of such audiomaterial in theclassroom.

Howoftenarethesenovelmaterialsbeingused?Inmostschoolsvisitedforthequalitativestudy,teachers had nowmade the ‘trainer in your pocket’ their personal phone formaking calls andsending text messages – this is not necessarily undesirable since using the phone for otherpurposesmightalsopromptthemtousethephone-basedtrainingmaterialsmorefrequently.By

55 ‘Isomorphism’ and ‘isomorphicmimicry’ are terms from evolutionary biology, popular since the 19th century,which refer todifferentorganismsevolvingtolooksimilarwithoutactuallybeingrelated.Inparticular,isomorphicmimicryistheprocessbywhichoneorganismmimicsanothertogainanevolutionaryadvantage.Latelythetermhascaughtonintheareaofeducation,courtesyof LantPritchett, and isused to refer to fundamentallydysfunctionaleducation systems that look likewell-performing systems.Suchdysfunctionalsystemspretendtoconductteachingand learning likethekindthatgoeson infunctionaleducationsystems,butwithouttheircoreunderlyingfunctionalities.


©EDOREN 77

andlarge,itappearedthatusageofthesematerialsvariedsubstantiallyfromcasetocase,largelydependentonteachers’owninitiativeandinterest.TherewasateacherinJigawawho,byhisowninitiative,nowrecordednewsongsandpoemsontothephoneandplayedthembacktohisclass.Ontheotherendofthespectrum,therewasalsoateacherwhothought‘trainerinyourpocket’essentiallyreferredtotheperdiemhecouldpocketeverytimeheattendedaclustermeeting.Onthewhole itwasnotrare forheadteachers tosendapupilora teacher tohishometo fetchamobileamplifierbecausevisitors,i.e.theresearchteam,hadarrivedintheschooltotalktothemaboutTDP.Similarly,itwasnotuncommontoseeentirelyunusedlessonplansandteacherguidesstored in the school library. In sum,while thereare individual casesof teacherswhomayhavetaken to the new materials and technology more effortlessly, there is limited evidence ofperceptibleshifts towards theirusageand it ishopedthatas theprogrammematures teachers’useofmaterialswillincreaseovertime.

6.3.3 Toomanyteachertrainingprogrammes?

Training of primary school teachers in northernNigeria is a crowded space, i.e. inmany cases,schoolsandteachersareinreceiptofmultiplestate-ordonor-fundedin-serviceteachertrainingprogrammes.Forinstance,allofTDP’sschoolsinJigawaarealsoinreceiptofanotherDFID-fundedprogramme,ESSPIN,whichalsotrainsteachersinEnglishandmaths,andinfactbothprogrammesusesimilarmaterials.Multipletrainingmeanseachprogrammecomeswiththeirownsetofjargonrelatedtopedagogicalpracticesandtechniques.Thisoften leadstosituationswherebysomeofthesamepedagogicalconceptsarereferredtobydifferentterms,leadingtoconfusionamongtherecipientteachersandeventrainers.

For instance, teachers suspected what was being taught to them as using ‘no-cost/low-costmaterials’ in theclassroomwasessentially thesameas theconceptof ‘improvisation’ taught totheminpre-servicetraining(forthosewhohasanNCE).Similarly,someteachersinZamfarawereinreceiptofJollyPhonicsaswellasTDPtraining,bothofwhichtrainonhowtoteachchildrentophoneticallybreakdownwords inorder to facilitatereadingskills.However,clustermeetings inthe Zamfara clearly demonstrated teachers’ (and trainers’) confusion about how to teach usingphonicdeconstruction,oftenmixingupnamesofletterswiththeirsoundswithinthesameword.While it is not being suggested here that a single programme such as TDP should take up themantleofharmonisingtheseoverlappingtechniquesacrossprogrammes,theresultingconfusioniscertainlysomethingtoconsiderwhendesigningpedagogicaltrainingmaterialforaprogramme.

Table29Summaryofstrengthofevidenceunderpinningassumptionsrelatingtocontentandusageoftrainingmaterials


Strengthofevidence

Materials–Bothprintedandaudio-visualtrainingmaterialsarefit-for-purposeandappropriateforthelevelsofskillsandknowledgeteachershaveandtheday-to-daychallengestheyencounterinclassrooms.

• Trainingmaterialsarerelevanttothesubjectknowledge,pedagogicalandlanguageskillsofteachersandtheirday-to-dayclassroomchallenges

No Strong

• Therearemechanismstofeedbackchallengesfacedbyteachersinusingthesematerials(toTDPforactionandrectification)

Mixed

• TrainingmaterialsareactuallyusedbyTDPteachersregularlyforself-learninganddiscussionswithotherteachers

Mixed


©EDOREN 78

6.4 TFsareappropriatelyselectedandtrained,andhaveamanageableworkload

Asdescribedabove,TFsareavitalcadreoffrontlinetrainingpersonnelwhointeractwiththeTDPbeneficiaryteachersmonthlythroughclustermeetingsandschoolsupportvisits.Theyreceivetwodays’trainingeachmonth(oneeachforEnglishandmaths)fromtheTDTs(‘mastertrainers’)andsubsequentlytraintheteachersforadayeachonthetwosubjects.Thislastsub-sectionlooksathowTFswere selected, theirbackgrounds, the training receivedby them,and theirworkload–andwhatthisimpliesfortheireffectivenessasteachertrainers.

5.1.1. Backgroundandskills

ItappearedfrominterviewswiththeTFsthatavastmajorityofthemhadpreviousbackgroundsasclassroom teachers and/or head teachers, with requisite NCE (or higher qualifications); thustheoreticallyplacingtheminagoodpositiontounderstandtheconstraintsteachersfacedailyintheir jobs,especially in the classroom,and to try toaddress these through training. Inpractice,while TFs appeared to bewell aware of extraneous constraints faced by teachers (‘disengagedparents’, ‘untalented pupils’, ‘low salary’), most of them seemed fairly unaware (or at leastunwilling todiscuss) teachers’ ownweaknesses in termsof knowledgeand skills. Poor teachingqualityandlowteachereffectiveness,inthewordsofoneTF,camedowntoteachersessentiallybeing ‘lazy’ and ‘incompetent’. Thus, ultimately it remains unclear how much of their ownbackgroundandexperienceTFsareabletoharnessforeffectivetrainingofteachers.

AfurtherquestiononTFsrelatestotheadequacyofthetrainingreceivedbythem,andwhetherthis prepares themwell to undertake the difficult task of teacher training. A TF described thetrainingshereceivedasfollows:

At firstwewere taken to Abuja for 10 days’ training on special skills and techniques ofteaching. They started with mathematics, English and said they are going to includesciences.Andwhen they started, they said theyaregoing to trainedusonhow to teachprimary 1, 2 and 3, then 4, 5 and 6, then JSS1–JSS3.Wewere trained on how to teachmathematics and English in primary 123. After the 10 dayswe return andwewere stilltrainedintheLGEAheadquartersandwesteppeddownthetrainingtootherteachers.(TF)

This initial training was followed by a two-daymonthly training session with the TDTs on thatmonth’smaterials, justbefore theTFs themselves trained the teachers. Is this trainingoptimal?TFs themselves largely felt thequantityof trainingwasadequate,but thequalityof teaching inclustermeetings varied substantially from one TF to another, as evidenced by clustermeetingobservations.Asmentionedabove,asinschoolclassrooms,TFsdisplayedlimitedusageof‘child-centred learning’, spending largeamountsof timewritingon flipchartsor theblackboard,oftenwith a series of spelling or grammatical errors, against a backdrop of teachers being clearlydisengaged and with large amounts of instruction but little discussion. TDTs, who are oftenavailable and at hand to correct TFs, often have to intervene to correct them or manage themeeting. Having said this, it is not a foregone conclusion that more training for TFs wouldnecessarily leadtobetterteaching–constraints imposedbyabsorptivecapacitymightsoonkickin, combinedwith the fact that TFs usually also have to deliver their regular jobs in the LGEAsduringtheremainingdaysofthemonth.Thisisdiscussedfurtherinthesub-sectionbelow.


©EDOREN 79

6.4.1 TFs’workloadandtrainingresponsibilities

On paper, a TF’s role includes three key tasks: a) attendingmonthly trainingwith TDTs beforecluster meetings (two days); b) conducting monthly cluster meetings and reporting onprogrammestoTDTs(twodays);andc)conductingmonthlyschoolsupportvisits(threedays).Inpractice, however, the gamut of activities TFs perform goes above and beyond these threeactivities. It thus requires a fair amount of displacement of time from their regular jobs in theLGEA (say, as school inspectors or support officers) or, in some cases, as regular classroomteachers. TFs directly or indirectly, inmany cases, spoke of the immenseworkload on them inorganisingclustermeetings,includinglogisticsandadministrativeactivities.InconversationswithTFs, it was calculated that on average a TF spent about seven days for cluster meetings andinspections per month (two days for TDT training, two days for cluster meetings, andapproximately three days for school support visits) but this does not take into account anyperipheral preparatory reading they had to undertake for cluster meetings. Most importantly,these seven days do not take into account large amounts of administrative tasks TFs have toundertakebeforeclustermeetings–not to forget their regular jobsasLGEAstaff.TFs routinelycomplainedofnotreceivingadequatecopiesofteacherguides,lessonplansandotherhandouts,whichmeant theywouldhave tomakephotocopies themselves (oftenusing theirownmoney).TFs similarly voiced frustration regarding having to spend their own money supplementingstationary, flipcharts, etc. received fromTDP inorder to carryout the vastnumberof activitiesexpectedtobecompletedduringasingleclustermeeting.

Table30Summaryofstrengthofevidenceunderpinningassumptionsrelatingtoappropriateselection,trainingandmotivationofTFs


Strengthofevidence

TFswhoconstitutethecorefrontlinetrainingworkforcefortheprogrammeareappropriatelyselected,trained,andmotivatedtodelivertheirroleeffectively

• TFsareappropriatelyselected:i.e.theyhaveasuitablebackgroundandexperiencetounderstandtheconstraintsfacedbyteachers,andaddressthemeffectively

Mixed

• TFsareappropriatelytrained Mixed

• TFsareappropriatelymotivated,andinparticulartheirworkloadandsuppliesfromtheprogrammeallowthemtodelivertheirroleeffectively

No Strong


©EDOREN 80

7 Conclusionanddiscussion

One of the primary objectives of this mixed-methods report is to provide baseline results forpotential adjustments to TDP’s design and implementation, as deemed appropriate by theprogramme and DFID Nigeria. The section below draws the report to a conclusion by firstsummarising key results and then highlighting their implications for the programme. Plannedfollow-upevaluationactivitiesarealsodiscussed.

7.1 Pupillearninglevelsandoverallexperienceofschooling

Pupil learning levels in English,maths and science are very low in the treatment and controlareas.OnlyverysmallgroupsofGrade3pupilsdemonstratethebasicEnglishliteracyskills(3%),basicnumeracyskills(6%),andscientific literacy(15%),expectedattheendofGrade2.Ofevenmoreconcernarethesubstantialproportionsofpupilswhoarebelowtheexpectedgradelevelbyabout two full grades. In all three subjects, girls andpupils from thepoorest householdwealthquintile perform significantly worse than boys and pupils from the richest household wealthquintile,respectively.

Qualitative accounts frompupils depict a vicious cycle linkinghouseholdpoverty toexclusionfromlearninginschoolandbroadersocialexclusion.Thisiscorroboratedbyquantitativeresultswhich show that differences in pupils’ own family backgrounds and community-levelcharacteristics explain the majority of variation in pupil learning. Teachers often allege thatpupilsdonotattendschoolregularlybecauseof(paidorunpaid)work,orbecausetheirparentssimultaneously enrol them in Islamic education lessons outside the school. Although primaryschoolingisfreeinNigeria,schoolssometimeschargefeesforexamsorPTAcharges,orchargestowardsthecostofuniforms.Thereare,therefore,indirectschoolingcostsevenwhenthedirectcosts of schooling are presumably minimal or even free – putting pupils from the pooresthouseholdatadisadvantage.Mostpupils’accountsrevealanacceptanceofthefactthatteachersoften do not come to class on time, or at all. Pupils in schools with poor infrastructure areconsciousof theproblemsthiscauses for their teachers indeliveringtheir roles– infrastructurefeaturesprominentlyamongproblemspupilswouldsortoutiftheyweregrantedmagicalpowers.

Thispictureoflowlearninglevelsandpupils’experienceofschoolingatbaselineconfirmsthatthecoreobjectiveoftheprogrammeofimprovingpupils’learninglevelsishighlyrelevantintheprogramme LGAs. Pupils’ own family backgrounds and community-level characteristicsmay bebeyondthescopeofateacherdevelopment intervention,yettheseare likelyto impingeontheeffectivenessofTDP–thisisindeedasoberingfindingforaprogrammethatisdrivenbyschool-basedinterventionsalone.Thesefactorsneedtobetakenintoaccountintheprogrammedesignandinassessingtheprobabilityoftheprogramme’ssuccess.Itisimportanttoensurethatpupilsfrompoorerbackgrounds,aswellasgirls,areconsideredasatargetgroupinthecoreobjective,and that the teacher training materials and activities consider how schools can address theirparticularlearningneedsinwaysthataremoreinclusive.


©EDOREN 81

7.2 Teachereffectiveness

7.2.1 Teachers’subjectknowledgeandabilitytomonitorpupils’progress

Forpupilstobeabletolearnwhileintheclassroomteachersmusthavesufficientknowledgeofthesubjecttheyareteaching.ThevastmajorityofteachersintheimpactevaluationareashavegrosslyinadequatesubjectknowledgeinEnglish,mathsandsciencetobeconsideredeffectiveintheclassroom.Just0.1%ofteachershavesufficientsubjectknowledgeinscienceand0.4%inEnglish, and the groupwith sufficient subject knowledge inmaths, although larger, is still verysmall,at8%.Themostcriticalgroup,whichhasonlylimitedsubjectknowledgeandwhich,withoutextensiveandcontinuoustrainingandsupport,cannotbeconsideredeffective intheclassroom,comprises 63% of teachers for science, 53% for English, and 17% for maths. Teachers, headteachers, and LGEA officials also demonstrate fairly limited acceptance that lack of subjectknowledgeisanissue–fewrespondentsshowunderstandingofthechallengesencounteredbyteachersregardingtheirsubjectknowledgeandpossiblewaystoaddressthis.The average teacher is also unable to correct errors in pupils’ work and to provide relevantfeedback, or to identify the learning needs of individual pupils and monitor their academicprogressovertime.Teachersclaimtoknowwhetherpupilsarelearningornotfromclasstestsorclasswork.However,itdoesnotseemthatregularassessmentofpupilsiscommonpractice–onlyoneoftheninecasestudyschoolsconductscontinuousformativeassessment.Teachersdospenda large amountof time correctingpupils’ notebooksbut usually spend very little time checkingeachnotebookbeforetickingitandhandingitback.Evenwhenteachershavemarkedtheworkascorrect,itappearsthatpupilsdonotknowhowtowritelettersorthemeaningofwhattheyhavewritten.Thissuggeststhatteachersareeitherunabletopayproperattentiontothepupils’work,forexamplebecauseoflargeclasssizes,orlacktheknowledgetomarktheworkappropriately,orboth.

7.2.2 Teachers’pedagogicalskills

Theaverageteacherinvolvespupilsforaboutaquarterofthetotallessontimeinpupil-centredteachingpracticesthatcharacteriseeffectiveteachersandclassroompractice.Simplywritingorreadingfromtheblackboard,consideredtobeaneutralteachingpractice,takesup40%oftotallessontime.

Teachers rely heavily on textbooks and curriculum guides, especially in the absence offoundationalsubjectknowledge,ratherthanbeingabletodevisetheirownlessonplanstoreachaspecified learning goal. They demonstrate limited ability to adapt the textbookmaterial to theconditionsandconstraintstheyfaceintheclassroom.Inparticular,teachersdemonstratelimitedideas about how pupils learn and about how teachers can respond to different learning levelswithin the same classroom to ensure all pupils achieve at least a minimum level. In general,teachers’priorpedagogical trainingdoesnotseemtohaveadequatelyequippedthemfortherealities they face in the classroom, including large class sizes, multi-grade teaching, limitedresources,andlimitedabilityofparentstosupporttheirchildren.

There is substantial loss of instructional time due to shorter lesson length,which necessarilylimitsthepotentialforin-classlearningtotakeplace.Classroomobservationsfoundthat45%oflessonsweremorethanfiveminutesshorterthanastandard35-minutelesson.


©EDOREN 82

7.2.3 Teachers’motivationandattendance

Thisbaseline report findsmixedevidenceof teachersbeing intrinsicallymotivated to improvetheirteaching,totakepartinlearningopportunitiesandtoapplynewknowledgewhentheygetit. Teachers gave varieddescriptionsof their attitudes towards the teachingprofession, rangingfromthosewhodescribedteachingasaninherentlynobleprofessionadvancingsociety,tothosewhoacceptedteachingjobsbecausetheywereunabletofindanythingelse.

There is strong evidence that teachers feel demotivated due to a range of extrinsic factors.Amongthesefactors,teacherspredominantlytalkedaboutlowandlatesalarypayments,thelackof learning resources for pupils and poor infrastructure, over-crowded classrooms, irregularattendance of children, ‘untalented’ pupils, lack of promotion or perceived unfairness inpromotions,andarbitrarytransfers.

Theaveragedailyteacherabsenteeismoverafive-dayperiod,accordingtotheschools’records,is14%.Themostcommonreasonscitedbyteachersfortheirabsenceare:ownorfamily illness(58%);collectingsalaryandfamilyreasons(20%);andsocial/religiousobligations(10%).However,qualitative researchers found that teacherscombine their teachingworkwith farmingor smallbusinessesoutsidetheschool.Someteachersareundergoingfurthertrainingoreducationatthesametimeasteaching,takingthemawayfromtheirregularjobs.Inmostschools,researchersalsosawteachersarrivinglate.Classroomabsenteeismisacommonobservationinthequalitativesurvey.Teachers’specialisedtraining intheirNCEoftendoesnotmatchthesubjectthat isneededinaschoolwhentheyarepostedthere,andshortagesofteacherscanariseinspecificsubjectareasevenwhenthereisnooverallshortageofteachers.

7.2.4 Contextualfactorsthataffectthequalityofteaching

About 88% of head teachers reported that their schoolswere in need ofmajor repairs. Onlyabout11%ofschoolshadanelectricitysupply.Inteacherinterviews,70%ofteachersnotedthattheirschool’sbuildingwasina‘poorcondition’,and33%saidtheyhaveinadequatematerialstodotheirjobproperly.Inadequateclassroomresourcesandpoorschoolinfrastructurewererankedamong the worst constraints faced by teachers.Urban or semi-urban schools were generallyfoundtobemuchbetterequippedthanruralschoolsintermsofphysicalinfrastructure.

Head teachers frequently noted that their request for additional infrastructure, repairs, andresources from the LGEA/SUBEB go unanswered. Political connections were highlighted as animportantdeterminantofhowwellequippedschoolswere.

Textbooksarealsonotroutinelydistributedamongpupils.Thisissometimesbecausethereareinsufficienttextbooks.Intherarecaseswhereschoolresourcesaresufficient,thereisevidencethat these are not always managed and allocated efficiently (for example, unused teachingresourcesandtextbooksinaheadteacher’sofficekeptunderlockandkey).

7.2.5 ImplicationsofteachereffectivenessbaselineresultsforTDP

This baseline survey investigatedwhether teachers aremotivated to attend school and lessonsregularly; whether they have sufficient subject knowledge and pedagogical skills; and whethertheyhaveaccesstosufficientinfrastructureandmaterials.


©EDOREN 83

Foravastproportionofschoolsstudiedaspartofthisbaselinesurvey,noneoftheassumptionsholdupfullywithstrongevidence.This isnottosaythattheydonotholdupatallbutthat,atleastinsomecases,theevidenceindicatesthattheyarebinding.Forexample,itmaybethatonlyaminorityofteachersfrequentlymissschool,butabsenteeismratesarehighenoughto leadtodoubts about the impact of any teacher development activity on pupils’ learning outcomes.Similarly,notallschoolshaveverylargeclasses,butasubstantialproportiondo.

Certain dysfunctionalities are pervasive across schools, irrespective ofwhether their teachershavehigh,typicalor lowlevelsofsubjectknowledge. Infrastructureandresourcesaretypicallybetterinthehigh-performingschools.However,manyofthesamedysfunctionalitiesareobservedinbothhigh-andlow-performingschools.Forexample,almostacrosstheboard,andinbothruralandurbanschools,pupils’absenteeismfromschoolandteachers’absenteeismfromschoolandclassroomwasobserved.Thissuggeststhatthechallenges identified inthequalitativestudyarenotlimitedtotheworst-performingschoolsbutaremorepervasive.

Someofthe issuesraisedhereare likelytoremainbeyondthescopeofaprogrammesuchasTDP. Infrastructureproblemsand large class sizes require actionat the federal, state, and localgovernmentlevel,possiblycombinedwithactionstobuildthecapacityofSBMCsandparentstoholdschoolstoaccount.It isalsodoubtfulwhetherTDPcanproducearadicalchangeinteachersubjectknowledge,as thisknowledge is the resultof teachers’entireeducation,andcannotbereversed with a relatively short training intervention. Yet without addressing these broadersystemicissues,thereisunlikelytobeameasurableimprovementinpupils’learningoutcomes.

Action for TDP could, however, focus on how the teacher trainingmaterials and activities itundertakescantakeintoaccounttherealityofthedifficultcontextsinwhichteacherswork,andpitch their content to the very limited pedagogical skills and subject knowledge that mostteacherscurrentlypossess.

Itisworthinvestigatingwhetherthereisspacetopoliticallypushforthegovernment’slanguagepolicy–ofteachinginHausainGrades–-3andinEnglishthereafter–tobeactuallyenforced,which would mean producing and using Hausa textbooks and assessments for early gradelearning. This would still be a radical change as teachers may not be familiar with textbookswritten in Hausa, but it would at least mean engaging with the reality of teachers’ currentunderstandingofEnglishandwiththefactthat99%pupilsinthissurveyreportedHausaastheirmain home language. The switch to English in Grade 4would remain difficult, but arguably nomoresothancurrently.

7.3 SLM

EffectiveSLMiscrucialforthesuccessofTDP’sin-servicetrainingactivitiesfortworeasons.OneofthekeyoutputsoftheTDP’sin-serviceteachertrainingactivitiesisenhancedSLM.Inaddition,teacherscannotbeassumedtobemotivatedtoadoptnewteachingpracticeswithoutappropriateleadershipandmanagementfromheadteachers.

The baseline survey reveals strong evidence that, in most cases, challenges faced by headteacherssignificantlyweakenstheassumptionsuponwhichTDP’sTOCrests.

Headteachersarenotextrinsicallyincentivisedandinmanycasesnotintrinsicallymotivatedtoimprove SLM or encourage teachers’ adoption of the teaching practices that TDP seeks topromote.Thereisariskthatheadteacherswillnotexercisetheirabilitytoinfluencesuchchangewhere they encounter resistance to new practices and where they risk disrupting otherwise


©EDOREN 84

positive relationships within the school. This would mean that newly acquired subject andpedagogicalknowledgeisnot,orisonlysuperficially,appliedbyteachersinaclassroomcontext.

Headteachersaregenerallyunabletoeffectivelyaddressthewidespreadproblemofteachers’absenteeismfromschoolandclassroom.Thissuggeststhat,evenifteachersattendTDPtrainingandimprovetheirsubjectandpedagogicalknowledge,thereisasignificantriskthatteacherswillnotapplythisknowledge intheclassroomcontextbecausetheyspend largepartsoftheschooldayoutsideofclasses.

Headteachersremainat leastpartlyableto influenceteachingtechniquesandcorrectsubjectknowledgemistakesthroughlessonobservationsandfeedback.Thissuggeststhatheadteacherswill be able tomotivate andpersuade teachers that already attend lessons andhave improvedsubject and pedagogical knowledge to adopt the positive teaching practices that TDP seeks topromote.

Many of the factors that shape this context – such as salary payment systems, and theresponsiveness of LGEAs/SUBEBs to head teachers’ requests – are beyond the scope of TDP.However,therearepotentiallessonsthatcanbelearnedfromthebaselineresearchtoinformfutureTDPactivities:

• Lowteacherattendanceisawidespreadproblembutheadteachersaloneinmostcasesdonothavethemeanstosolveit.TDPcouldperhapsrequestthatgoodclassroomattendancebe an additional criterion for selecting future beneficiaries of TDP training, therebymaximisingthechancesthatnewknowledgeisappliedinaclassroomcontext.

• Head teachers are rarely held to account and therefore they are not incentivised toimproveperformance.ThisfactmakesTDPmorereliantonheadteachers’ownmotivation,which is also lacking in many cases. TDP could consider including activities designedspecificallytomotivateheadteachers.

• Headteachersaresometimesunclearoftheirrightsandresponsibilitiesinrelationtootheractors in the education system, particularly LGEAs/SUBEBs. TDP’s planned head teachertrainingactivitiescould increasetheirawarenessofthese,andequipthemwiththeskillsnecessarytonavigatesuchaconstrainingsystem.

• AlargenumberofTFsareLGEAstaff.GiventhatmanyoftheconstraintsoneffectiveSLMarecausedbydynamicsatthesystemlevel,TDPmaywishtoconsiderhowtheseTFscouldfoster an honest dialogue between influential communitymembers, SUBEBs, LGEAs andschoolstocreateacommonunderstandingofthedriversofschoolperformance.

7.4 TDP’simplementationmodelforin-servicetraining

Qualityof implementationand robustdeliveryofoutputs are vital toTDP’s success.The finalareaofTDP’sTOCistheprogramme’simplementationmodelfordeliveringin-servicetraining.Inparticular,thisreportdiscussedclustermeetingsandpeer-to-peerlearning,trainingmaterialsandTFs,withtheaimofascertainingwhetherassumptionsrelatedtoTDP’soperationaleffectiveness–whichunderpinthelogicoftheTDPTOC–holdinpractice.

Anumberofchallengesrelatingtotheprogramme’simplementationweakenitsTOCandreduceitschancesofmeetingoutcome-andimpact-levelgoals.Weakparticipationinclustermeetings,


©EDOREN 85

duetofactors likelanguagecomplexitiesandbarrierssuchasgender, ispersistent.Thereisalsolimitedevidenceofpeer-to-peerlearningmaterialisinginpracticeduetolowteachermotivationandhighabsenteeism,leadingtolimitedtimedevotedtopeerinteraction.TDP’strainingmaterialstry to address the constraints faced by classroom teachers (e.g. large class sizes) and proposetechniquesforeffectiveteaching inthemidstoftheseobstacles,howeverobservations intheseearly stages of implementation reveal limited application of these techniques in classrooms, orsuperficial adoption at best. Confusion among teachers and trainers between concepts andterminologyusedbyvariousteachertrainingprogrammesiscommon.Whilemechanismsexistforroutingfeedbackonthetrainingfromteacherstomaterialdevelopersandprogrammemanagers,itisnotclearwhetherthisfeedbackthenpromptsactionandrectification.Finally,TFshavefairlystrongbackgrounds inclassroomteachingandschooladministrationbutencountersomeof thesamepedagogicallimitationsintheirskillsasarefacedbyteachersthemselves,andtheyalsofacean immenseworkload in organising anddelivering clustermeetings (alongsidedischarging theirdayjobs,mostlyintheLGEAs).It is worth emphasising that teachers in northern Nigeria have incredibly tough professionalduties todischarge– they are expected to teach childrenwhere classrooms sometimesdonotexist, textbooks are often scarce, and class sizes are large. Teachers’ intrinsicmotivation to bebetter educators is often eroded by broad-based poverty and low salaries, and long delays inreceivingthem.Thisleadstoperdiemsfromclustermeetingsthenbecomingakeyincentiveforparticipation in training. A number of these challenges appear to be well beyond theprogramme’sdirectambitofcontrol.Forexample,issuesoflowsalariesandlatepaymentrelateto federal- and state-level governance and public financial management issues and it is notstraightforward for TDP to influence these issues, despite these challenges having an immenseimpactonprogrammesuccess.Itshouldalsobeborneinmindthatthisprocessassessmentwasundertakenintheearlystagesof implementation and it is expected that some of the operational teething issues will bealleviatedastheprogrammematures.However, therearesomechallengeswhichappearmorewithintheprogramme’sabilitytoaddressintheshortormediumterm.

• To begin with, teachers and their trainers would find it immensely helpful to receivematerialsinadequatequantitiesandontime,toavoidthemhavingtopayout-of-pockettomakephotocopies,buystationery,andsoon.

• Thedeliverymodelwillbeenhancedconsiderablyiffeedbackmechanismsfromteachersregardingthetrainingaremoreeffectivelyroutedtothosewhocanaddressthisfeedback,andiffeedbackisfolloweduponwithactionsanddecisions.

• The programme might further want to consider what value the idea of training someteachersinaschoolwhileleavingoutothersisadding,giventhediscussiononlimitedpeerexchangesamongselectedteachersandthepossibilityofhostilitybetweenTDPandnon-TDPteacherswithinaschool.

• Finally, the lack of a direct emphasis on subject knowledge in the TDP’s trainingcurriculum is a concern, and something for TDP andDFID to address. Ineffectiveness oftraining is often driven by limitations imposed by teachers’ own absorptive capacity fornew materials and pedagogical techniques. In a context where teachers’ own subjectknowledgeisgrosslyinsufficient,moretrainingonnewerpedagogicaltechniqueswillonly


©EDOREN 86

build on a weak foundation of subject knowledge and will limit the absorption andadoptionofnewpedagogicalcontent,thusweakeningchancesforprogrammesuccess.

7.5 Follow-upquantitativeandqualitativesurveys

The findingsof thisbaselinesurveycall forurgentattention to the fact thatpupils in treatmentand control schools are on a flat learning trajectory and the effectiveness of teaching in theseschoolsisofseriousconcern,andthatmost–ifnotall–assumptionsunderlyingthelogicofTDP’sTOCdonotholdinpractice.TheseresultsnaturallyhaveseriousconsequencesforTDP’sabilitytoachieveitsdesiredimpactofimprovedpupillearning.

Theevaluationframeworkhadoriginallyproposedfollow-upquantitativeandqualitativemidlineand endline surveys in 2016 and 2018 respectively. The timing of the midline was planned tocoincidewiththeexpansionofTDP’sin-servicetrainingactivitiestoKaduna,NigerstateandKano,andthustoinformthedesignofthescale-uptothesestatesbasedonlessonslearnedfromPhase1states.However,withthereleaseofthebaselinesurveyresultsshowingworryinglylowlevelsofpupil learning and teacher effectiveness, discussionswere held inOctober 2015 betweenDFID,TDP,EDORENandtheprogramme’sannualreviewersregardingwhetherthemidlinewouldhaveallowedadequatetimetodetectanyperceptiblechangestokeyindicatorsofinterest.Itwasfeltthat the quantitative midline survey would not add much value in terms of informing theprogramme’s future implementation, and especially its scale-up, and instead a series ofimplementation-focused studies covering operational and process questions might be morevaluable.

After discussions and consultation, the TDP evaluation’s steering committee agreed on thefollowingthreequestionsfora‘midline’processevaluation:

• Giventhelowlevelsofsubjectknowledgeandpedagogicalskillsamongteachers,asestablishedbythebaselinesurvey,howcouldTDPmaketheTDTs,teachertrainers,clustermeetingsandschoolsupportvisitsmoreeffective?

• TowhatextentdoTDPtrainedteachersunderstandthecontentoftheprintandaudio-visualtrainingmaterialstheyhavebeengiven?Towhatextentdotheyusethesematerialsbothinsideandoutsidetheclassroom?Howcantheprogrammemakethematerialsmoreusefulandmoreused–intermsofcontent,languageandusage?

• HowcouldTDPimproveschoolleadership(especiallyvis-à-vismanagementandgovernance)inTDPschools–intermsofmoreeffectiveheadteachers,educationsecretaries,inspectors,qualityassuranceofficers,SBMCsandparents?

ThesestudieswouldbeconductedbyEDORENandwouldbedueintimefortheplannedscale-upin late2016.Theendlinemixed-methodssurveyswillbeconductedasplanned in June2018, toallow for measurement of any programme impact on teacher effectiveness and pupil learninglevels.ThebaselinepanelofpupilswhowereatthebeginningofGrade3atOctober2014willbeonthevergeoftakingtheirprimaryschoolleavingexamsattheendofGrade6inJune2018.


©EDOREN 87

Bibliography

BangladeshDirectorateofPrimaryEducation.2013.“NationalStudentAssessmentforGrades3and5.”BangladeshDirectorateofPrimaryEducation,MinistryofPrimaryandMassEducation.

Bayer,Sonja,EckhardKlieme,DavidKaplan,andSvenjaVieluf.2012.TeachingPracticesandPedagogicalInnovations.TALIS.OECDPublishing.http://www.oecd-ilibrary.org/education/teaching-practices-and-pedagogical-innovations_9789264123540-en.

Beavis,Adrian,andRachelOuthred.2014.“Pre-LiteracyLiteratureReviewforEDOREN,TDPandGEPEvaluations.”OPM.

Bennell,Paul,andKwameAkyeampong.2007.“TeacherMotivationinSub-SaharanAfricaandSouthAsia.”DFID:EducationalPapers.DFID.

Bennell,Paul,SarahAnyawu,andMulkaDodo.2014.“TheSupplyofandDemandforPrimaryandJuniorSecondarySchoolTeachersinKatsinaState,2014–25.”EDOREN,Abuja.

Cameron,Stuart.2015a.“ESSPINCompositeSurvey2:OverallReport.”OPM,Oxford,UK.———.2015b.“TeacherMotivation:MethodsandResultsfromtheTDPin-ServiceBaseline

Survey.”EDOREN(unpublishednote).Chaudhury,Nazmul,JeffreyHammer,MichaelKremer,KarthikMuralidharan,andF.HalseyRogers.

2006.“MissinginAction:TeacherandHealthWorkerAbsenceinDevelopingCountries.”TheJournalofEconomicPerspectives20(1):91–116.

Das,J.,S.Dercon,J.Habyarimana,andP.Krishnan.2004.“TeacherShocksandStudentLearning:EvidencefromZambia.”CambridgeWorkingPapersinEconomics0514.FacultyofEconomics,UniversityofCambridge.https://ideas.repec.org/p/cam/camdae/0514.html.

De,Sourovi,andStuartCameron.2015.“ESSPINCompositeSurveyII:GenderandInclusionReport.”OPM.

De,Sourovi,StuartCameron,RobertMorris,andSaltanatRasulova.2015.“EDORENTeacherDevelopmentProgramme(TDP)ImpactEvaluationofIn-ServiceTeacherTrainingQualitativeResearch:ConceptNote.”EDOREN(unpublishednote).

De,Sourovi,andGunillaPettersson.2015a.“TDPIn-ServiceTrainingImpactEvaluationAnalysisPlan.”EDOREN.

———.2015b.“TDPImpactEvaluation:QuantitativeBaselineSurveyReport.”EDOREN.De,Sourovi,GunillaPettersson,Peter-SamHill,andRobertMorris.2015a.“TDPKatsinaReport.”

OPM.———.2015b.“TDPZamfaraReport.”OPM.De,Sourovi,GunillaPettersson,andRobertMorris.2015.“TDPJigawaReport.”OPM.DFID.2013.“EducationPositionPaper:ImprovingLearning,ExpandingOpportunities.”DFID.

https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/225715/Education_Position_Paper_July_2013.pdf.

DFIDNigeria.2012.“TeacherDevelopmentProgramme(TDP)BusinessCase.”DFID,Nigeria.Donovan,Jenny,MelissaLennon,GaylO’Connor,andNoniMorrissey.2008.“NationalAssessment

Program–ScienceLiteracyYear6Report,2006.”MinisterialCouncilonEducation,Employment,TrainingandYouthAffairs(NJ1).http://eric.ed.gov/?id=ED534659.

EDOREN.2014.“TDPEvaluationFrameworkandPlan.”EDOREN.Oxford,UK.ESSPIN.2013.“OverallFindingsandTechnicalReportofESSPINCompositeSurvey1(2012).”

ESSPIN060.ESSPIN,UKAID.http://www.camb-ed.com/Portals/0/Documents/ESSPIN-Composite-Survey-Technical-Report.pdf.

———.n.d.“ESSPINSchoolBasedManagementCommittee(SBMC)Booklet.”https://www.esspin.org/.../SBMC%20Booklet%20English%20North.


©EDOREN 88

FederalRepublicofNigeria.2004.“NationalPolicyonEducation2004.”NERDCPress.Goldstein,Harvey.2015.“RaschMeasurement:AResponsetoPayanides,RobinsonandTymms.”

BritishEducationalResearchJournal41(1):176–79.doi:10.1002/berj.3170.Guerrero,GabrielaEPPI-Centre.2012.WhatWorkstoImproveTeacherAttendanceinDeveloping

Countries?:ASystematicReview.London:EPPI-Centre,SocialScienceResearchUnit,InstituteofEducation,UniversityofLondon.

Hanushek,EricA.,andStevenG.Rivkin.2006.“Chapter18TeacherQuality.”InHandbookoftheEconomicsofEducation,2:1051–78.Elsevier.http://linkinghub.elsevier.com/retrieve/pii/S1574069206020186.

Hopkins,David,AlmaHarris,andDavidJackson.1997.“UnderstandingtheSchool’sCapacityforDevelopment:GrowthStatesandStrategies.”SchoolLeadership&Management17(3):401–12.doi:10.1080/13632439769944.

Humphreys,Sara,andLeeCrawfurd.2014.“ReviewoftheLiteratureonBasicEducationinNigeria.”EDOREN.

Johnson,D.2008.“AnAssessmentoftheDevelopmentNeedsofTeachersinNigeria–KwaraStateCaseStudy.”

Johnson,David.2010.“AnAssessmentoftheProfessionalWorkingKnowledgeofTeachersinNigeria:ImplicationsforTeacherDevelopment,PolicyandImplementationJigawaState(JG303).”ESSPIN/UKAID.

Johnson,David,andJennyHsieh.2014.“AnAssessmentoftheDevelopmentNeedsofTeachersinNigeria-KatsinaStateCaseStudy.”

Lemke,JayL,andNoraHSabelli.2008.“ComplexSystemsandEducationalChange:TowardsaNewResearchAgenda.”EducationalPhilosophyandTheory40(1):118–29.doi:10.1111/j.1469-5812.2007.00401.x.

Marshak,Anastasia,DavidK.Evans,andShwetlenaSabarwal.2014.“ThePermanentInputHypothesis:TheCaseofTextbooksand(No)StudentLearninginSierraLeone.”WPS7021.TheWorldBank.http://documents.worldbank.org/curated/en/2014/09/20173246/permanent-input-hypothesis-case-textbooks-no-student-learning-sierra-leone.

McCormick,Robert.2014.“TDPResultsandEvidenceStrategy.”CambridgeEducation.Moore,Mark.2015.“CreatingEfficient,Effective,andJustEducationalSystemsthroughMulti-

SectorStrategiesofReform.”RISEWorkingPaper15/004.http://www.riseprogramme.org/content/rise-working-paper-15004-creating-efficient-effective-and-just-educational-systems-through.

NevesdeJesus,Saul,andWillyLens.2005.“AnIntegratedModelfortheStudyofTeacherMotivation.”AppliedPsychology54(1):119–34.doi:10.1111/j.1464-0597.2005.00199.x.

OECD.2014.TALIS2013Results.TALIS.OECDPublishing.http://www.keepeek.com/Digital-Asset-Management/oecd/education/talis-2013-results_9789264196261-en#page165.

OPM.2015.“EQUIP-TanzaniaImpactEvaluation:FinalBaselineTechnicalReport,VolumeI;ResultsandDiscussion.”OPM.

Outhred,Rachel.2015.“ConceptNoteonLearningOutcomeMeasurementacrossEDORENEvaluations.”EDOREN(unpublishednote).

Panayides,Panayiotis,ColinRobinson,andPeterTymms.2010.“TheAssessmentRevolutionThatHasPassedEnglandby:RaschMeasurement.”BritishEducationalResearchJournal36(4):611–26.doi:10.1080/01411920903018182.

Pettersson,Gunilla,andGeorginaRawle.2015.“EQUIPTanzaniaImpactEvaluationBriefingNote1:Teachers’Knowledge,BehaviourandSupportinSomeoftheMostDisadvantagedDistrictsinTanzania.”OPM.


©EDOREN 89

Robinson,V.M.J.,C.A.Lloyd,andK.J.Rowe.2008.“TheImpactofLeadershiponStudentOutcomes:AnAnalysisoftheDifferentialEffectsofLeadershipTypes.”EducationalAdministrationQuarterly44(5):635–74.doi:10.1177/0013161X08321509.

Siraj,Iram,BrendaTaggart,EdwardMelhuish,PamSammons,andKathySylva.2014.“ExploringEffectivePedagogyinPrimarySchools:EvidencefromResearch.”https://research.pearson.com/content/plc/prkc/uk/open-ideas/en/articles/explore-eppse/_jcr_content/par/articledownloadcompo/file.res/Exploring%20Effective%20Pedagogy%20in%20Primary%20Schools.pdf.

Suryadarma,Daniel,AsepSuryahadi,SudarnoSumarto,andF.HalseyRogers.2006.“ImprovingStudentPerformanceinPublicPrimarySchoolsinDevelopingCountries:EvidencefromIndonesia.”EducationEconomics14(4):401–29.

TDP.2014.“TDPIn-ServiceTeacherTrainingStrategy.”TDP.UNESCO.2004.EducationforAll:TheQualityImperative.Paris:UnitedNationsEducational,

ScientificandCulturalOrganization.———.2014.TeachingandLearning:AchievingQualityforAll.

http://unesco.nl/sites/default/files/dossier/gmr_2013-4.pdf?download=1.Watts,Michael,andTerryAllsop,.2015.“EDORENTeacherManagementReport(Kadunaand

Katsina).”EDOREN.Westbrook,J.2013.“Pedagogy,Curriculum,TeachingPracticesandTeacherEducationin

DevelopingCountries.”DFID.https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/305154/Pedagogy-curriculum-teaching-practices-education.pdf.

Wobmann,Ludger.2000.“InstitutionsoftheEducationSystemandStudentPerformance:TheInternationalEvidence.”InSept.2000ConferenceofISNIE,AvailablefromHttp://www.ISNIE2000.Org.

WorldBank.2012a.“KenyaEducation:ServiceDeliveryIndicators.”http://www.sdindicators.org/kenya-education/.

———.2012b.“TanzaniaEducation:ServiceDeliveryIndicators.”http://www.sdindicators.org/tanzania-education/.

Yoshikawa,Hirokazu,KathleenMcCartney,RobertG.Myers,KristenL.Bub,JulietaLugo-Gil,MariaA.Ramos,FeliciaKnaul,andUNICEFInnocentiResearchCentre.2007.“EarlyChildhoodEducationinMexico:Expansion,QualityImprovementandCurricularReform.”InnocentiWorkingPaperinwopa07/41.UNICEFInnocentiResearchCentre.http://ideas.repec.org/p/ucf/inwopa/inwopa07-41.html.

Thispagehasbeenintentionallyleftblank.

House 2, 16 Mafemi Crescent,

Utako, Abuja, Nigeria

Phone: +243 (0)810 727 8718 | +234 (0)817 667 8243

Website: www.edoren.org | Email: [email protected]

This evaluation is being carried out by EDOREN (Education Data, Research and Evaluation in Nigeria) under its Teacher Development Programme (TDP)evaluation and support activities (Workstream 2) on behalf of the Department for International Development (DFID) funded TDP. The workstream leader is Sourovi De ([email protected]), and the team leader for this evaluation is Ian MacAuslan. The client reference number for the project is 8022. The contact point for the client is Esohe Eigbike at DFID Nigeria ([email protected]) and Bukola Oyinloye ([email protected]) at the TDP.

EDOREN is a consortium of leading organisations in international development and education managed by Oxford Policy Management (OPM) and includes the Institute of Development Studies (IDS) at the University of Sussex. EDOREN is supported by UK Aid. Also visit our website, www.nigeria-education.org, to subscribe to reports, articles and other materials.

DISCLAIMER

EDOREN is a consortium of Oxford Policy Management (OPM) and the Institute of Development Studies (IDS) at the University of Sussex, and is supported by UK Aid. EDOREN cannot be held responsible for errors or any consequences arising from the use of information contained in this report. Any views and opinions expressed do not necessarily reflect those of OPM, IDS and EDOREN or any other contributing organisation.

ABOUT THE PROJECT

Cover Photo Credit: Catholic University of America, International Development Program