evaluating and rewarding the quality of teachers

Evaluating and Rewarding the Quality of Teachers

InternatIonal PractIces

ORGANISATION FOR ECONOMIC CO-OPERATIONAND DEVELOPMENT

The OECD is a unique forum where the governments of 30 democracies work together to address the economic, social and environmental challenges of globalisation. The OECD is also at the forefront of efforts to understand and to help governments respond to new developments and concerns, such as corporate governance, the information economy and the challenges of an ageing population. The Organisation provides a setting where governments can compare policy experiences, seek answers to common problems, identify good practice and work to co-ordinate domestic and international policies.

The OECD member countries are: Australia, Austria, Belgium, Canada, the Czech Republic, Denmark, Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Japan, Korea, Luxembourg, Mexico, the Netherlands, New Zealand, Norway, Poland, Portugal, the Slovak Republic, Spain, Sweden, Switzerland, Turkey, the United Kingdom and the United States. The Commission of the European Communities takes part in the work of the OECD.

OECD Publishing disseminates widely the results of the Organisation’s statistics gathering and research on economic, social and environmental issues, as well as the conventions, guidelines and standards agreed by its members.

This publication is a product of the co-operation agreement established between the Government of Mexico and the OECD, which aims to improve the quality of education in Mexico.

ISBN 978-92-64-06198-9 (print)

ISBN 978-92-64-03435-8 (PDF)

Photo credits: ©UNESCO/José Gabriel Ruiz Lembo.

Corrigenda to OECD publications may be found on line at: www.oecd.org/publishing/corrigenda.

© OECD 2009

You can copy, download or print OECD content for your own use, and you can include excerpts from OECD publications, databases and multimedia products in your own documents, presentations, blogs, websites and teaching materials, provided that suitable acknowledgment of OECD as source and copyright owner is given. All requests for public or commercial use and translation rights should be submitted to [email protected]. Requests for permission to photocopy portions of this material for public or commercial use shall be addressed directly to the Copyright Clearance Center (CCC) at [email protected] or the Centre français d’exploitation du droit de copie (CFC) at [email protected].

This work is published on the responsibility of the Secretary-General of the OECD. The opinions expressed and arguments employed herein do not necessarily reflect the official views of the Organisation or of the governments of its member countries.

Design Components of Incentive Pay Programmes

in the Education Sector

65

Evaluating and Rewarding the Quality of Teachers: International Practices © OECD 2009

Chapter 3

Introduction .................................................................................................................................... 66

Incentive structure ..................................................................................................................... 67•Rank-ordertournaments ................................................................................................ 67•Fixedperformancecontract ......................................................................................... 67

Unit of accountability ............................................................................................................... 71•Individualunitofaccountability ............................................................................. 71•Groupunitofaccountability ...................................................................................... 72•Hybridmodelsofaccountability ............................................................................. 74

performance measures ............................................................................................................ 75•Educationalinputs ............................................................................................................. 75•Educationaloutputs .......................................................................................................... 79•Multiplemeasures–linkingeducationalinputsandoutputs ............... 82

performance standards and thresholds ...................................................................... 83•Linearmodels ........................................................................................................................ 83•Stepfunctions ........................................................................................................................ 83•Limitedlinearmodels ...................................................................................................... 84•Thresholdlevels ................................................................................................................... 85

Size and distribution of bonus awards ........................................................................ 85•Sizeofbonus ......................................................................................................................... 85•Distributionofaward ....................................................................................................... 87•Non-monetaryincentives .............................................................................................. 88

payout frequency ......................................................................................................................... 89

Cultural considerations ........................................................................................................... 90

Conclusion ....................................................................................................................................... 91

Notes .................................................................................................................................................... 92

References ........................................................................................................................................ 94

chapter 3 Design Components of inCentive pay programmes in the eDuCation seCtor

Evaluating and Rewarding the Quality of Teachers: International Practices

66

© OECD 2009

IntroductIon

Educationreformisincreasinglyfocusedonhumancapitalformationbecausehigh-qualityteachersareseenas the most direct and effective pathways to improving student achievement. Based on the argument thatprevailing compensation practices provide weak incentives for teachers, and that inefficiencies arise fromrigiditiesincurrentcompensationpolicies,severalnationalsystemsofpubliceducationhaveexploredteachercompensationreforms(PodgurskyandSpringer,2007).1Proponentsarguethatfinancialincentivescanmotivateteacherstoachievehigherlevelsofperformance,enticemoreeffectiveteacherstojoinorremainintheteachingprofession,andalignteacherbehavioursandinterestswithinstitutionalgoals.

Inspiteoftheintuitiveappealincentivepayhasforsomestakeholders,asturdyandinfluentialbaseofindividualsandorganisations fundamentallyoppose itsuse ineducation (Eberts,2007;Goldhaber,2009;KingdonandTeal,2008).Opponentscontendthatsuchpayrendersschoolslesseffectivebycrowdingoutintrinsicrewards;that is, teacherswill lose interest in theprofessionas theyare increasingly rewarded forstudentoutcomes.Criticsfurtherarguethattheeducationsystemlacksappropriatemeasuresforevaluatingteacherperformance,thatrewardprogramswilldecreaseteachercollaboration,andthatthecurrentbodyofevidenceontheimpactofpay-for-performanceprogrammesisinconclusive.

Recentexperimentalandquasi-experimentalevidencepaintsamixedpictureoftheimpactofteacherincentive-payprogrammes.MuralidharanandSundararaman(2008)andLavy(2002,2007)foundthatteacherincentiveprograms in India and Israel, respectively, improved student outcomes and promoted positive changes inteacherbehaviourand/orclassroompedagogy.Glewwe,IliasandKremer(2008)similarlyreportedthatstudentsinstructedbyteacherseligibletoreceiveabonusawardinKenyademonstratedbetterscoresonhigh-stakestests;however,nodiscernibleimpactwasfoundonlow-stakesteststakenbytreatmentgroupstudentsoronthesamestudentswhentheytookhigh-stakestestsduringthepost-interventionschoolyear.

Thedesigncomponentsofincentiveprogrammesintheeducationsectorhavereceivedlittleattentionintheliterature,particularlywithregardtotherelativeadvantagesandlimitationsofdifferentoptionsformeasuringandrewardingteachers.Sincethedesignofanincentiveprogrammecanleadtodramaticdifferencesinitsimpact on students, teachers, and administrators, it is important to learn more about how specific designelementsaffectteachers’responses.

ThefollowingdiscussionislargelyinformedbythetheoreticalliteratureonincentivesystemsaswellasempiricalevidencefromevaluationsofpastandpresentprogrammesandpoliciesintheUnited States(U.S.)andothercountries.Wedrawonexperiencesfrombothwithinandoutsidetheeducationsectortoassessspecificdesigncomponentsofincentive-payprogrammesandpoliciesandtheirpotentialimpactonstudentoutcomes,teacherattitudesandbehaviour,andinstitutionaldynamics.Specifically,weassess:

• incentivestructure;

• unitofaccountability;

• performancemeasures;

• performancestandardsandthresholds;

• sizeanddistributionofbonusawards;

• payoutfrequency;and

• culturalconsiderations.

Thischapterpresentsabriefdiscussionoftherelationshipbetweenincentive-payprogrammecharacteristicsandculturalcontextbeforeofferingconcludingremarks.

67Design Components of inCentive pay programmes in the eDuCation seCtor chapter 3


IncentIve structure

Incentive structure refers to the scheme or mechanism that guides the allocation of awards in a pay-for-performancesystem.Insomecasesonlyalimitednumberofemployeescanearnanaward,whileinothersanyemployeewhomeetsapredeterminedperformancestandardwillreceiveanaward.Thetwomainformsofincentivestructurearerank-ordertournamentsandfixedperformancecontracts,eachofwhich,undercertainconditions, couldbeanoptimal labourcontract yielding thefirstbestoutcome.Both forms represent verydifferentcompensationschemeswithdistinctadvantagesandlimitations.

rank-order tournaments

Rank-ordertournamentsareincentivestructuresthatlimitperformance-basedrewardstoafixedpercentageoftherelevantworkgroup.Thekeyfeaturethatdistinguishestournamentsfromotherincentive-paystructuresis that compensation depends on relative performance rather than absolute performance. For example, anindividuallevelpay-for-performanceexperimentimplementedinIsraelandevaluatedbyLavy(2004)comparedtheperformanceofteachersinsimilargradesandsubjectsandthenawardedindividualbonusestotheteacherswithpositivegainsscoringinthetopperformancequartiles.

Aprimarystrengthoftournamentincentivestructuresislowerinformationcosts(Baker,JensenandMurphy,1988).Forexample,ifaschooldistrictannouncedthatthetoptenteachersinaschoolcouldwinbonusesofUSD5000,with20schoolsinthedistrictparticipatingintheprogramme,andeachschoolhad40teachers,theschooldistrictcouldanticipatethatthecostofthebonuseswouldbeUSD1millionayear(10winningteachersx20schoolsxUSD5000perwinning teacher).However, if the incentivestructure isdesignedso thatall teacherscouldpotentiallywinanawardbyexceedingafixedbenchmark,thefinancialexposureislesscertainandpotentiallymuchgreater.ThecostofoperatingtheprogrammecouldrangefromUSD0(noteachersmeetthestandard)toUSD4million(40winningteachersx20schoolsxUSD5000perwinningteacher).

Stakeholderswantingtointegratearank-orderedtournamentaspartofanincentive-payprogrammeshouldexploreconstructingcomparablepeergroupstomeasureandevaluatetheperformanceofindividualteachersorteamsofteachers.Neal(2009)suggeststhatschoolsystemsconstructleague-specifictournaments;thatis,aschool’sperformanceisconsideredrelativetotheperformanceofschoolsthatservesimilartypesofstudentsandfamilies.Withoutthesecomparisonsetsofhomogenousschoolsthatareregularlyupdated,Nealargued,it is impossible to obtain credible estimates of a school’s or teacher’s contribution to student achievementoutcomes.Conceivably,thisnotionofleague-specifictournamentscanbeappliedattheclassroom-levelwhilefurtherspecifying thematchingcriteriaonwhich leaguesare formed(i.e.subject taught,numberofclassestaught,averageclasssize),thusminimisingwithin-schoolvariation.

Rank-ordertournamentscansufferfromonewell-recogniseddefect:promotionofcompetitionamongteachersorgroupsofteacherscanleadtoabreakdowninthecollegiateethos,therebyreducingratherthanincreasingperformance.Thiscanbecomeaseriousthreatifteachersinthesameschoolarenolongerwillingtohelponeanother.Forexample,thegradefivemathematicsteachermayoptagainstsharingalessonplanwiththeothergradefivemathematicsteacherbecausetheyarecompetingforthesamepotofmoney.Indeed,allowingonlysometeachers,ratherthanallwhometapredeterminedstandard,toreceiveabonuswasattributedwiththecollapseofmanymerit-payprogrammesinthe1980s.

Makinghigh-stakescomparisonsofteachersorteamsofteacherswithinaschoolcanencourageteacherstosabotagetheperformanceoftheirpeers(Burgesset al.,2001;Neal,2009).MurnaneandCohen(1985)suggestedsabotagecantakeplaceintheformofteachersusingfriendshipswithstudents’parentstospreaddamagingrumoursaboutacolleague’sabilityasateacher.Gonring,TeskeandJupp(2007)didnotfindany“insider-evidence”ofunhealthycompetition,however,createdbyDenver’sProfessionalCompensationSystemforTeachers(ProComp).



68

© OECD 2009

Anotherprimeconsiderationindevelopinganincentive-payprogrammeisthatdistrictofficials,legislatorsandotherindividualsresponsibleforfundingbonusestendtoberisk-averse.Elmore,AbelmanandFuhrman(1996)notedthat,“Althoughlegislatorsandgovernorsfrequently‘talktough,’theyareverysusceptibletopressuresbyschoolpeopletobackdownandsoftenpoliciesperceivedtoberestrictiveorpunitive”(pp.85-86).IntheirrecentreviewoftheNewPublicManagementreformmovement,whichisinextricablylinkedtotheresurgenceofinterestinperformance-relatedpaypolicies,Perry,EngbersandYun(2009,p.14)similarlyobservedthat,“Itisironicthatthesamepoliticianswhopromoteperformance-relatedpayalsomayvoteagainstappropriationstofunditiftheyperceivefiscalrestraintserveslargerpoliticalends.”

Incentive-payreformsarealsovulnerabletoshiftsinpoliticalleadershipandpublicopinion.Whensupportinglegislators,superintendentsand/orschoolboardmembersleaveoffice,thepoliticalwilltocontinueeducatorincentive-payprogrammescandisappearrapidly(Ballou,2001;BallouandPodgursky,1997;McCollum,2001).Giventheuncertainandpotentiallysubstantialcosts,incentive-payprogrammesbasedonfixedperformancecontracts, for example, are likely to be among the first line items on the political chopping block.Thus itis important toconstruct an incentive-payprogramme that appearsfinanciallypractical toawide rangeofeducators,aswellastoelectedandappointedofficials.

Fixed performance contract

Thesecondmajorformofincentivestructureisthefixedperformancecontract,whichdefinestheperformancestandardteachers,teamsorschoolsmustmeettoearnanaward.Thestandardcantakeonanumberofforms,includingasinglethreshold,multiplethresholds(e.g.astepfunction),oracontinuous,linearstandardaftersomethresholdhasbeenmet.Incontrasttoarank-ordertournament,anyteacherwhomeetsthepredeterminedperformancestandardbenefitsregardlessoftheperformanceofotherteachers.

Theprimaryrationaleforadoptingafixedperformancecontract ispragmatic–fixedperformancecontractsavoid competition among teachers.Teaching is characterised by widespread compliance to an implicit orexplicitcodeofprofessionalethicswhichcanbedrivenby self-interested reputation, internalisedmorality,peeresteemandaserviceideal(Mallick,unknown).Competitionmaythreatenprofessionalnorms,particularlythoserelatedtocollaborationandteamwork,whichsomestudieshavefoundtobekeyingredientsinhighlyeffectiveschools(BrykandSchneider,2002;HallingerandMurphy,1986).

Manyteachers’unionsstronglyopposetheintroductionofcompetitivepressuresintoschoolsorschoolsystems,irrespectiveofwhetherpressurescomeintheformofanincentive-paypolicyorfrommarketcompetition,suchas schoolchoice. Inananalysisofmultiple-wavesofdatacollected in theU.S.DepartmentofEducation’sSchools and Staffing Survey,Ballou (2001) found a strong inverse relationshipbetween theuseof pay-for-performance programmes and the degree of union influence.Tomlinson (2000) similarly reported that theNationalUnionofTeachersopposedpay-for-performanceproposalinEnglandandWalesbecause:

TheGovernment’sproposalswouldprofoundlydamagetheprofessionalcultureandteamworkthatisattheheartofsuccessfullymanagedschools.Thepressurescausedbytheperformancemanagementstructurewouldgeneratedistrustandcounter-productivecompetitionbetweencolleagues.Teacherscouldbedemotivatedanddeprivedoftheconfidencetheyneedtobesuccessfulintheclassroom.Stress,anxiety,apprehensionandselfdoubtarenotconducivetosuccessfulteaching(p.6).2

Fixed performance contracts set clear standards which enable teachers to set personal goals in relationto those standards. Goal-setting theory, which refers to the establishment of a purpose in reference to aperformancestandard,helpsdirectattentionandaction,furthermotivatingindividualstowardthestandard



and,ultimately, leadingtobetter taskperformance(Locke,1968;Locke,LathamandSmith,1990;Locke,Shaw,SaariandLatham,1981).Recognizingthatrank-ordertournamentincentiveschemesarecontingentona“do-your-best”mentality,teachersaremorelikelytorespondtoafixedperformancestandardbecausea relative performance standard can be perceived as impossible to attain, or teachers may conclude theevaluationsystemisbiasedbecauseteachersdonotknowwhatstandardtheyneedtoachieveinordertoearnanawarduntilafterthefact.

Conversely,thelargefinancialexposureinherentinmostperformancecontractshasbeenassociatedwithteachers’skepticismaboutincentive-paypolicies.IntheUnited Kingdom’s(U.K.)PerformanceThresholdandUpper-ScalePaySystem,teachersandprincipalsquestionedwhetheradequatefundingexistedtoimplementandsustaintheprogrammegiventherewasnoquotaonthenumberofteacherswhocouldearnanaward(Adnett,2003;CutlerandWaine,2004;MarsdenandBelfield,2006).Skepticismcanreducethelikelihoodthatteacherswillincreasetheireffortinpursuitofabonusaward.Itcanalsoincreaseopportunisticbehaviouron thepartof teachers,andcandecrease their levelofbuy-in (Hamilton,2005;Kelley,2002;LewisandSpringer,2008).

80

70

60

50

40

30

20

Percentage of students scoring proficient

2001 2002 2003 2004 2005 2006 2007 School years

Source: OECD.

Reading Mathematics

Figure 3.1Percentage of schools with a minimum of 72% of third grade students scoring proficient

in reading and mathematics

Fixinginadvancethetotalnumberofindividuals,teamsofteachers,orschoolsthatcanearnbonuses(i.e.arank-orderedtournament)helpsmitigateseveralcomplicatingfactorsassociatedwithjudgingtheirperformance.Forexample,Figure3.1displaysthepercentageofschoolswithatleast72%oftheirthird-gradestudentsscoringproficient in reading and mathematics as measured by Minnesota’s minimum competency accountabilityprogramme.Thepercentageof schoolsmeeting the72%standardmore thandoubled from the2000-01 to2006-07schoolyear,animpressiveincreaseparticularlygiventherelativelyshorttimeseries.



70

© OECD 2009

NowimaginetheMinnesotalegislaturedesignedandimplementedaperformance-payprogrammethatrewardsanyperformanceaboveaminimumthreshold,whereintheunitofaccountability is theschool.Assumetheprogrammealsoreliesonasingleperformancestandardfordeterminingbonusawardeligibility–anyschoolwithatleast72%ofitsstudentsscoringproficientreceivesUSD2500perfull-timeequivalentinstructionalemployee.Inthefirstyearoftheprogramme(2001-02schoolyear),thiswouldhavemeantthatapproximately255schools,orroughly30%ofeligibleschools,mettheperformancestandard,translatingtoUSD19.2millioninbonusawards(wewillassume30teachersperschoolthroughoutthishypotheticalexample).

Since the legislature opted for a fixed performance contract incentive structure, the amount of resourcesnecessarytoremunerateteachers increasesdramatically inarelativelyshortperiodof time. Inthe2002-03schoolyear,morethan420schoolsmettheperformancestandard,whichmeansbonusawardstotaledaroundUSD32.8millionaccordingtothehypotheticalparameters.Twoyearslater,morethan560schoolsmetthestandard,addinganadditionalUSD10milliontothe2002-03figure.Thiswouldplaceahugefinancialburdenonthetaxpayer,aswellasonthebudgetoftheagencyoperatingtheprogramme.3

Some literature suggests school personnel and bargaining organisations are prone to exploit the fixedperformancecontract incentivesystem.Employees“expend time,effort,and ingenuity” to influenceothers’decision-making in an attempt to maximise their own benefit (Holmstrom and Milgrom, 1987). Since thedesign (or reauthorization)ofan incentive-payprogrammewill involve teachers,and/or thebargainingunitprotecting the interests of the “median” teacher, these individuals may advocate for a lower performancestandard (Indjejikian,LenkandNanda,2000;MerchantandManzoni,1989;Murphy,1999).AsHeneman,MilanowskiandKimball(2007)noted:

Overtime,teachersexertpressuretolowerperformancestandards,increasingthenumberofteacherswhobecomeeligibleforperformancepay…whilesuchaneventualitymayenhanceacceptanceoftheplan,italsodrivesupcosts,creatinglong-termfundingissues(p.6).

Schoolsystemsalsoneedtobemindfulofapotentialratcheteffectwhenelevatingtheperformancestandard(Freixas,GuesnerieandTirole,1985;Weitzman,1980).Ifaschoolsystemincreasestherigouroftheperformancestandard that an individual has to meet to earn a bonus award, teachers and other school personnel mayperceivetheheightenedawardthresholdasapunishment.Ifso,therecalibrationprocesscanpotentiallyreduceproductivity,compromisetrustandinformationsharingamongschooladminstrators,schools,andteachers,andcreatesufficientoppositionsuchthatthefutureoftheincentive-payprogrammeisputintoquestion(Lazear,1995).Onestrategytoovercomenegativereactiontoincreasingthestandardistoincreasethesizeofthebonusorthepercentageofemployeeseligibletoearnthebonusatthesametimethestandardsarerevised.

Anotherconsiderationindeterminingperformancestandardsisensuringthestandardisnotbiasedinfavourof particular schools, teams, or teachers. In their evaluation of New York city’s School-Wide PerformanceBonusProgram(SPBP),SpringerandWinters(2009)studiedtherelationshipbetweentheperformancetargetassignedtoparticularcategoriesofschoolsandtheprobabilitythatschoolsinthosecategoriesmetatleastpartoftheirperformancetarget.Schoolswereclassifiedaccordingtothepreviousyear’sprogress-reportdata,withCategoryOneschoolsbeingthehighestperformingandCategoryFivethelowest.Accordingtotheauthors,theprobabilityofCategoryFourschoolsandCategoryFiveschoolsearningatleastpartoftheirperformancebonusawardwasnearlytentimesgreaterthantheprobabilityofCategoryThreeandCategoryTwoschools.Whiletheauthors’emphasisedthesefindingscamefromthefirstyearoftheprogramme,ifasimilarpatternemergesinsubsequentyearsonemightarguetheSPBPisrewardingmediocritygiventhatapercentageofschoolsinCategory Four and Category Five will meet their performance target irrespective of any effort put forward.Moreover,theseschoolswerethelowestperformingschoolsidentifiedbythedistrict’saccountabilitysystem.



unIt oF accountabIlIty

Thenextdesigncomponentofanincentive-payprogrammeistheunitofaccountability.Theunitofaccountabilityrefers to theentity responsible forameasurableproductorservicewhoseperformanceon thatmeasurabledimensiondeterminesbonuseligibility.Theunitofaccountabilitycanbedefinedinamyriadofways,includingtheindividualteacher,agrade-levelordepartmentalteamofteachers,allemployeeswithinaschool,orsomecombinationthereof.

Therearedistinctadvantagesandlimitationsthatpolicymakersneedtocarefullyconsiderwhendefiningtheunitofaccountabilityinincentive-payprogrammes.Thisnextsectionbeginswithadiscussionoftheindividualastheunitofaccountabilityandthenshiftsattentiontothegroup,wherethegroupmaybedefinedasawithin-schoolteamofteachersoranentireschooltakenasasingleunit.Thefinalsectionprovidesasimilarreviewofhybridmodelsofaccountability.

Individual unit of accountability

Inanincentive-payplanthatreliesonanindividualunitofaccountability,theperformanceoftheindividualteacher determines award eligibility.The individual unit of accountability creates the strongest connectionbetween variation in award size received by teachers and the variation in teacher effectiveness. Since theinability todirectly impactone’schancesofearningaperformanceawardcanleadtodecreasedeffort, theindividualunitofaccountabilityisoftenidentifiedasoptimalbecausetheindividualassumessoleresponsibilityforhisorherperformance(FreemanandGelber,2006).

Incentive-payprogrammesintheeducationsectorthatrewardteachersattheindividuallevelofaccountabilitymaximiseconditionsforalteringthecompositionof theteacherlabourforce.Themosteffectiveteachersinthesystemaremorelikelytoberetained,sendingastrongsignaltothoseteachersnotreceivingarewardtoimproveorexit.Additionally,prospectiveteachersmostlikelytoberewardedundertheperformanceappraisalsystemaremorelikelytojointheprofession.Thus,asPodgurskyandSpringer(2007)argue,teacherturnoverwouldbecomepartofavirtuouscycleofqualityimprovement,ratherthanaproblemtobeminimised.

The theoreticalunderpinningsof an incentive-payprogramme that rewards individual teacherperformanceiscriticallyimportantinlightofinefficienciesincurrentteacherhiringpracticesandlabourmarketselection(Lazear,2000).Intermsofhiringpractices,principalsandbuildingadministratorsmustusenoisysignalsof“true”teachereffectiveness,suchasyearsofexperience,highestdegreeheld,orpast-employerrecommendations.Informationaldeficienciesinthehiringprocessareovercomeinmostprofessionsbyemployeeperformanceassessmentsandtheclosecouplingofpayincreasesandpromotiondecisionswithactualproductivity.Withoutsuchstructuresineducation,combinedwithanofteninsurmountabletenuresystemthatmakescontractnon-renewaldifficult,payforperformancebecomesallthemorerelevant.

Intermsoflabourmarketselection,anincentive-payprogrammetendstoattractandretainindividualswhoareparticularlygoodattheactivitytowhichincentivesareattached,whilerepellingthosewhoarenot.Whileincentivescanraisetheproductivityofthetypicalworker,anincentivesystemcanalsoraisetheoverallqualityoftheworkforcesimplythroughdifferentialrecruitmentandretentionofmoreeffectiveworkers(PodgurskyandSpringer,2007).Lazear’s(2000)widely-citedcasestudyofSafeliteGlassCorporation,forexample,reportedthattransitioningthecompany’scompensationsystemfromhourlywagestopiecerateswasassociatedwitha44% increase inworker productivity, half ofwhich resulted from the sortingofmore ableworkers. In acasestudyofaBritish columbia tree-plantingfirm,PaarschandShearer(2000)similarlyfoundincreasesinproductivityattributabletobothincreasedeffortandsorting.Ofcourse,theproductionprocessandoutputforteachersandschoolsismuchmorecomplex.



72

© OECD 2009

Arecentandprovocativestudyofteacherturnoverfoundevidencethatthemigrationofhighabilitywomenoutofteachingbetween1960andthepresentprimarilyresultedfromthe“push”ofteacherpaycompression,whichtookawayrelativelyhigherearningsopportunitiesforteachers,asopposedtothe“pull”ofmorelucrativenon-teachingopportunities(HoxbyandLeigh,2004).Althoughremunerativeopportunitiesoutsideofteachingforteachersofhighandlowabilitygrewoverthisperiod,HoxbyandLeigh(2004)argueditwaspaycompressionwithintheeducationsystemthatacceleratedtheexitofhigherabilityteachers.Similarly,Corcoran,EvansandSchwab(2004) foundthelikelihoodthata femaleinthetopofherupper-secondaryschoolclasswillenterteachingdecreaseddramaticallybetween1957and1992.4

However, the literature on optimal incentive structures questions whether the individual as the unit ofaccountabilityistheoptimalincentivedesignwhenoutputscannoteasilybeattributedtoanindividualteacher.Ifwork tasksof twoormore individuals influence themeasuredand rewardedactivity inan incentive-payprogramme,relyingontheindividualastheunitofaccountabilitymaycausesometeacherstotakeexceptiontohighperformersbecausetheymayaffectthestandarduponwhichallotherteachersarejudged(BaronandKreps,1999;BowlesandGintis,2002;Deutsch,1985).Moreover,asdiscussedpreviously,rewardingteachersfortheirindividualperformanceisalsocontrarytothehighlycollaborativenatureofteachingandmayevenreducetheincentivetocooperatewithothers(Mallick,dateunknown;MilgromandRoberts,1990;MurnameandCohen,1986).

Therearenumerousotherinterdependenciesthatcomplicateeffortstoisolatethecontributionofanindividualteachertostudentoutcomes.Iftheindividualistheunitofaccountability,howshouldtheevaluationsystemtake into account the cumulative effects of learning from one school year to the next?Are there practicalstrategies for taking intoconsiderationdepreciationorappreciation in learningover time,andwhether theinstructionalstrategiesoftheprioryear’steacherinfluencelearninggainorloss(i.e.teachingtothetestversusteachinghigherorderskills)?Identifyingtheindividualcontributionofateachertostudentlearningisfurthercomplicated,forexample,whenstudentsenrollinareadingandanEnglishlanguageartscourse,eachofwhichis taughtbyadifferent instructor.5 Individual incentive-payprogrammescanalsocreate resentmentamongteachers,asteachersnotreceivinganawardmaybelievetheycontributedtothesuccessofstudentslinkedwithateacherthatreceivesabonusaward.

Factorsinfluencingstudentlearningalsoextendoutsidetheschoolbuildingwalls.Forexample,intheirhighlyinfluential work on consumer preferences and school choice, Schneider, Teske and Marshall (2000) notedthat interactions among students and parents make parents co-producers of educational outcomes; parentsundoubtedlyshapetheeducationalgoalsoftheirchildren.Socialscienceresearchhasalsoidentifiedanumberof neighbourhood factors that can influence student outcomes, including social disorganisation, communityresources,environmentaldeterioration(e.g.abandonedbuildings),crimeanddelinquency(e.g.drugdealingandviolentcrime)(Ginther,HavemanandWolfe,2000;LeventhalandBrooks-Gunn,2004;Wilson,1987).Theseandothercomplicatingfactorsmakeitdifficulttoisolateanindividualteacher’scontributiontostudentoutcomes.

Group unit of accountability

The group unit of accountability refers to award eligibility as the product of aggregated performanceamong members of a group, where the size of a group can range from as few as two employees to allemployeeswithinafirm.Sometimesreferredtoasprofitsharing,gainsharing,teamincentives,goalsharing,achievementsharing,winsharing,andresultssharing(GaynorandPauly,1990;Holmstrom,1982),groupincentivestructureshaveexperiencedtremendousgrowth inpopularity. In theU.S., thenumberofgroupincentive-payprogrammesoperatinginfirmsincreasedfromjustover2000in1945tonearly500000in1991(NalbatianandSchotter,1997).



Itisimportanttostressthatthegroupunitofaccountabilityisnotlimitedtotheschool-level,acommonmistakeencounteredamongincentive-payprogrammesthroughouttheeducationsector.Groupincentivesystemscantakeonanumberofdifferentforms,includingmonitoringandrewardingperformanceofgrade-levelteamsofteachers,disciplinaryorinter-disciplinarydepartmentsofteachers,oranyotherrecognizableunitwithintheschool.

Organisational theory suggests group incentives can promote social cohesion, feelings of fairness, andproductivitynorms(Lazear,1998;Pfeffer,1995;Rosen,1986).Improvedsocialcohesionamongworkerscanfosterknowledgetransferandmutuallearningthatresultinincreasedproductivityinthelongrun(CheandYoo,2001).Forexample,inthecasestudyofgarmentplantsreportedinBerget al.(1996)andHamilton,NickersonandOwan(2003),theformationofteamswithworkersofvaryingabilitiesfacilitatedinteractionsamonghigh-andlow-abilityworkerssothatmoreableworkerstaughtlesseffectiveworkershowtobetterexecutetasksandbecomemoreproductive.

Although productivity gains from knowledge transfer and mutual learning may yield results similar to anindividualincentiveprogramme,someempiricalresearchdescribesamuchmorenuancedstorywhenfirmshave adopted team-based incentive programmes. Weiss (1987) and Hansen (1997) found that while theperformanceoflower-abilityworkersimproved,theproductivityamongthehighestproducersdecreased.Thusthegroupincentive-payprogrammemaynothavethesamecompositionaleffectontheworkforceand,asaconsequence,theexpectedbenefitofthegroupunitofaccountabilitymaybeallfornot.

Thefree-riderproblemandshirkingareotherconcernsdesignersofincentive-payprogrammesneedtorecognisewhenconsideringthegroupastheunitofaccountability.Iftheschoolistheunitofaccountability,orthesizeofthewithin-schoolteamisgreaterthansixtoeightteachers,specificindividualsonateammaybelesslikelytoshouldertheirfairshareoftheworkload.Theyknowthecapabilitiesofteammatescanmakeupfortheirsubparperformance.Becauseanindividualwhoexertedminimaleffortwillstillreceiveabonusawardiftheperformanceoftheothermembersofhisteamoffsethismediocreeffort,groupincentivesystemsinevitablyresultintheinefficientallocationofsomeresources.

KandelandLazear (1992)andothershaveargued thatas longas the sizeofawithin-organisation team isnottoolarge,thefree-riderproblemcanbesolvedthroughpeerpressure.Forinstance,peermonitoringandtheenforcementofsocialpenaltiesintheformofshame,guilt,empathy,andmutualmonitoringcanleadtoindividual teammembersbeingaccountable for theirperformance toothermembers. Ifa teacherhasbothmonetaryandsocial incentives tonotshirk,KandelandLazear (1992)contendthemotivational forces thatwouldhavebeen“chokedoff”bythefree-riderproblemarerecovered.

However,even though the theoretical literaturehasargued that the free-riderproblemcanbeovercome incertainsituations,alarge-scalerandomassignmentevaluationrevealedindividualteacherincentiveselicitedgreatergainsandproductivityascompared togroup incentiveprogrammes.Knownas theAndhraPradeshRandomized Evaluation Study (AP RESt), the impact of two output-based incentive systems in India (anindividualteacherincentiveprogrammeandagroup-levelteacherincentiveprogramme)andtwoinput-basedresource interventions (one provided an extra-paraprofessional teacher and another provided block grants)were evaluated. Muralidharan and Sundararaman (2008) found that students enrolled in a class instructedbyateacherselectedforthegroupincentiveinterventionoutperformedstudentsincontrolconditionclassesthat receivedonly theirusualallocationsof resources,onboth themathematicsand languageexams (0.28and0.16standarddeviations,respectively).6Studentsenrolledinschoolsassignedtotheindividualincentiveconditionoutperformed students in both the group incentive condition and the control condition after thesecond year of implementation. In terms of cost effectiveness, moreover, the authors reported both of theincentive-pay interventionswerepreferable as compared to the input-based resource interventions and thecontrolcondition.



74

© OECD 2009

Anincentive-payprogrammerelyingonthegroupunitofaccountabilitymayalsofail torecognisesomeofthemosteffectiveteachersinaschoolorschoolsystemifthoseindividualsareassignedtoateamwithlower-ability teachers.High-ability teacherswhoarenot rewardedmay leave the school systemor profession asaconsequenceofbeingdiscouragedwithaperformancemanagementsystemthatdoesnot recognise theirindividualperformance.Rivkin,HanushekandKain (2005) furtherdeducedschool-basedincentivesystemsmayalsobeahighly-inefficientapproachforrewardingperformancegiventhepresenceofsignificantwithin-schoolvariationinteachereffectiveness.Recognizingthattwoofthekeygoalsofincentive-payprogrammesare to improvethecompositionof the labour forceandorganisationalproductivity, it iscritically importantthatstakeholdersdesigninganincentivesystemcloselymonitorandevaluatetheprogrammeforanysignsofperversesorting.

Afinalconsiderationwhenadoptingagroup incentiveplan isprocedural.Howshould thecompositionofwithin-schoolteamsbeformed?Thepurestapproachtoassigningteacherstoteamsisasimplerandomlotterywherein each teacher has an equal chance of being assigned to a given group (Lazear and Gibbs, 2009).Assumingthenumberofindividualsinthelotteryislargeenoughinrelationtothenumberofteams,randomlyassigning teachers to teamsmeansworkabilitywillbebalancedacross the teams.Anotherstrategy is foraschoolprincipaloradministratortoassignindividualteacherstoteams.Mostfirmsrelyonsupervisorstoselectthecompositionofteams,whichmakesthemostsenseifthepoolofcandidatesbeingassignedtoteamsisnewtothecompany(LazearandGibbs,2009).LazearandGibbs(2009)furthernotedanalternatedrawistypicallyidentifiedastheoptimalstrategyifschoolswanttomaketeachersresponsibleforthemake-upofthewithin-schoolteams.7

Hybrid models of accountability

Hybridmodelsofferathirdtypeofaccountabilitysystem.Ahybridmodeljoinselementsofbothindependentandinterdependentwork,meaningtheincentivestructureconsiderstheuniquecontributionofanindividualteacherwhilealsosupportingteamworkandcollaborationamongteachers.HybridmodelsaremorecommonlyfoundintheU.S.and,asdiscussedlaterinthischapter,mayberelatedtotheculturalcontextinwhichanincentivesystemoperates.

Initiatedin1999bytheMilkenFamilyFoundation,theTeacherAdvancementProgram(TAP)isawell-knownincentive-pay programme drawing on a hybrid model of accountability. Bonus calculations incorporatestandardisedtestsscores forbothindividual teacherperformanceandtheperformanceof theentireschool.Whileeachschoolcanultimatelydecidetheexactpercentagetoattributetoeachunitofaccountability,TAPguidelinesrecommendabreakdownof20%schoolleveland30%individuallevel.8

Establishinganappropriatebalanceamongdesigncomponentscanposeanumberofchallenges.Iftoolittleweight isplacedonsomeaspectof the schoolingprocessperceivedby teachersandother stakeholdersasimportant,thentheinfluenceoftheincentiverunstheriskofbeingdilutedtothepointthatthepayprogrammebecomes irrelevant.Sucha scenariocanalso senda strong signal thataparticularaspectof the schoolingprocess isnot valued,whichmay lead todiscouragementandabreakdown in trustbetween teachers andadministrators.Forexample, in their studyof teacherperceptions toFlorida’s BetterEducatedStudentsandTeachers (BEST)Teaching Salary Career Ladder Program, Mullen and Slagle (2007) reported that teachersperceivedashighly-flawedthewayinwhichtheBESTprogramconceptualisedteaching.

Ahybridmodelofaccountabilitynecessitates that the technical requirementsofbothgroupand individualunits of accountability be in place. Systems that currently have the capacity for linking records betweenindividualteachersandstudentswillnothavesignificantfurtherrequirementsifalsoadoptingaggregateunitsofaccountability.Conversely,thoseschoolsystemsstartingwithagroupastheunitofaccountabilitymaynothave



thedatalinkagenecessaryformeasuringperformanceattheindividualteacherlevel,orthetechnicalcapacitytomanageanincentive-payprogrammewithamoredisaggregatedaccountabilityunit.Currentcapacityandfuturecostsofusingahybridmodelcouldbesignificantandshouldbeinvestigatedbeforehand.9

Alackofgoalclarityismorelikelytoaffectthemotivationaleffectofanincentive-payprogrammerelyingonahybridunitaccountability.Inasummaryreportofperformance-relatedpaypoliciespreparedfortheU.K.’s NationalUnionofTeachers,Richardson (1999,p.20)noted, “…amultiplicityof goals is likely tocauseproblems,becauseitreducesgoalclarity.Beingunderpressuretomeetmanygoals,especiallywheresomeofthemareambiguous,makesitdifficultforemployeestofocustheireffortsproperly.”Recognisingtheinverserelationshipbetween thenumberofdesigncomponents incorporated into incentive-payprogrammesandteachergoalclarity,itisimportantforhybridmodelsofaccountabilitytoclearlyspecifyhowtheprogrammeworks.

Somechallenges regardinggoalclaritycanbemitigated if theprogrammedesign teamestablishesamulti-prong strategy to enhance teachers’ and school personnel’s understandingof theprogramme. Stakeholderscan train schooladministratorson the ins-and-outsof theprogrammes tohelp facilitate implementationaswellasbuy-inamongteachers. Infact, intheirevaluationofschool-basedperformanceawardprogrammesin Kentucky and North carolina, Kelley (1999) reported programme buy-in is largely conditional on theconfidenceteachershaveaboutthesystemfunctioningasintendedandontheresponsivenessofadministratorstoquestionsfromteachersabouttheprogramme.10Chapter7presentsadetailedreviewofimplementationandtechnicalassistanceissues.

PerFormance measures

Performance measures refer to the evaluation criteria for gauging employee performance. Despite manyeducationsystemshavingalonghistoryofevaluatingteachersandschools,thereremainsconsiderablevariationinstakeholders’perceptionsofwhatshouldbeevaluated,howappraisalcriteriashouldbelinkedtorewards,andwhichmeasuresandinstrumentscanreliablyandaccuratelyreflectperformance.

Table3.1displaystheprimaryperformancemeasuresimplementedinahandfulofhigh-profileincentive-payprogrammes.Slightlymorethanhalfoftheprogrammesincludeaninput-oraprocess-orientedperformancemeasure, which can include professional development, job enlargement, student attendance, and teacherattendance.11 Educational outputs cover student achievement levels and gains at the student, teacher, andschool-levelaswellasaccumulationofcredithoursandstudentretentionrates.

Severalprogrammesalsoincorporaterecruitmentandretentionincentivesorincentivesforteachinginahard-to-staff school or subject. Hard-to-staff school incentives are most often offered for teaching in high-needsschoolsordistricts,typicallyeitherhigh-poverty,low-performing,orgeographicallyremoteschools.Likehard-to-staffsubjectincentives,theseincentivesaredesignedspecificallytoaddressmarketfactors.12AspresentedinTable3.1,weclassifythesemarket-basedcompensationreformsundereducationalinputsandprocesses.

educational inputs

Educationalinputsrefertomeasuresthatrewardteachersforactivitiesbelievedtoimproveinstructionalquality.Examples of the types of activities rewarded under an input-focused incentive system include completingan advanced degree, serving as a mentor or master teacher, taking on a leadership role or other specialduties, participating in professional development activities, and teacher attendance. We first review moreprominenttypesofinput-focusedprogrammesandthendiscussperspectivesregardingsubjectivemeasuresofperformance.



76

© OECD 2009

Knowledge-basedpayandskill-basedpayprogrammesarethemostprominenttypeofincentive-paypoliciesthat focus on rewarding educational inputs. Developed by a team of researchers with the Consortium forPolicyResearchinEducation(CPRE)attheUniversityofWisconsin,theseincentiveplansinduceteacherstoobtainnewskillsrelatedtoclassroommanagementandknowledgeofcurriculumdevelopmentandcurriculumcontentincoresubjects.Severalhigh-profileinitiativeswerelaunchedintheU.S.inthelate-1990s,includingprogrammes in Cincinnati (Ohio), Douglas County (colorado),Vaughn (california), and Washoe County(Nevada)(Milanowski,2003).13

Theacquisitionofnewknowledgeandskills isbelieved to improve instructionaleffectivenessofa teacherandconsequently studentoutcomes (OddenandKelley,1997). Input-based incentivesystemsaredesirablebecause theyareunlikely tocause “teaching to the test”,or to increase thechanceof teachers focusingadisproportionateamountofefforttowardthosestudentsalreadynearsomeperformancestandardinhopesof

Table 3.1Characteristics of incentive pay programmes in the United States and other countries

period

teacher performance Measures Size of bonus (USD, normal)

Unit of accountability Minimum Maximum % Monthly Salary

United StatesSchoolIncentiveProgram(Dallas,Texas) 1992-1995 School $450 $1 000 10%-22%

MeritPayProgram(StateofMichigan) 1996-1997 Teacher $1 000 $5,000 12.5%-62.5%

TeacherAdvancementProgram (UnitedStates) 1999-present Hybrid(TeacherandSchool) $2 500 $12 000 45%-216%

AchievementChallengePilotProject (LittleRock,Arkansas) 2005-2007 Teacher $350 $7 600 8%-174%

GovernorEducatorExcellenceGrantProgram(StateofTexas) 2005-2008 Varies(Teacher,Team,and/orSchool) $75 $15 000 1.3%-270%

ProfessionalCompensationSystemforTeachers(Denver,Colorado) 2005-present Hybrid(TeacherandSchool) $400 $2 500 7.2%-45%

ASPIREProgram(Houston,Texas) 2005-present Hybrid(TeacherandSchool) $250 $8 600 4.5%-154.8%

POINTExperiment(Nashville,Tennessee) 2006-2009 Individual $5 000 $15 000 90%-270%

TexasEducatorExcellenceGrantProgram(StateofTexas) 2007-present Varies(Teacher,Team,and/orSchool) $20 $20 462 0.4%-365%

TeamIncentiveProject(RoundRock,Texas) 2008-present Team $6 000 108%

School-WidePerformanceBonusProgram(NewYorkCity,NewYork) 2008-present School $1 400 $3 600 25.2%-64.8%

InternationalSchoolPerformanceProgram(Israel) 1996-1997 School $1 000 $2 500 30%-75%

InternationalChristelijkSteuenfondsIncentiveProgram(BusoandTeso,Kenya) 1998-1999 School $26 $51 21%-43%

Salario al Merito(Bolivia) 1998-1999 Teacher $300 $333 66%

Teacher-IncentiveExperiment(Israel) 2001 Teacher $1 750 $15 000 10%-40%

Carrera Magisterial(Mexico) 1993-present Teacher … … 27%-224%

Sistema Nacional de Evaluacion de Desempeno de los Establecimientos Educacionales(Chile)

1996-present School $439 $439 4.7%-7.2%

PerformanceThresholdandUpperPayScaleSystem(England) 1999-present Individual $2 000 9%

Incentivo Colectivo a Escuelas(Bolivia) 2001-present School $281 $281 5%-19%

RandomizedEvaluationProject(AndhraPradesh,India) 2006-present TeacherorSchool $2.25 $450 33%-50%

PerformancePayProgram(Portugal) 2007-present Teacher … … …

Sources:Atkinsonet al. (2009);ClotfelterandLadd(1996);Eberts,Hollenbeck,andStone(2002);Glewwe,Holla,andKremer(2009);Glewwe,Ilias,andKremer(2008);Ladd(1999);Lavy(2002,2004);McEwanandSantibanez(2005);MizalaandRomaguera(2004);MuralidharanandSundararaman(2008);PodgurskyandSpringer(2007);Rodriguez(2002);Santibanezet al.(2007);SchacterandThum(2004);Springer,Ballou,andPeng(2008);Springeret al.(2008,2009);SpringerandWinters(2009);Taylor,Springer,andEhlert(2009);VegasandUmansky(2005);Winters,Ritter,Greene,andMarsh(2009).



maximizingchancesofearningaperformanceaward.Firestone(1994,p.550)furtherarguedthattheseinput-basedpayprogrammesare“compatiblewithanoutcomesfocusbecausetheresultingflexibilityallowsworkerstodowhatisnecessarytoachievethoseoutcomes”.

Anotherappealingfeatureofinput-basedsystemsisrootedinthebeliefthattheopportunitytopursueadditionalcourseworkortotakeonmoreresponsibilitiesincreasesteacherinterestandmotivation(OddenandKelley,1997).This feature isveryattractive if in fact itdoesfindaway tobalance thepotential threatofexternalrewardsreducingintrinsicmotivation,particularlyconsideringteachersaredrawntotheprofessionbyintrinsicfactors(Kohn,1999;Lortie,1979;Rosenholtz,1984).AsnotedbyJohnson(2004,p.46),“Initself,higherpayisunlikelytoretainteachers–particularlythemostableamongthem–iftheyfindthattheycannotattaintheintrinsicrewardsforwhichtheyinitiallyenteredteaching.”

Socialpsychologicalliteratureonpublicservicemotivationfurtherarguesthatindividualsseektocontributeto the public good in order to satisfy personal needs, not necessarily in response to incentives offered byorganisationsforperformanceorcommitment(Courty,Heinrich,andMarschkeet al.,2005;PerryandPorter1982; Rainey 1982;Wise 2004).Wise (2004), in particular, suggested that public-service motives had thepotential toempowerpublic servants to“overcomeself-serving interests,moral inertiaand riskavoidance”(as cited in Courty, Heinrich and Marschke, 2005), an effect to which Crewson (1997) attributed greaterorganisationalcommitmentandloweremployeeturnover.

Despiteproponents’contentionthattheactivitiesrewardedbyknowledge-andskill-basedpayprogrammesareassociatedwithstudentachievement,agrowingnumberofempiricalstudiesindicateteachereffectivenessis,atbest,marginallycorrelatedwithobservableteachercharacteristicssuchasteachingcertificateheld,levelofeducation, licensingexamscores,and teachingexperiencebeyonda fewyears (Goldhaber,BrewerandAnderson,1999).Forexample,Goldhaberet al.(1999)foundthatobservableteachercharacteristicsexplainonlyabout3%ofthedifferencesinstudentachievementthatareattributabletotheteacher.Inarelatedstudy,Goldhaber (2002, p. 50) argued that, “The evidence shows that good teachers make a clear difference instudentachievement.Theproblemisthatwereallydon’tknowwhatmakesagoodteacher.”

Manystakeholderscontendpaymentforoutputislikelytobeamoreefficientandproductivewaytoremunerateteachersgivenvariationinteachereffectivenesscannotbeexplainedbyactivitiesrewardedinknowledge-andskill-basedpayprogrammes.IntheU.S.,forexample,97%ofschooldistrictsremunerateteachersoncriterialargelyunrelatedtoschoolingoutcomes,whilecompensationpaymentsforinstructionalpersonnelaccountforapproximately55%ofK-12currentexpendituresand90%ofinstructionalexpenditures(Springer,2009).

Inputmeasureshavebeenfoundtoberelatedtostudentperformance,notablyobservationalevaluationsthatcanbeconductedbypeers,supervisors,orexternalevaluators.Studieshavereportedthatastandards-basedevaluationcanpredictstudentachievementandcanthereforeprovideperformanceevaluationscoresasthebasis foraperformance-basedincentive-payprogrammeorotherpersonneldecisions (Kimballet al.,2004;Milanowski,2004).Additionally,Odden,Borman,andFermanich(2004)reportedobservationalevaluationsnotonlyprovidedamore“comprehensivemodel”ofteachereffectsonstudentachievement,butalsohavethepotentialtoinformteacherpracticebyidentifyingspecificareasofimprovement.

However, the theoreticalandempirical literatureson incentive-payprogrammes ineducation identifya largenumber of concerns associatedwith subjectiveperformancemeasures. Prendergast (1999) argued systematicerrors, if frequentlyobservedwhenemployers rankworkers, threaten the legitimacyofsubjectiveassessment,oftenleadingtoleniencybias,centralitybias,andrent-seeking.Leniencybiasisrootedinsupervisors’reluctancetorankthepoorestperformingworkers,therebydistortingtheoveralldistributionofrankings(Prendergast,1999).Centralitybias,definedasthetendencyforrankingstotruncatetheactualrangeofperformance,introducesfurther



78

© OECD 2009

distortion into thisdistribution; supervisorsmaydeliberatelydistortorganisational standardsandoutcomes inordertoredistributerewardsonthebasisofpersonalpreference(Adnett,2003;CutlerandWaine,2004;MarsdenandBelfield,2006).Finally,andasafunctionofthesepreferences,subjectiveassessmentinvitesrent-seeking;workersfocuseffortawayfromcriticaltasksandtowardinfluencingsupervisors(PrendergastandTopel,1993).

Further, raters’ knowledge of prior performance appears to affect information processing by framing oranchoringcurrentjudgments(Huber,NealeandNorthcraft,1987).Theorderinwhichtheobserverseesgoodandpoorperformancethereforeaffectsperformancerating,withbiasedjudgmentsaboutinconsistentextremeperformancemovingtowardthepriorgeneralimpression.Teamsofratersmayalsoattachgreaterweightstoselectelementsthanisappropriate(Schmitt,NoeandGottschalk,1986)ortheymaydistortratingsforpoliticalreasons(Longeneckeret al.,1987).

Thecomplexnatureof teachingmakes itdifficult to identify specificcriteria that shouldbepresentduringan observational evaluation. While some have developed observational rubrics from supposed “research-identified”effectiveteachingbehaviours(Danielson,1996;SchacterandThum,2004),extensivetrainingforratersisnecessary.Observationsarealsoveryexpensiveastheyrequiremultipleobservationsonanindividualteacheroverthecourseofaschoolyear.

Inarecentstudyfocusedentirelyonthepredictivevalidityofsupervisorevaluations,JacobandLefgren(2005)assessedtherelationshipbetweenteacherperformanceratings,asidentifiedonadetailedprincipalevaluation,andteachereffects,asmeasuredbystudentachievementgains.Inestimatingteachereffectivenessmeasuresfor202teachersofmathematicsandreadingingradestwothroughsix,JacobandLefgrenfoundastatisticallysignificant and positive relationship between value-added measures of teacher productivity and principals’evaluationsofteacherperformance.

Anotherinterestingdimensionofthisstudywasan“outofsample”predictionof2003studentachievementscoresbasedonprincipalratingsandteachervalue-addedestimatesfrom1998through2002.Studentshadhigheraveragescoresinmathematicsandscienceiftheyhadteacherswithnotonlyhighermeasuredteachereffectiveness inprioryearsbutalsohigherprincipal ratings. JacobandLefgren (2005)demonstrated furtherthattheprincipalevaluationremainedastatisticallysignificantpredictorofcurrentstudentachievementevenwhenteachervalue-added(inthepreviousyear)wasincludedinthemodel.Thisfindingsuggeststhatprincipalevaluationsprovideanimportantindependentsourceofinformationonteacherproductivity.

Subjectiveevaluationsalsohavethepotentialtoserveasimportantformativeassessmentsforteachers,althoughsomehavedeemedsubjectiveperformanceappraisals toofallibleforhigh-stakespersonneldecisions. If theteaching standardsdescribeeffective instructional strategies, as arguedbyOdden,Borman, andFermanich(2004), then when these evaluative tools are implemented in the classroom, student achievement shouldincrease.However,assummarisedbyWeisberget al.(2009),teacherevaluationsystemshavefailedtoproducecredibleandaccurateinformationaboutinstructionalpracticesofindividualteachers.

Eventhoughstudieshaveindicatedthatprincipalsarerelativelyadeptatidentifyingabove-andbelow-averageteachers,itisunclearwhetherprincipalevaluationswouldpersistinahigh-stakesperformance-payprogramme.Thefactthataprincipalidentifiesateacheras“inadequate”onananonymoussurveydoesnotmeannecessarilythatshewilldoso inahigh-stakesenvironment.Take, forexample, twointerventionsdesignedto increaseteacherattendanceratesinpartsofIndiaandKenya.Thefirstprogrammeofferedratherlargebonusesifeligibleteachersregularlycametowork,asmonitoredbytheschoolprincipal.Anevaluationoftheprogrammedidnotdetectanydiscernibleimpactsonanumberofteacherandstudentoutcomes(KremerandChen,2001).Analysis further revealed a majority of school principals distributed bonus awards to teachers, irrespectivewhetherthatteacheractuallymettheperformancestandardsdefinedbytheintervention.



In the second teacher attendance intervention, absence patterns were tracked using tamper-proof cameraswith timeanddate functions, and treatmentgroup teachers receiveda salarycommensuratewith the totalnumberofdays theyattendedworkeachmonth (Duflo,HannaandRyan,2005).Treatmentgroupteachersalsoreceivedamodestbonusforeachdaytheyattendedinexcessof21daysinasinglemonthoranequally-sizedpaydeductionforeachofthe21daystheydidnotshow-upforwork.Dufloet al.(2005)reportedanaveragereductioninteacherabsencerateof19%andanincreaseinstudentperformanceontwostandardisedassessments.

Intheircomprehensivereviewofincentive-paypoliciesindevelopingcountries,Glewwe,HollaandKremer(2009) indicatedtheimpactof theinterventionevaluatedbyDufloandcolleagues mayhavebeendifferentfromthosereportedintheKremerandChen(2001)studybecausethehumanelementwasremovedfromthemonitoringprocess.The literature fromoutside theeducation sector further lends support in thatmanagersoftenfeeluneasyaboutmonitoringcolleaguesincontextswithstronginterdependencies,whichsuggestshigh-stakesperformanceevaluationsconductedbypersonnelwithinthesameschoolcanpossiblybeunreliableandsubjecttoperversebehaviours(MurphyandMargulies,2004).

educational outputs

Incentive-pay programmes have become increasingly focused on rewarding teachers or groups of teachersusingoutcome-orientedperformancemeasures.Testscoresmeasuredonstandardisedassessmentshavegainedpopularityinlargepartbecauseassessmentinstrumentsareroutinelyadministeredbymosteducationsystems,andstandardisedassessmentsrepresentanobjectivemeasureofperformance.Inanexaminationofprofessionalcompensationstructuresthataregenerallysympathetictoreform,forexample,Bok(1993),asquotedinBallou(2001),notedincentive-payprogrammescouldnotworkintheeducationsectorbecauseof the“inherentlysubjectiveandcontentiousprocess”ofevaluatingteachers.

Despitetheconsiderablegrowthinoutput-focusedmeasuresofschool,team,andteacherperformance,theseperformancemeasureshavereceivedmuchcriticism(Rothstein,2009).Unlikesalesorthebillablehoursofadoctororlawyer,ateacher’soutputisnotmeasuredreadilyinareliable,validandfairmanner(PodgurskyandSpringer,2007).Teachinginvolvesacomplexarrayoftasksthatvarybythenumberofstudentstaught,thegradelevelsofthestudents,thecoursesbeingtaught,andtherelationshipofthosecoursestothetests(McCaffrey,HanandLockwood,2009).Thesecomplexitiesresultinuncertaintiesabouthowteachersshouldbeevaluated,andthewayinwhichtheseuncertaintiesareresolvedwillultimatelyaffectthesignalssenttoteachersandotherpersonnel(McCaffrey,HanandLockwood,2009).

Monitoringandrewardingindividualteachersorgroupsofteachersaccordingtostudenttestscorescapturesonlyafractionofthecontributionofschoolpersonnelaswellastheoverallmissionofaschool.IntheU.S.,forexample,approximatelytwo-thirdsofteachersdonotinstructinatestedgradeorsubject.Moreover,thetypicalstudentengagesinalargenumberofacademicandextra-curricularactivitiesbeyondsubjectstestedbymosteducationsystems.Asolerelianceonobjectivemeasures,suchastestscores,thusrunstheriskofteachersfocusingexcessivelyonthemetred,rewardedactivitytothedetrimentofimportantdimensionsofschooling(i.e.focusingontest-takingskillsasopposedtocreativeorcriticalthinkingskills).Additionally,cultivatingandsustainingteachersupportandbuy-incanbeproblematicifateacherbelievestheincentive-payprogrammedoesnotcaptureasignificantaspectof their joborthespecificperformanceobjectiveisunclear(MurnaneandCohen,1986).Asaconsequence,thislackofbuy-inwillweakenbehaviouralresponsestotheincentiveprogramme(Chamberlinet al.,2002).

Traditionallyhigh-andlow-performingstudentscouldbeworseoffinanincentive-payprogrammeexcessivelyfocusedonoutputmeasures.Teachersmayfocusadisproportionateamountoftheireffortonthestudentswho



80

© OECD 2009

aremostlikelytomaximisetheteacher’schancesofearninganaward.Inthe1970s,forexample,performancecontractingexperimentsintheU.S.werefoundtobeassociatedwithteachersfocusingexcessivelyonstudentsinthemiddleofthedistributiontothedetrimentoftheirhigher-andlower-performingpeers(GramlichandKoshel,1975;Hannaway,1996;Rapple,1990).Morerecently,studieshaveexaminedasimilarresponsetothe 2002 No Child Left Behind (NCLB)Act (Ballou and Springer, 2009; Booher-Jennings, 2005; Neal andSchanzenbach,forthcoming;Reback,2008;Springer,2008).

Numerousotheraccounts fromoutside theU.S.havedocumented similarpatternsof teacherbehaviour inresponseto incentive-payprogrammes.Forexample, in1861, theU.K.’sNewCastleCommissionreformedtheteachercompensationsystemasameans“toinstituteasearchingexamination…ofeverychildineveryschool…andtomaketheprospectsandpositionoftheteacherdependent,toaconsiderableextent,ontheresultsoftheexamination,”(BourneandMacArthur,1970,p.20,ascitedinChamberlinet al.,2002).However,theincentive-payprogrammegroundtoahaltsome30yearslater,burdenedbyalegacyinwhich“teacherstaught to the test, were confined to a narrow, boring curriculum, attempted to arrange the school intake,cheated,ignoredbrightchildrenanddrilledandbeatthesloweronesuntiltheycouldsatisfytheall-powerfulinspectors,”(Chamberlinet al.,2002,p.32).

Whileincentive-payprogrammesintheU.K.werelinkedwithteachersfocusingexcessivelyonthemetred,rewardedactivitysome150yearsago,amuchlongerandmorestoriedhistoryofdysfunctionalbehaviouralresponseshavealsobeendocumented.Implementedin606andlastinguntilabout1905,Imperialchina’sKejuexaminationsystemwaslargelyusedtoidentifythefutureemploymentandsocialstatusforeachgenerationof students that completed formal schooling. However, according to Suen andYu (2006), the Keju systemsufferedfromconstruct-irrelevanceandconstruct-underrepresentationwhichincludedrotememorization,anexcessivefocusontest-takingskills,andcheatingintheformofnepotism,bribery,communicatingwithoutsideconfederatesduringtheexamination,andstudentsandtheir familieshiringsubstitutes tocomplete thetest.Although theKeju system is certainly anextremeexampleofpotentialnegative responses to ahigh-stakestestingpolicy,thefindingsarerelevanttothecurrentdiscussioninthatmanyofthesenegativeconsequencescouldhavebeenreducedbybroadeningtheperformancemeasures.

AmorerecentexampleofmanipulationcomesfromtheInternationalChildSupportIncentiveProgram(ICSIP),a group incentive intervention that randomly assigned 100 schools in rural Kenya to either a treatment orcontrolcondition.AcomprehensiveevaluationcompletedbyGlewwe,IliasandKremer(2008)revealedthatthepracticesofteachersinthetreatmentgroupstayedrelativelythesame,apartfromofferingextratestpreparationsessionsoutsideofclasstime.Thepositiveincreaseinthetestscoresofstudentsenrolledintreatmentgroupclassroomsdidnottranslatetonon-testedsubjectsandwerenotsustainedinsubsequentyears.Ananalysisofitem-leveltestdatafurtherindicatedthattreatmentconditionstudentsweresignificantlylesslikelytoleaveatestquestionblank.14

Additionalproblemsarelikelytosurfacewhenstudentandteacherinterestsarenotwell-aligned.Whilemanystandardised tests,suchasupper-secondaryschoolexitexams,areassociatedwithhigh-stakes forstudents,there are many situations when exam performance has no bearing on the student (Bishop, 1996). In anincentiveprogrammedevelopedandimplementedbyIsrael’sMinistryofEducation,teachersweremeasuredbytheirstudents’performanceonexamsrequiredforacertificateofmatriculation,whichhaslargefinancialimplicationsforthestudents(Lavy,2004).Eventhoughteacherseligibleforincentivesledstudentstolargergains inachievement, thedifferencemaynothavegeneralised to situationswhere tests arehigh-stakes forstudentsinalternatingyears:ifasixthgradestudentisrequiredtopassinordertoadvancetothenextgradebutseventhgradescoresdonotcarrythesameweight,thentheseventhgradeteachergainsmayexhibitbiasduetodecreasedperformancestemmingfromlowerpersonalmotivation.



Evaluating teacherperformance solelyon standardised test scoresof studentspresents anon-trivial risk forteachers.Intheirseminalstudyofschoolaccountabilitymeasures,KaneandStaiger(2001)noted:

Theimprecisionoftestscoremeasuresarisesfromtwosources.Thefirstissamplingvariation,whichisaparticularlystrikingprobleminelementary[(i.e.primary)]schools.Withtheaverageelementaryschoolcontainingonlysixty-eightstudentspergradelevel,theamountofvariationstemmingfromtheidiosyncrasiesoftheparticularsampleofstudentsbeingtestedisoftenlargerelativetothetotalamountofvariationobservedbetweenschools.Thesecondarisesfromone-timefactorsthatarenotsensitivetothesizeofthesample;forexample,adogbarkingintheplaygroundonthedayofthetest,aseverefluseason,adisruptivestudentinaclass,orfavorablechemistrybetweenagroupofstudentsandtheirteacher.Bothsmallsamplesandotherone-timefactorscanaddconsiderablevolatilitytotestscoremeasures.

Volatilityandotherformsoferrorintheperformancemeasuremeansteacherearningsarelesscertainbecauseof factorsbeyonda teacher’scontrol.Recognizingmost teachersalariesaremorepredictable than inotherprofessions(Liang,1999),asthenumberoffactorsoutsidethecontrolofteachers’increases,notonlywillthestrengthoftheincentivebeweakenedbutitcouldalsoleadteacherstodemandhigherlevelsofcompensation.Thus,assummarisedinareviewoftheeconomiccomplexitiesofincentivereforms,Asch(2005)notedthat,“Theoptimummixofthebaseandtheriskcomponentdependsonthedegreeofriskaversion,earningvariability,andtheworker’scostofeffort,”(p.316).15

Practitionersandpolicymakersinterestedindesigningandimplementinganincentive-payprogrammeneedtoknowthateventhoughVAMhavegainedareputationforprovidingfaircomparisonsofteachers,thisdoesnotnecessarilymeantheycanvalidlysupportsuchsystems.16McCaffrey,HanandLockwood(2009)notedthatVAMresearch todatehas focusedon thestatisticalpropertiesof themeasures from theperspectiveofmethodological research rather than from theperspectiveofanalgorithm that translates rawadministrativedata on students and teachers into information for managing human capital. Furthermore, to illustrate themanycomplexitiesanddecisionsthatmustbemadewhendesigninganincentive-payprogramme,McCaffreyandcolleaguesalsodevelopedparametersforaseriesofhypotheticalincentive-paymodelsandthentesttheperformanceofseveralValueAddedmodeling(VAM)strategieswithinthiscontext.17

AsVAMbecomesincreasinglypopularintheeducationpolicyarena,itiscriticalthatschoolsystemsunderstandtheproblemsandlimitationsoftheirmanagementinformationsystemsinsofarastheyrelatetothecreationofVAM.Commonproblemsincludemismatchofteacherandstudentdata,multipleteacherswithresponsibilityforthesamestudent,unclearteamorgroupcomposition,alackofdataandcapacitytomeasureteachervalue-added,andbonusesawardedtothewrongpersonsordeniedtotherightpersons.Theseproblemswillbeanembarrassmenttothedistrictandmayjeopardisethecredibilityoftheprogramme.Stakeholdersshouldalsobearinmindthateventhoughtheuseofperformanceincentivesisontherise,theprivatesectorhasmovedawayfromthesolerelianceonquantitativemeasuresofindividualorgroupperformanceandbeguntodesignincentive-payprogrammesthatincorporatebothinput-andoutput-basedmeasuresofemployeeperformance(Rothstein,2009).

Finally,poorlydesignedincentive-payprogrammescancreateperverseincentiveswherebyteachersmoveawayfromlow-performingschoolsinordertomaximisetheirchancesofearningadditionalpay(Ladd,1999;Neal,2009),potentiallyexaggeratingthealreadyunequaldistributionofteachersacrosshistoricallyadvantagedanddisadvantagedcontexts.InastudyofNorth carolina’sschoolaccountabilitysystem,forexample,Clotfelter,Ladd andVigdor (2004) found the recruitment and retention of high-quality teachers even harder in low-performingschools.



82

© OECD 2009

multiple measures – linking educational inputs and outputs

Schoolingisoftendescribedasamulti-product,multi-purposeenterprise;theperformanceofaworkerandthemissionofaschoolcomprisemultipledimensions.Bothempiricalandtheoreticalliteratureonincentivepayclearlyindicatethatifthereisdisconnectbetweenanorganisation’smissionandtheactivitytowhichincentivesareattached,employeesmayshiftworktowardthemetred,rewardedactivity,andawayfromotherimportantactivities(Dixit,2002;Hannaway,1992;HolmstromandMilgrom,1991).Manyadvocatesarguethattheuseofmultiplemeasuresinanincentive-payprogrammemeanstheperformanceofschools,teamsofteachers,orindividualscanbemonitoredmoreeffectivelyatvariouspointsthroughouttheschoolyear,allofwhichcanreducethechancesforsystem-gaming.

Alargenumberofstudieshavedocumentedhownarrowly-focusedperformancemeasurescancreategreateropportunityinthelongrunforemployeecheatingandopportunisticbehaviour.Inanincentive-payprogrammethatrewardsteacherperformancesolelyonstudenttestscores,dysfunctionalbehaviourscantaketheformofchangingstudentresponsesonanswersheets,providingstudentswithcorrectanswersduringtheexamination,orobtainingcopiesoftestmaterialpriortoadministrationinordertopreparestudents(JacobandLevitt,2003).18

Although linking educational inputs and outputs can reduce the likelihood of system-gaming, there canbe significant costs associated with an incentive-pay programme that evaluates teachers across multipleperformancemeasures.GuthrieandPrince(2009)reportedthatbothPhiladelphia(pennsylvania)andSteamboatSprings(colorado)cancelledincentive-payprogrammesafterlearningofthecostofimplementingtheteacherperformanceappraisalsystem.Similarly,severalschooldistrictsinFloridawereinterestedinparticipatinginthestatepay-for-performanceprogramme,butabandonedtheprogrammeafterlearningthecostofadministeringthebatteryofstudentassessmentsrequiredtoparticipate(GuthrieandPrince,2009).

Eachofthestakeholdersinvolvedinthedesignandimplementationofanincentive-payprogrammemayadvocateforintegratinguniqueperformancemeasuresthatsatisfytheirpersonalinterestorthatoftheirconstituents.Forexample,parentsexpectschoolsto“fostercreativity,curiosity,self-esteem,tolerance,goodcitizenship,athleticperformance,andahostofotherobjectives”(Hannaway,1996,p.103),whichmaycorrespondimperfectlytothegoalofschoolingasdefinedbyapolicymaker.Eventhoughadoptingmultiplemeasurescangoalongwaytowardcapturingthecomplexityoftheschoolingprocess,stakeholdersneedtobecognisantofthepotentialforconflictinggoalsengenderingthemisallocationofresources(Baker,1992,2002;BankerandDatar,1989;FelthamandXie,1994;HolmstromandMilgrom,1991).

Similartoahybridunitofaccountability,performanceevaluationsusingmultiplemeasuresrequirethatweightsbeassignedtoeachdimensionofthesystem,anaspectofincentive-payprogrammesthathastendedtobeignoredintheeducationsector-specificliterature.ProgrammeslikeNew York city’sSchool-WidePerformanceBonusProgramplacedthegreatestweightonstudenttestscores(30%studentachievementlevelsand55%studentachievementgains),whileMexico’s Carrera Magisterialadoptedasystemwhereby70%ofateacher’sevaluationreliedoninput-basedmeasuressuchasseniority,academicdegree,andprofessionalpreparation.Stakeholderswillcertainlyrequestanexplanationastowhyvariouscriteriaareweightedthewaytheyare,andtheseweights,forbetterorworse,cansendastrongsignaltostudents,parents,educators,andpolicymakersaboutthevalueprogrammedesignersplacedonspecificaspectsoftheschoolingprocess.

Whileperformancemeasurementsystemsthatincorporateamyriadofmetricsarelikelytoreduceunintendedconsequencesbyofferingamorebalancedevaluationsystem,atpresent,aconsiderablenumberof schoolsystemsdonothavetheorganisationalcapacitytoimplementaprogrammeincorporatingmultiplemeasuresof teacher, team,or schoolperformance. Even themostbasicof incentive-payprogrammes requires that asignificantamountofpersonneltimebeallocatedtomonitoringimplementation,respondingtoandresolving



inconsistenciesinthedesignoftheprogramme,andaddressingquestionsandcomplaintssubmittedbyschoolpersonnelandotherstakeholders.

In total, contemporary incentive-pay programmes have a lot of room for improvement when it comes toidentifyingtheoptimalmixofinput-andoutput-focusedperformancemeasuresandassigningweightstoeachoftheevaluationcriteriaadopted.Werecommendthatschoolsystemsneedtorelyonrigourous,formativeevaluationsoftheirincentiveprogrammesastrialanderrorwillneedtoinformprogrammedevelopmentanddesigninboththeshort-andlong-run(CourtyandMarschke,2003;PodgurskyandSpringer,2007).Moreover,rushingimplementationbeforeteachersandotherschoolpersonnelunderstandhowtheprogrammeworkswillinalllikelihoodsetthestageforfailure.

PerFormance standards and tHresHolds

Performance standards and thresholds determine the required level of performance for a school, team ofteachersorindividualteachertosecureareward.Thestandardsandthresholdsdefinedbytheincentive-payprogrammedictate thenumberofunits thatcanearnabonusaswellaswhatscaleorminimumstandardstheseunitsmustmeet.Ourdiscussionreviewsthreeformsofincentive-paymodels,includinglinearmodels,stepfunctions,andlimitedlinearmodels.Wealsodiscussthefinebalancethatmustbeestablishedbetweenperformancethresholdsandthenumberofunitsprojectedtoearnabonusaward.

linear models

Linearmodels,alsoknownascontinuousstandards,refertorewardsbasedontheaverageleveloforaveragegainin,desiredoutcome(Lazear,2003).Inschoolsettings,anexampleusingchangeintestscoresisoutlinedbyLazear(2003),wherebrepresentstherelationshipbetweenearningsandtestscores,andKisaconstantthatsetssalariesattheirappropriatelevel.

Teacher’ssalary=N(D Score)b − K

Linearmodelshave several attractive features. Linearmodels are sufficiently robust,donotdependon thetimingoftheagent’sinformation,anddonotrequireallavailableinformationtomakeabonusdetermination(HolmstromandMilgrom,1987).Anotherimportantfeatureisthatlinearmodelsvalueprogressatallpartsoftheachievementdistribution(Lazear,2003).Insteadoffocusingonstudentswhoareclosetoacertaincutoffpoint,alinearmodelallowsfortheaggregategrowthofallstudentstodeterminebonuseligibility.

Counteringargumentsinfavouroflinearmodels,Kole(1997)demonstratedthatanincentive-payprogrammeusing a standard broad categorization ignores many important aspects of compensation contracts and canunderstateincentives.Herfindingssuggestthatexogenousfactorsthatinfluenceperformancemaybiasresultsif they, for example, impacthigh-or low-achieving studentsdifferentially. Furthermore, linear systemsmaypresentmoreofalogisticalchallengeforsystemsduetogreatercomplexityofbonuscalculations.Increaseddifferentiationofsalaryamongemployeesmayrequirebonusdeterminationatanindividuallevel,arequirementthatcouldplaceundueburdenonhumanresourcespersonnel.

step functions

Stepfunctions,ornon-continuousmodels,distributerewardsbasedonmeetingaminimumstandardorthreshold.Whilemultiplestandardsorstepsarepossible,stepfunctionsestablishacutoffpointbelowwhicheithernobonusisawardedorpenaltiesareincurred.Stepfunctionsareoptimalwhentheoutputisverysensitivetotheeffortputforwardbytheunitofaccountabilityinthegeneralvicinityoftheperformancethreshold(Holmstrom,1979;Mirrlees,1999).Inotherwords,stepfunctionscreatestrongincentivesforschools,teamsofteachers,orindividualteachersfacingasignificantreductioninpayiftheyreducetheireffortevenmarginally.



84

© OECD 2009

Theliteratureonincentivecontractshasdemonstratedthatstepfunctionsaretooeasilymanipulated,especiallywhentheycoverafixedperiodoftime(Dixit,2002).Insuchacontext,stepfunctionsprovidecomparativelyweakincentivesforefforteitherafterthethresholdisreachedorafteritbecomesunattainable(Asch,1990;HolmstromandMilgrom,1987;Oyer,1998).Forexample,inarecentstudyofphysicianmedicalgroups,Mullen,FrankandRosenthal(2009)foundthatphysicians’groupswithbaselineperformanceatorabovetheperformancethresholdforreceiptofabonusimprovedtheleast,butgarneredthelargestshareofthebonuspayments.

limited linear models

Itispossibletocombineelementsofbothlinearandstepfunctionmodelsinanincentiveprogrammeusingalimitedlinearmodelframework.AsoutlinedbyMurphy(2001),Figure3.2displaysthetheoreticalrelationshipbetweenaperformancemeasure(x-axis)andtheannualbonus(y-axis)withinthecontextofalimitedlinearmodel.Thebaselineperformancethresholdestablishestheminimumpossiblelevelofperformanceassociatedwithabonusaward.Beyondthisperformancethresholdfloor,thereisapositivelinearrelationshipbetweenincreasingperformanceandanannualbonusaward.

AlimitedlinearmodelisfurthercharacterisedinFigure3.2bythefactthatatsomepointalongperformancecontinuumthemaximumsizeofabonusaward iscapped.Any increase in theperformanceof theunitofaccountabilitybeyondthispredeterminedleveldoesnotearnaschool,teamofteachers,orindividualteacheranymoremoney.Therangeofperformancerewardedunderanincentive-payprogrammeiscalledtheincentivezone,andmidwaybetweenthelower-andupper-performancethresholdsisapredictedlevelofperformanceandcorollarybonusthatcanbeusedinbudgetprojections.

Note: Adapted from Murphy (2001).

Pay/performancerelationship

Annual bonusIncentive zone

Bonus “Cap”

Target bonus

Performancethreshold

Performancestandard

Performance measure

Figure 3.2Performance standards and thresholds



Severalaspectsofalimitedlinearmodelareparticularlyattractive.Themodelallowsforimprovementintheperformanceof theunitofaccountability to increase thesizeof theannualbonusawardedatanypoint inthe incentivezone.The limited linearmodelalso restricts thebonus sizeof the lowestperformers, therebyallowingforthepossibilityoftheincentive-payprogrammetosendastrongsignaltothoseatthebottomoftheperformancecontinuum.Finally,thefinancialrisktypicallyimposedbyextremelyhighperformanceisreducedbythebonuscap.

threshold levels

Anotherconsiderationforthresholdsisthetargetednumberofschools,teamsofteachers,orindividualsthatdesignersofanincentive-payprogrammeexpecttoobtainabonusaward.Inherentinthisdecisionisthebalancebetweenattainabilityandexclusivity,whichnotonlyplaysavitalroleinelicitingbehaviouralresponses,butalsodictateswhetheranincentive-payprogrammewillrealiseitsfullpotential.

RewardprogrammesinBoliviaandenglandprovideexamplesateachextreme.Ontheonehand,inBolivia’smeritwageprogramme,only2%ofteachersperformedsufficientlyoncontentknowledgeexamstoobtainarewardduringthefirstyearoftheprogramme.Whilepublicoutcrysubsequentlyledtoslightlymorethan25%ofteachersearningarewardinthesecondyearofthemeritwageprogramme,continualresistanceonthepartofteachersledtodissolutionoftheprogramme(MizalaandRomaguera,2004).

In england’s PerformanceThreshold System, on the other hand, about 88% of those teachers eligible toparticipateintheincentive-payprogrammeelectedtodoso.Amongparticipatingteachers,anastounding97%metthepredeterminedstandardestablishedforearningabonusaward.Ultimately,thelowlevelofperceivedvalueandworthofattainingaperformanceawardwasattributedwiththeweakmotivationalinfluenceelicitedbytheSystem(Atkinsonet al.,2009).

sIze and dIstrIbutIon oF bonus awards

Thesizeofbonus,orpayoutlevel,referstotheamountofthetotalbonusawardaschool,teamofteachers,orindividualcanearn.Distributionrelatestotheshareofteachersthatreceivedabonusawardandhowbonusesvaryamongemployees.Whilenoclearguidanceexistsontheoptimalsizeofabonusinateacherincentive-payprogramme,severalstudiessuggestedthesizeofbonusawardsforteachershavebeensosmallthatthemotivationalvalueofmost incentive systemshavebeencompromised (Chamberlin,et al.,2002;Heinrich,2007;Malen1999;TaylorandSpringer,2009).

Inthissection,wefirstdiscussvarioustopicsfoundinthetheoreticalandempiricalliteratureinanefforttoofferabroadperspectiveonthesizeofbonusesanddistributionofbonusawards.Wethenprovidesomegeneralinformationonnon-monetary incentives,whichcantake the formofpromotionandadvancement,positivefeedback, public or private recognition, decreased regulations or requirements, increased status, greaterdecision-makingcapacity,accesstoin-houseeducationorformaltraining,andsoon.Whilethemechanismmaydifferfrommonetaryrewards,theunderlyingprincipleofinfluencingbehaviourremainsconstant.

size of bonus

Table3.2displayssummarystatisticsonthesizeofbonusawardsforanumberofincentive-payprogrammes.19 IntheU.S.,thesizeofthebonusawardattheprogrammelevelrangesfromalowof0.4%toahighof365%ofateacher’smonthlysalary.BothoftheseestimatescomefromthetexasEducatorExcellenceGrantprogramme.Estimatesfrominternationalincentive-payprogrammesaredisplayedinthebottomhalfofTable3.2,wherethe smallest bonus award was associated with chile’s Sistema Nacional de Evaluación de Desempeño de los Establecimientos Educacionales (SNED)programme.Mexico’s Carrera Magesterialprogrammegenerallyawardedthelargestbonuses,whichinsomecasesexceeded200%ofateacher’smonthlysalary.



86

© OECD 2009

Table 3.2Performance measures in incentive pay programmes

period

performance Measures

educational Inputs and processes educational Outputs

United StatesSchoolIncentiveProgram(Dallas,Texas) 1992-1995 Studentattendance Studentachievementlevels;Studenttest

scoregains;Studentretentionrates

MeritPayProgram(StateofMichigan) 1996-1997 Studentevaluationofteacher Studentretentionrates

AchievementChallengePilotProject (LittleRock,Arkansas)

2005-2007 … Studenttestscoregains

TeacherAdvancementProgram(UnitedStates) 1999-present Professionaldevelopment;Careerladder(threelevels);Supervisorevaluation

Studenttestscoregains (group-andschool-level)

GovernorEducatorExcellenceGrantProgram (StateofTexas)

2005-2008 Studentattendance;Teacherattendance Studentachievementlevels; Studenttestscoregains

TexasEducatorExcellenceGrantProgram (StateofTexas)

2007-present Studentattendance;Teacherattendance Studentachievementlevels; Studenttestscoregains

ProfessionalCompensationSystemforTeachers(Denver,Colorado)

2005-present Knowledgeandskills;Professionalevaluation;High-needschools;Hard-to-staffsubjects

Studenttestscoregains;Schoolperformance

ASPIREProgram(Houston,Texas) 2005-present Teacherattendance Studenttestscoregains (group-andschool-level)

TeamIncentiveExperiment(RoundRock,Texas) 2008-present … Studenttestscoregains

ProjectonIncentivesinTeachingExperiment(Nashville,Tennessee)

2006-2009 … Studenttestscoregains

Q-Comp(StateofMinnesota) 2006-present Carreeradvancement;Professionaldevelopment;Professionalevaluation;

High-needschools;Hard-to-staffsubjects

Studentachievementlevels; Studenttestscoregains

School-WidePerformanceBonusProgram (NewYorkCity,NewYork)

2008-present Studentattendance;Student,parentandteacherperceptionofschoollearning

environment

Studentachievementlevels; Studenttestscoregains

InternationalSchoolPerformanceProgram(Israel) 1996-1997 … Numberofcreditunitsearnedper

student;Studentreceivingmatriculationcertification;Schooldropoutrate

InternationalChristelijkSteuenfondsIncentiveProgram(BusoandTeso,Kenya)

1998-1999 … Studenttestscoregains;Studentachievementlevels

Teacher-IncentiveExperiment(Israel) 2001 … Studentachievementlevels

RandomizedEvaluationProject(AndhraPradesh,India)

2006-present … Studenttestscoregains

Carrera Magisterial(Mexico) 1993-present Seniority;Academicdegree; Professionallevel;Professionalpreparation;

Professionalperformance

Studentachievementlevels

Salario al Mérito(Bolivia) 1998-1999 Teachercontentknowledge …

Incentivo Colectivo a Escuelas(Bolivia) 2001-present Schoolorganization;Teachertrainingandretention;Regularityinschoolmanagement;

Numberofstudentsperclass; Student-teacherratio;Teachinginitiatives

developedbyteachers;Parentparticipation

Studentpassrates;Studentdrop-outrates;Studentrepetitionrates

PerformanceThresholdSystem(England) 1999-present Supervisorevaluation Studenttestscoregains

Sistema Nacional de Evaluación de Desempeño de los Establecimientos Educacionales(Chile)

1996-present Schoolinitiative; Improvedworkingconditions

Studenttestscoregains;Studentachievementlevels;Studentpassrates;

Studentretentionrates

Plan de Estímulos a la Labor Educativa Institutional (ElSalvador)

2000-present Schoolmanagement;Educationalmanagement;Institutionalplanning;

Teachermanagement.

Studentdrop-out;Studentretentionrates

PerformancePayProgram(Portugal) 2007-present Teacherattendance;Attendanceattrainingsessions;Managementandpedagogicalduties;Involvementinresearchproects;

Parentfeedbackaboutteacher

Studenttestscores

Sources:Atkinsonetal.(2009);ClotfelterandLadd(1996);Eberts,Hollenbeck,andStone(2002);Glewwe,Holla,andKremer(2009);Glewwe,Ilias,andKremer(2008);Ladd(1999);Lavy(2002,2004);McEwanandSantibanez(2005);MizalaandRomaguera(2004);MuralidharanandSundararaman(2008);PodgurskyandSpringer(2007);Rodriguez(2002);Santibanezetal.(2007);SchacterandThum(2004);Springer,Ballou,andPeng(2008);Springeretal.(2008,2009);SpringerandWinters(2009);Taylor,Springer,andEhlert(2009);VegasandUmansky(2005);Winters,Ritter,Greene,andMarsh(2009).



Table 3.2 presents information that suggests the magnitude of the maximum bonus award has increasednoticeablyover time. Lower-secondary schoolmathematics teachersparticipating in apay-for-performanceexperimentinNashville,tennessee,conductedbytheNationalCenteronPerformanceIncentives,setbonusawardsas largeasanadditionalUSD15000peryear,or theequivalentof280%of theaveragemonthlysalary.TheNationalCenteronPerformanceIncentivesisalsoevaluatingateacherincentiveexperimentthatawardsthetopone-thirdoftreatmentteamswithaUSD6000bonusperteacher.Casualempiricismsuggeststhemagnitudeof themaximumbonus increasedover timenotonly in response topast reformefforts thatprovidedunappealingawardamounts,butalsotoincreasedinterestinlearningwhetherteachersrespondedtosubstantialbonusawards,eveniftheincentivesystemlackedacompletearrayofmeasures.

Figure3.3displayshowoftenteachersparticipatingintheGovernorEducatorExcellenceGrant(GEEG)programmereceivedabonusaward,bybonusawardamount(Springeret al.,2008).20BonusawardsrangedfromalowofUSD20toahighofUSD20462,withmostteachersreceivingbetweenUSD1000andUSD3000.Whatismoststrikingabouttheinformationisthatnearly90%oftheteachersthatreceivedabonusawardearnedlessthanUSD3000,despiteprogrammeguidelinesthatencouragedteachersandschoolstoofferbonusesbetweenUSD3000andUSD10000(Springeret al.,2008).ThepatterninFigure3.3suggestsmostschoolsapprovedincentive-payprogrammeswitharelativelylargenumberofsmallbonusawards,which,accordingtoTaylorandSpringer(2009),didnotappeartohaveinducedanysignificantchangesinteacherproductivity.

4000

3500

3000

2500

2000

1500

1000

500

0

Number of teachers

0 9 000 1 200 15 000 18 000 21 000 24 000Actual bonus award amount (USD)

3 000 6 000

Source: Adapted from Springer et al. (2008).

Figure 3.3Distribution of actual awards in the Governor Educator Excellence Grant programme

distribution of award

Bonusawarddistributionsystemsdeterminehowevenlyanincentive-payprogrammedistributesrewardstoeligibleemployees.Anegalitariandistributionplandistributesincentivemoneywidely,incontrasttoplansthatrewardlargersumsofmoneytofewerschools,teamsofteachers,orindividuals.Thereisnoclearguidance,however,whetheran incentive-payprogrammeshould rewarda largenumberof relativelysmallawards to



88

© OECD 2009

teachersinaschool,orrewardasmallernumberofteacherswitharelativelylargeaward(Taylor,SpringerandEhlert,2009).Ourdiscussionregardingthedistributionofbonusawardsthereforedrawsheavilyonthegeneralpersonnelincentiveliterature.

Proponentsarguethatindividualistrewardplanshelpcreateameritocracyabletoretainanorganisation’shighestperformers,attractsimilartalentoverthelongrun,sendaclearsignaltothelowestperformerstoimproveormoveelsewhere,andaremorecost-effective(EhrenbergandSmith,1994;MilgromandRoberts,1992;PfefferandLangston,1993;Zenger,1992).Atthesametime,agrowingbodyofresearchsuggestsegalitariandistributionspromotecooperationandgroupperformance,whicharecriticalinparticipativeorganisations.MilgromandRoberts(1992)suggested,moreover,thatgreaterpaydispersionmayelevatetheperformanceofthelowestperformers.

Ina thoroughreviewof thecharacteristicsof incentive-payprogrammesthathavebeenadoptedinBolivia,chile,el Salvador,andMexico,MizalaandRomaguera(2004)reportedthatBolivia’s Incentivo Colectivo a Escuelas(ICE)awardedapproximately5%ofeligibleteachersabonusrangingbetween5%and19%ofannualincome.Chile’sSNEDdistributedamodestlylargernumberofbonusawardsbuttheamountofthoseawardswasslightlysmallerinsize,whileMexico’sCMawardedthelargestsums,withbonusesrangingbetween27%and224%ofannualincome(MizalaandRomaguera,2004).IncomparisontotheICEandSNEDprogrammesofferingbonusestoabout5%and27%ofeligibleteachers,respectively,theCMincentiveprogrammeprovidedbonusawardstoapproximately75%ofeligibleprimaryschoolteachers(MizalaandRomaguera,2004).

AstudyoftheGovernor’sEducatorExcellenceGrantprogrammeintexasexaminedtheproposeddistributionofawardstoschoolpersonnelwithinschoolsparticipatingintheprogramme.Taylor,SpringerandEhlert(2009)foundthat,onaverage,theproposeddistributionofawardswasmoreequalthanthedistributionofdisposableincome in the U.S., but less equal than the distribution of teacher salaries in the schools.Acknowledgingthe fact that virtually all schools participating in the GEEG programme adopted highly egalitarian awarddistributionschemes,aswellastheprogrammenothavinganydiscernibleassociationwithteacherproductivity,asubsequentstudyoftheprogrammesuggestedthatwhengiventheopportunity,teachersappeartodesignrelativelyweakincentivesystems(TaylorandSpringer,2009).

Stakeholders interested indesigningan incentive-payprogrammeneed toconsider the typeof educationalinstitutionwhenestablishingthesizeanddistributionofbonusawards.Forexample,inastudyoftherelationshipbetween wage dispersion and productivity in institutions of higher education, Pfeffer and Langston (1993)foundlessegalitariansalarystructuresdecreasedproductivityandincreaseddissatisfactionamongfaculty.Yet,thenegativeeffectswerenotasapparentinprivatecollegesanduniversities,whichtheauthorsattributedtothefactthatemployeepayinmostprivateinstitutionsisnotnecessarilyamatterofpublicrecord.

Newspapers,blogs,andothermediaoutletscanobtaininformationonthedistributionofbonusawardsthroughthe Freedom of InformationAct (FOIA) in the U.S. Several newspaper outlets have obtained teacher levelbonusinformationandthencreatedvenuesforthegeneralpublictoaccessandsearchthisinformationviatheInternet.21Thewayinwhichinformationaboutwagesiscommunicatedtoemployeescangreatlyaffecttheirattitudetowardtheprogrammeandbehaviour(GerhartandMilkovich,1990).Werecommendstakeholdersnotonlycommunicate regularlywith themedia,butalsoengageandworkwith teachersandotherschoolpersonnelsotheyarewellinformedofthepotentialforpublicationoftheirbonusawardamount.22

non-monetary incentives

While this chapter has focused primarily on programmes involving monetary incentives, non-monetaryincentivescanalsofunctionasworkplacemotivators.Manyschoolsandschoolsystemsnominateandrecognisecertainteacherswhoperformaboveexpectations,orofferteachersimprovedworkingconditions,unpaidleave,



mentoringandinductionprogrammes,andjobenlargement.Infact,inareviewofreformstrategiesadoptedinresponsetotheproblemofruralteacherrecruitmentamongmorethan20developingcountries,McEwan(1999) identified special training programmes, reduction in seniority requirements before promotion, andlongervacationsasthemostprevalentformsofnon-monetaryincentives.

Stigmaandpenaltieshavealsobeenidentifiedaspotentiallyhigh-powerednon-monetaryincentives.SeveralstatesinGermanydevelopedareportingprogrammethatpublicisedstudentlearninggainsassociationwithschools, which Juerges, Richter and Schneider (2004) reported was a positive influence on the quality ofteachingandstudentoutcomesbecauseteacherswantedtopreserveandprotecttheirreputation.StigmaandthreatsofsanctionhavealsobeenfoundtohavemostlypositiveeffectsinFlorida(Chiang,forthcoming;FiglioandRouse,2006;WestandPeterson,2006).Atthesametime,whilesomestakeholdersarguedthatinformationprovidedbytheGermanreportingsystemshouldbeusedtoinformpersonneldecisions(i.e.hiring,firing,andpromotion/tenure),twonationalstrikesorganisedbytheteacherunionresultedinpoliticalleadersinGermanycommittingtonotusetheresultsinthisway(Juerges,RichterandSchneider,2004).

Payout Frequency

Payoutfrequencyreferstotherateofawarddistributionaswellasthetimeintervalbetweenassessmentoftheincentivisedactivityanddistributionoftheperformanceaward.Theliteratureinpsychologysuggestsincentivesaremosteffectivewhentheunitunderobservationisawardedconsistentlywithminimumtimebetweenactionandreward(Skinner,1981).Whilesomedelayingratificationispossible,immediacyminimisesthedifficultyinassociationbetweenanindividual’sbehaviourandreward.Further,expectancytheorynotesmorefrequentdistributionandminimaltemporaldelaysolidifyconnectionsbetweenoutcomeandreward(Vroom,1964).Thedelaythatnowoccursbetweenteacherperformanceandpaymentofpremiumislongerthananybehaviouralpsychologistwouldsuggestforpurposesofreinforcinggoodactions.

Mostincentive-payprogrammesintheeducationsectordistributeawardsonanannualbasis,correspondingtotheacademicyear,oftenduetothetimingofresultsfromend-of-yearassessments.Whilepractical,minimizingthetimeintervalbytrackingprogressthroughouttheyearmaypromoteamoretransparentlinkbetweenactionandreward.Further,assessingandrewardingbehavioursmore frequentlyduring theschoolyearwillallowforconsistentreinforcementandlikelyresultinamorepronouncedinfluenceonbehaviour(HollensbeandGuthrie,2000).Thisconnectionispresentinthebusinesssector;researchconductedbytheConsortiumforAlternativeRewardStrategiesfoundcompensationplanswithgreaterpayoutfrequencywerelinkedtobusinessperformanceimprovements(McAdamsandHawk,1992).

Eberts,HollenbeckandStone(2002)studiedtheeffectofanincentiveschemeinanalternativeupper-secondaryschoolinMichiganthatrewardedteachersatmultiplepointsthroughouttheschoolyear.Theprogrammewasdesignedtoaddressagrowingdropoutrateproblem,andintroducedabonussystemthatpaidteacherstoraisecoursecompletionratesofstudentsatriskofdroppingout.Althoughtheprogrammereliedonaratherlimitedsetofperformanceindicators,theinterventionteacherswerepaida12%bonuseachquarterifatleast80%ofstudentsfromthepreviousquarterwereretained.

Increasing the payout frequency may also decrease the likelihood of system gaming since behaviours areassessedonamoreregularbasis.AsfoundinastudyofNavyrecruiters,forexample,whentheirperformancewasevaluatedat12-monthintervals,therecruiters’productivitysignificantlyincreasedastheevaluationdateapproached.Thisspikeinperformancewascharacterisedbyasizeablereductioninoutputshortlyaftertheperformanceevaluation.Similarly,CourtyandMarschke(1997)reportedthat trainingcentersunder the JobTrainingPartnershipActfocusedeffortsonmorequalifiedparticipantsenteringtheprogramme,andthetimingoftheirgraduation,tomeetcertainstandards.



90

© OECD 2009

cultural consIderatIons

The impact of an incentive-pay system may reflect characteristics inherent in cultural contexts. Culturaldimensions,asdescribedinHofstede(1980,2001),providecategoriesinwhichcountriescanbecomparedregardingrewardpreferences,includingpowerdistance,collectivismversusindividualism,femininityversusmasculinity,anduncertaintyavoidance.Powerdistancereferstohowpeoplefeelpowershouldbedistributedaswellashow thosewithpower shouldbeviewed:ahighpower-distanceenvironmentwould saypowershouldbelimitedtofewerindividualswhosedecisionsshouldbeaccepted,whilelowerpowerdistancewouldhavemorepeople incontrolandwouldallowthechallengingof leaders.Acollectiveculturewouldplacetheallegianceto thegroupabovethatof theindividualwiththisbeingreversedinanindividualistculture.A feminine culture would value caring for others and establishing social support while a more masculinesocietywouldplaceahighervalueonachievement,theaccumulationofwealth,andassertiveness.Uncertaintyavoidancereferstotheextentacultureavoidssituationsthatareambiguousorhaveunpredictableoutcomes.

Reward preferences canbedirectly linked to the chanceof an incentive-pay programmeactuallymeetingindividualneedsandculturalvaluesfoundwithinparticularcountries.IntheircasestudyoffourcountriesthatvariedonthedimensionsestablishedbyHofstede(1980,2001),ChiangandBirtch(2005)foundthatcountrieswithhighmasculinityscoresweremorelikelytovaluematerialrewardsascomparedtothosecountrieswithmore feminine scores. They found a similar preference for financial rewards between individualistic andcollectivecultures,whileresponsestoincentive-payprogrammessignificantlydivergedaccordingtoculturalfactors.EventhoughtheevidencereportedbyChiangandBirtch(2005)isbasedonobservationsfromalimitednumberofcountries,theirfindingssupporttheargumentthatculturalinfluencesareanimportantaspectforeducationpractitioners andpolicymakers to considerwhendesigning and implementing an incentive-payprogramme.23

Furthermore,inTable3.3asdefinedbyHofstede(1980,2001),weillustratethepowerdimensionscoresinseveralcountrieswithincentiveprogrammestofurtherassesstheuniquenatureoftheculturalcontextofeachincentive-payprogramme.Take,forexample,theprogrammedesignedandimplementedbyIsrael’s MinistryofEducation,whichwasconsideredhighlycompetitivebecausetheincentivestructurereliedonarankordertournamentstructure;thatis,therewerealimitednumberofpeoplewhocouldearntherewardandteacherseligibleforabonusawardwereknowinglyindirectcompetitionwithoneanother.Aplausibleexplanationfortheacceptanceoftheprogrammebyschoolpersonnel,whilesimilarincentiveprogrammedesignshavetypicallyfailedinothercontexts,isthatIsraeliculturehasacomparativelyhighindividualscore.Whilemoreresearch is definitely needed to determine specific relationships between design features of incentive-payprogrammesandtheculturaldimensionsdisplayedinTable3.3,itisimportanttoconsidertheculturalcontextwhendesigninganincentive-payprogramme.

Table 3.3Cultural dimensions for selected countries with incentive pay programmes

power Distance Individualism Masculinity Uncertainty avoidance

India 77.0 40 48 56

Israel 13.0 81 54 47

England(GreatBritain) 35.0 35 89 66

Kenya(EasternAfrica) 64.0 52 27 41

Chile 63.0 23 28 86

Mexico 81.0 30 69 82

World average 56.5 43 51 65

Note:AdaptedfromHofstede(1980,2001).



conclusIon

Inthischapterwereviewedandsynthesisedmanyofthetheoreticalandempiricalargumentsconnectedwithvariousdesignelementsfoundinincentive-payprogrammes.Wefocusedonspecificdesignelementsincludingtheincentivestructure,unitofaccountability,performancemeasures,performancetargetsandthresholds,sizeanddistributionofbonusawards,andpayoutfrequency.Wealsodiscussedthepotentialrelationshipbetweenculturaldimensionsandthedesignofincentive-payprogrammes.

Incentivestructurescanimpactthenumberofteacherslikelytoreceiveabonusawardaswellastherequiredfinancialcommitmentneededtoimplementandsustaintheincentive-payprogrammeovertime.Rankorderedtournamentsclearlylimitthefinancialexposureofafixedperformancecontract,whichisadvantageousconsideringthevolatilityandnoiseinherentinmanyperformancemeasuresfoundintheeducationsector.Schoolpersonnelandorganisationshavehistorically resisted tournament incentivestructures,however,because theycan fostercompetitionandotherconditionsthatareincompatiblewiththenormsoftheteachingprofession.

Incentive-payprogrammeshaveoverwhelminglyconceptualisedtheunitofaccountabilityasbeingeitherattheindividual-orschool-level,oracombinationofthetwo.Yet, thisconceptualizationmissesapotentiallyimportantunitfoundwithinschools:grade-levelordepartmentalteamsofteachersoranyotherwithin-schoolunit.Educationstakeholdersshouldconsidermorecloselywithin-schoolteamsastheunitofaccountability,whichseveraltheoreticalandempiricalstudiesfromoutsidetheeducationsectorindicatedhavethepotentialtocombinethecohesivebenefitsofgroupaccountabilitywiththeindividualbeliefthatproductivitydirectlyimpactsbonuseligibility.Theeducationsystemshouldalsocontinuetoexploremorecomplexhybridmodelsthatmaybemoresuitableinmulti-product,multi-purposeenvironments.

Carefulconsiderationneedstobegiventohowtheperformanceofschools,teamsofteachers,orindividualsismeasured.Alargeliteratureindicatesemployeebehaviourwilltendtoshiftinthedirectionofincentivisedactions, and both potential positive and negative outcomes should be anticipated. Multiple measures canoftenminimisegamingbehaviourandprovideamorevalid,reliable,fairandcomprehensiveassessmentofteacher,teamandschooleffectiveness,allofwhicharecrucialtothemotivationaleffectoftheincentive-payprogramme.Furthermore, therearereportsofmoreandmoreprivatesectorfirmsadoptingboth input-andoutput-basedmeasureswhenassessingemployeeperformance.

In regards toperformancestandardsand thresholds, incentive-payprogrammeshaveusuallyadoptedeitheralinearorstepfunctionmodel.Mostresearchershaveadvocatedforlinearmodelsduetodesignsimplicityandtheirrobustnature,eventhoughsomeofthebehaviouraleconomicliteraturecautionsthatalinearmodelignoresimportantinformationabouttheactualproductionprocess.Giventherelativelysparseliteratureonperformancestandardsandthresholds,itisimportantforfutureevaluationresearchanddevelopmentworktocloselyexaminethesestructuresinanefforttofurtherinformstakeholdersoftherelativemeritsandweaknessesofeachsystem.

The size and distribution of incentive-pay awards have tended to vary dramatically among programmes,althoughafewrelativelyrecentstudieshavesuggestedthatteachersmaybepredisposedtoadoptinghighlyegalitarianawarddistributionschemes.Whendesigninganincentive-payprogramme,itiscriticallyimportanttokeepinmindthatbonusawardsneedtobesufficientlylargetoelicitbehaviouralresponsesandnotsolargethatgamingorcheatingbecomesaconcern.We further recommend that incentive-payprogrammes in theeducationsectorincreasethefrequencyinwhichperformanceisevaluatedandawardsaredistributed.Tothepointthattheyarefeasible,multipleevaluationsandperformanceobjectivesorbenchmarksduringtheschoolyearstrengthentheconnectionbetweenbehaviourandreward.Sucharrangementsmayhelpteachers,teamsofteachers,orschoolstomonitorandalterbehaviouratseveralpointsintimeinsteadofbasingtheirperformancebonusonaone-timeannualassessment.



92

© OECD 2009

Cultural norms and preferences toward design elements of incentive-pay programmes are likely importantconsiderations during the design and implementation phase. While a rank-order tournament evaluatingindividualteacherperformancemayhaveelicitedsupportandhadapositiveimpactonstudentachievementandteacherbehaviourinIsrael,countriespredisposedtowardmorecollectiveculturalprocessesandnormsmightbemoresuccessfuladoptingagroup-levelorhybridmodelusingafixedperformancecontractincentivesystem.Evenwithinacountry,regionsmayalsodifferinsocio-culturalcharacteristicsaswellastheeconomicsituationsthatinfluencetheidealdesignofincentivesystems.

It is clear the literature on the design components of incentive-pay programmes in the education sectorstill needs tobedevelopedbeforewecandefinehow thefirstor second-best system shouldbedesigned.Educational research, practice, and policy needs to continue to support policy innovation combined withrigourous,independentevaluationsofboththeshort-andlong-runimpactsonstudentachievement,teacherattitudesandbehaviour,andorganisationaldynamics.

Notes

1. Incentive-pay programmes have grown in popularity. High-profile programmes have included Bolivia’s MeritWages and ICEprogram(VegasandUmansky,2005;MizalaandRomaguera,2004);chile’s Sistema Nacional de Evaluación del Desempeño de los Establecimientos Educacionales(SNED)(MizalaandRomaguera,2003);el Salvador’sschoolawardsprogram(Rodriguez,2002);Mexico’s Carrera Magisterial(McEwanandSantibanez,2005;Santibanezet al.,2007);New York city’sSchool-WidePerformanceBonusProgram(SpringerandWinters,2009);texas’ Governor’sEducatorExcellenceAwardPrograms(Springeret al.,2008;Springeret al.,2009);theU.S. TeacherIncentiveFund(PodgurskyandSpringer,2007);programmesdevelopedbyIsrael’sMinistryofEducation(Lavy,2002;2007);andexperimentsinAndhraPradesh,India(MuralidharanandSundararaman,2008)andintheBusiaandTesodistricts of western Kenya (Glewwe, Ilias, and Kremer, 2008). In the U.S., pay-for-performance experiments have recently beenimplementedandarecurrentlybeingevaluatedinNashville,tN,NewYorkCity,NY,andRoundRock,tX.

2.Inresponsetotheadoptionofportugal’sperformance-paypolicy,thereweretwonationalstrikesledbyteachersandtheirunions(Martins,2009).

3.Whileatfirstsightthereappearstobeasignificantincreaseinstudentlearning,thepercentageofschoolsmeetingtheproficiencystandardmaynotinfactrepresentatrueincreaseinstudentachievement.Thereisahealthyliteratureonweaknessesofhigh-stakesassessments and test-based accountability systems, including test scores becoming artificially inflated because a state may haveadoptedanewstandardisedassessment,recalibratedproficiencystandards,orrepeatedlyadministeredasimilartestformfromoneschoolyeartothenext(see,forexample,Linn,2000;Koretz,2003;2008).Indescribingwhatiswidely-knownasthe“LakeWobegonEffect”[Thismayneedanexplanationfortheinternationalaudience](e.g.,scenariowhereschoolsorteachersreceiveoverlypositiveratings),Cannell(1987;ascitedinLinn,2000)arguedthat“standardized,nationallynormedachievementtestsgivechildren,parents,schoolsystems,legislatures,andthepressinflatedandmisleadingreportsonachievementlevels”(p.7).

4.IntheU.S.,ithasbeenarguedthattheincreasedcompressionofthefederalsector’scompensationsystemrelativetothatintheprivatesectorhasmadeitmoredifficultforthefederalsectortorecruitandretainhigh-qualityemployees(Gibbs,2001;Borjas,2002;Asch,2005).



5.Work being conducted by researchers at the National Center on Performance Incentives has begun to explore many of theoperationalcomplexitiesassociatedwithincentive-payprogrammesandtheeffectthesecomplexitieshaveonmeasuresofteacherandteamperformance.

6.Controlconditionreferstothoseclassroomsnotexposedtooneofthefourinterventions.

7.Analternatedrawisaselectionprocesswheretheleaderofthefirstteamselectsfromapoolofeligiblecandidatesandthentheleaderofthenextteamselectsfromtheremainingpoolandsoonuntilalleligiblecandidatesareselected.

8. Independent evaluations have been conducted by Springer, Ballou, and Peng (2008) and are currently being conducted byGlazerman,Allison,McKie,andCarey(2009).

9.LevinandMcEwan(2001)offeraverythoroughandpracticaloverviewofcost-effectivenessanalysiswhenevaluatingorconsideringeducationalprogrammesandpolicies.

10.TheCenterforEducatorCompensationReformoffersanumberofusefulresourcesregardingtheimplementationofincentive-payprogrammes.Formoreinformation,visithttp://www.cecr.ed.gov

11.Previousresearchtypicallycodessubjectiveevaluationsofteacherperformanceasaninputmeasure,outputmeasure,orprocessmeasure.Thesameholdstrueforstudentandteacherattendance.Inthisreview,weconsidertheseprocessmeasures.

12.Even thoughthevastmajorityofstates in theU.S.have fundedincentiveplansaroundteacherrecruitmentandretention,asobservedbyLoebandMiller (2007),mostof theseprogramsarenotwell-alignedwithteacher labourmarketrealities,nor is thereceiptofanincentiveawardusuallycontingentonteachereffectiveness.

13.Formoreinformationvisit,http://cpre.wceruw.org/index.php

14.Inhisreviewofthedifferenttypesofcorruptionandtheirpotentialcausesintheeducationsector,Heyneman(2004)recommendsfourtypesofreformtominimisetherisk.Theseincludeeducationalstructures,theprocessesofmanagementandadjudication,themechanismofpreventionwhenwrongdoingoccurs,andthesystemofsanctions.

15.Carnoy,Brodziak,Molina,andSolcias(2007)reportonconstraintsofaschool-basedincentiveprogrammewhenusingstudentperformancedatatorewardperformanceoninter-cohortgainsasopposedtointra-cohorttestscoregains.

16.ForamorecompletedescriptionofissuesrelatedtoVAMseeChapter5byDanGoldhaber.

17.Ballou(2009)discussedtheimplicationsofscalepropertiesinvalue-addedmeasurements.Heanalysedseveralpossiblemethodsofformulatingvalue-addedestimatesfromteststhatarescoredaccordingtoItemResponseTheory(IRT).HefoundmeasuringabilityonanintervalscaleusingIRTrequiresaconjointstructurebetweenstudentsandtestitems,conditionsthatarebothdifficulttomeetandverify.Value-addedassessmentsofteacherqualitymayintroducevolatilityduetothescalepropertiesofteststhatmayincorrectlybeattributedtoteachereffectiveness.

18.JacobandLevitt’s(2003)analysisofcheatingwithinthechicagoPublicSchoolsystemprovidedtwomethodsofdetection.Thefirstmethodinvolvedsearchingforlarge-scaleuniformpatternsinstudentresponses,patternswhichwouldbestatisticallyimpossibleinnormalstudentresponses.Secondly,unusuallyhighstudentperformancesinoneyearthatarefollowedbydeclinesinperformanceinsubsequentyearswereselectedforcloserobservation.IninternationalprogramsinIsrael,Kenya,andIndiatherewerepenaltiesforstudentswhodidnottaketheexaminanefforttopreventschoolsandteachersfromintentionallydiscouraginglow-performingstudentsfromtestparticipation.Ineachofthesecases,testparticipationrateseitherstayedthesameorsignificantlyincreased(Lavy,2004;Glewwe,Ilias,andKremer,2003;MuralidharanandSundararaman,2008).

19.Table3.2buildsonevidencereportedinTaylor,Springer,andEhlert(2009).

20.TheinformationdisplayedinFigure3.3doesnotincludethoseteachersthatparticipatedintheTEEGprogrambutdidnotearnabonusawardduringthe2006-07schoolyear.

21.ThehoustonChroniclepostsinformationonbonusawardamountsdelineatedbyemployeename,position,andschool(see,for example, http://www.chron.com/news/houston/bonuses).The austin Statesman similarly publishes names of teachers earningbonusesunderthedistrict’sStrategicCompensationInitiative.

22. Burns, Gardner, Muesswen (2009) provide additional discussion within the context of austin Independent School District’sStrategicCompensationInitiative.

23.Inarelatedstudy,RamamoorthyandCarroll(1998)reportedasignificantrelationshipbetweenhighlevelsofindividualismandalternativehumanresourcemanagementpracticessuchasmeritpay.

http://www.cecr.ed.gov

http://cpre.wceruw.org/index.php

http://www.chron.com/news/houston/bonuses


References

94

© OECD 2009


adnett, N. (2003),“ReformingTeachers’Pay:IncentivePayments,CollegiateEthosandUKPolicy”,Cambridge Journal of Economics,Vol.27,No.1,pp.145-147.

asch, B.(1990),NavyRecruiterProductivityandtheFreemanPlan,RAND Monograph R-3713-FMP,SantaMonica,CA.

asch, B.(2005),“TheEconomicComplexitiesofIncentiveReforms”,inR.KlitgaardandP.C.Light(eds.),High-Performing Government: Structure, Leadership, Incentives,RANDCorporation,SantaMonica,CA.

atkinson, a, S. Burgess, B. croxson, p. Gregg, c. propper, h. Slater andD. Wilson(2009),“EvaluatingtheImpactofPerformance-RelatedPayforTeachersinEngland”,Labour Economics, Vol.16,No.3,pp.251-261.

Baker, G. (1992),“IncentiveContractsandPerformanceMeasurement”,Journal of Political Economy,Vol.100,No.3,pp.598-614.

Baker, G. (2002),“DistortionandRiskinOptimalPerformanceContracts”,The Journal of Human Resources, Vol.37,No.4,pp.728-751.

Baker, G., M. Jensen andK. Murphy (1988),“Compensationand Incentives:Practivevs.Theory”, Journal of Finance,Vol.43,No.3,593-616.

Ballou, D.(2001),“PayforPerformanceinPublicandPrivateSchools”,Economics of Education Review,Vol.20,No.1,pp.51-61.

Ballou, D. andM. podgursky (1997),“ReformingTeacherTraining&Recruitment:ACriticalAppraisaloftheRecommendationsoftheNationalCommissiononTeachingandAmerica’sFuture”,Government Union Review,Vol.17,No.4,pp.1-53.

Ballou, D. andM. podgursky (2009),“ResponsestoNoChildLeftBehindAmongTraditionalPublicSchools”,Peabody College of Vanderbilt University Working Paper,Nashville,TN.

Banker, r. andS. Datar (1989), “Sensitivity,Precision,andLinearAggregationofSignals forPerformanceEvaluation”,Journal of Accounting Research,Vol.27,No.1,pp.21-39.

Baron, J. andD. Kreps (1999),Strategic Human Resources: Frameworks for General Managers,Wiley,NewYork.

Berg, p., e. appelbaum, t. Bailey and a. Kalleberg (1996), “The Performance Effects of Modular Production in theApparelIndustry”,Industrial Relations,Vol.35,No.3,pp.356-373.

Bishop, J. (1996). “Signaling, Incentives, and School Organization in France, the Netherlands, Britain, and the UnitedStates”, inE.HanushekandD. Jorgenson(eds.), Improving America’s Schools: The Role of Incentives,NationalResearchCouncil,Washington,D.C.

Bok, D.(1993),The Cost of Talent: How Executives and Professionals are Paid and How It Affects America,TheFreePress,NewYork.

Booher-Jennings, J. (2005), “Below the Bubble: ‘EducationalTriage’ and theTexas Accountability System”, American Educational Research Journal, Vol.42,No.2,pp.231-268.

Bourne, r. andB. Macarthur (1970),The Struggle for Education: 1870-1970,SchoolMastersPublishingCompany,London.

Bowles, S. andh. Gintis (2002), “Schooling inCapitalistAmericaRevisited”,Sociology of Education,Vol.75,No.1,pp.1-18.

Bryk, a. andB. Schneider (2002),Trust in Schools: A Core Resource for Improvement,RussellSageFoundation,NewYork.

Burgess, S., B. croxson, p. Greggandc. propper(2001),“TheIntricaciesoftheRelationshipBetweenPayandPerformanceforTeachers:DoTeachersRespondtoPerformanceRelatedPaySchemes”,CMPO Working Paper,Bristol.

95


chapter 3 referenCes

chamberlin, r., t. Wragg, G. haynes, andc. Wragg (2002),“Performance-RelatedPayandtheTeachingProfession:AReviewoftheLiterature”,Research Papers in Education,Vol.17,No.1,pp.31-49.

che, Y. andS. Yoo (2001),“OptimalIncentivesforTeams”,The American Economic Review,Vol.91,No.3,pp.525-541.

chiang, h.(forthcoming),“HowAccountabilityPressureofFailingSchoolsAffectsStudentAchievement”,Journal of Public Economics.

chiang, F. and t. Birtch (2005), “A Taxonomy of Reward Preferences: Examining Country Differences”, Journal of International Management,Vol.11,pp.357-375.

clotfelter, c., h. Ladd andJ. Vigdor (2004),“DoSchoolAccountabilitySystemsMakeItMoreDifficultforLow-performingSchoolstoAttractandRetainHigh-QualityTeachers?”,Journal of Policy Analysis and Management,Vol.23,No.2,pp.251-271.

clotfelter, c., h, Ladd andJ. Vigdor (2007),“AreTeacherAbsencesWorthWorryingAboutintheU.S.?”,NBER Working Paper 13648, NationalBureauofEconomicResearch,Cambridge,MA.

corcoran, S., W. evansandr. Schwab(2004),“Women,theLaborMarket,andtheDecliningRelativeQualityofTeachers”,Journal of Policy Analysis and Management,Vol.23,No.3,pp.449-470.

courty, p.andG. Marschke(1997),“MeasuringGovernmentPerformance:LessonsfromaFederalJob-TrainingProgram”,American Economic Review, Vol.87,No.2,pp.383-388.

courty, p. andG. Marschke (2003), “DynamicsofPerformance-MeasurementSystems”,Oxford Review of Economic Policy,Vol.19,pp.68-284.

courty, p., c. heinrich andG. Marschke(2005),“SettingtheStandardinPerformanceMeasurementSystems”,International Public Management Journal, Vol.8.No.3,pp.321-347.

crewson, e. (1997),“Public-ServiceMotivation:BuildingEmpiricalEvidenceof IncidenceandEffect”, Journal of Public Administration Research and Theory,Vol.7,No.4,pp.499-518.

cutler, t. andB. Waine (2004),“PerformanceWithoutPay?ManagingSchoolBudgetsUnderPerformanceRelatedPay:EvidenceFromtheFirstYearofPRP”,Financial Accountability and Management,Vol.20,No.1,pp.57-75.

Danielson, c.(1996),Enhancing Professional Practice: A Framework for Teaching.AssociationforSupervisionandCurriculumDevelopment,Alexandria,VA.

Deutsch, M. (1985),Distributive Justice: A Social-Psychological Perspective,YaleUniversityPress,NewHaven,CT.

Dixit, a.(2002),“IncentivesandOrganizationsinthePublicSector”,Journal of Human Resources,Vol.37,No.4,pp.696-727.

Duflo, e., r. hanna andS. ryan(2005),“MonitoringWorks:GettingTeacherstoCometoSchool”,NBER Working Paper 11880, NationalBureauofEconomicResearch,Cambridge,MA.

eberts, r.(2007),“TeacherUnionsandStudentPerformance:HelporHindrance?”,The Future of Children,Vol.17,No.1,pp.175-200.

eberts, r., K. hollenbeck and J. Stone (2002), “Teacher Performance Incentives and Student Outcomes”, Journal of Human Resources,Vol.37,No.4,913-927.

ehrenberg, r.andr. Smith (1994),Modern Labor Economics: Theory and Public Policy,FifthEdition,HarperCollins.

elmore, r.F., c.h. abelmann,andS.h. Fuhrman, (1996).“Thenewaccountabilityinstateeducationreform:Fromprocesstoperformance”.InH.Ladd(ed.),Holding schools accountable: Performance-based reform in education.(pp.65±98).Washington,DC:BrookingsInstitution.

Feltham, G. andJ. Xie (1994),“PerformanceMeasureCongruityandDiversityinMulti-TaskPrincipal/AgentRelations”,The Accounting ReviewVol.69,No.3,pp.429-453.

Figlio, D. andc. rouse (2007), “DoAccountabilityandVoucherThreats ImproveLow-PerformingSchools”, Journal of Public Economics,Vol.90,No.1-2,pp.239-255.

Firestone, W.(1994),“RedesigningTeacherSalarySystemsforEducationalReform”,American Educational Research Journal,Vol.31,No.3,pp.549-574.



96

© OECD 2009

Freeman, r. anda. Gelber (2006),“OptimalInequality/OptimalIncentive:EvidencefromaTournament”,NBER Working Paper 12588,NationalBureauofEconomicResearch,Cambridge,MA.

Freixas, X., r. Guesnerie andJ. tirole(1985),“PlanningunderIncompleteInformationandtheRatchetEffectî,Review of Economic Studies,Vol.52,No.2,pp.173-191.

Gaynor, M. andM. pauly(1990),“CompensationandProductiveEfficiencyinPartnerships:EvidencefromMedicalGroupPractice”,Journal of Political Economy,Vol.98,No.3,pp.543-577.

Gerhart, B.andG. Milkovich(1990),“OrganizationalDifferencesinManagerialCompensationandFinancialPerformance”,Academy of Managerial Journal,Vol.33,No.4,pp.663-691.

Ginther, D., r. haveman andB. Wolfe(2000),“NeighborhoodAttributesasDeterminantsofChildren’sOutcomes:HowRobustaretheRelationships”,The Journal of Human Resources,Vol.35,No.4,pp.603-642.

Glewwe, p., a. holla andM. Kremer (2009), “Teacher Incentives in the DevelopingWorld”, in M.G. Springer (ed.),Performance Incentives: Their Growing Impact on American K-12 Education,BrookingsInstitutionPress,Washington,D.C.

Glewwe, p., N. IliasandM. Kremer(2008),Teacher Incentives in the Developing World,HarvardUniversity,Cambridge,MA.

Goldhaber, D. (2002),“TheMysteryofGoodTeaching.”Education Next,Vol.2,No.1.

Goldhaber, D. (2009),“ThePoliticsofTeacherPayReform”,inSpringer,M.G.(ed.),Performance Incentives: Their Growing Impact on American K-12 Education,BrookingsInstitutionPress,Washington,D.C.

Goldhaber, D., D. Brewer andD. anderson(1999),“AThree-WayErrorComponentsAnalysisofEducationalProductivity”,Education Economics,Vol.7,No.3,pp.191-208.

Gonring, p., p. teske andB. Jupp (2007),Pay-for-Performance Teacher Compensation: An Inside View of Denver’s ProComp Plan,HarvardEducationPress,Cambridge,MA.

Gramlich, e. andp. Koshel (1975),Educational Performance Contracting, BrookingsInstitution,Washington,D.C.

Guthrie, J. andc. prince (2009), “Paying for and Sustaining aPerformance-BasedCompensation System”,Center forEducatorCompensationReform,Washington,D.C.

hallinger, p.andJ. Murphy (1986).“TheSocialContextofEffectiveSchools”,American Journal of Education,Vol.94,No.3,pp.328-355.

hamilton, L. (2005),“LessonsfromPerformanceMeasurementinEducation”,inR.KlitgaardandP.C.Light(eds.),High-Performing Government: Structure, Leadership, Incentives,RANDCorporation,SantaMonica,CA.

hamilton, B., J. Nickersonandh. Owan(2003),“TeamIncentivesandWorkerHeterogeneity:AnEmpiricalAnalysisoftheImpactofTeamsonProductivityandParticipation”,Journal of Political Economy,Vol.111,No.3,pp.465-497.

hannaway, J. (1992),“HigherOrderSkills,JobDesign,andIncentives:AnAnalysisandProposal”,American Educational Research Journal,Vol.29,No.1,pp.3-21.

hannaway, J. (1996). “ManagementDecentralizationandPerformance-Based Incentives:TheoreticalConsiderations forSchools”,inE.HanushekandD.Jorgenson(eds.),Improving America’s Schools: The Role of Incentives,NationalResearchCouncil,Washington,D.C.

hansen, D. (1997),“WorkerPerformanceandGroupIncentives:ACaseStudy,”Industrial Labor Relations Review,Vol.51,pp.37-49.

heinrich, c.J. (2007),“Evidence-BasedPolicyandPerformanceManagement:ChallengesandProspects inTwoParallelMovements”,The American Review of Public Administration,Vol.37,No.3,pp.255-277.

heneman, h., a. Milanowski and S. Kimball (2007), “Teacher Performance Pay: Synthesis of Plans, Research, andGuidelinesforPractice”,CPRE Policy Briefs RB-46,Madison,WI.

hofstede, G. (1980),Culture’s Consequences: International Differences in Work-Related Values,SagePublications,BeverlyHills,CA/London.

http://ideas.repec.org/a/bla/restud/v52y1985i2p173-91.html

http://ideas.repec.org/s/bla/restud.html


97chapter 3 referenCes


hofstede, G. (2001),Culture’s Consequences: Comparing Values, Behaviors, Institutions, and Organizations Across Nations,SagePublications,ThousandOaks,CA.

hollensbe, e. and J. Guthrie (2000), “Group Pay-for-Performance Plans:The Role of Spontaneous Goal Setting”, The Academy of Management Review,Vol.25,No.4,pp.864-872.

holmstrom, B.(1979).“MoralHazardandObservability”,The Bell Journal of Economics,Vol.10,No.1,pp.74-91.

holmstrom, B.(1982),“MoralHazardinTeams”,Bell Journal of Economics,Vol.13,No.2,pp.324-340.

holmstrom, B.andp. Milgrom (1987),“AggregationandLinearityintheProvisionofIntertemporalIncentives”,Econometrica,Vol.55,No.2,pp.303-328.

holmstrom, B.andp. Milgrom (1991),“MultitaskPrincipal-AgentAnalysis:IncentiveContracts,AssetOwnership,andJobDesign”,Journal of Law, Economics, and Organization,Vol.7,pp.24-52.

hoxby, c.anda. Leigh (2004),“PulledAwayorPushedOut?ExplainingtheDeclineofTeacherAptitudeintheUnitedStates”,American Economic Review,Vol.94,No.2,pp.236-240.

Neale, M., V. huberandG. Northcraft(1987),“TheFramingofNegotiations:ContextualVersusTaskFrames”,Organizational Behavior and Human Decision Processes,Vol.39,pp.228-241.

Indjejikian, r., p. LenkandD. Nanda(2000),“Targets,StandardsandPerformanceExpectations:EvidencefromAnnualBonusPlans”,SSRN Working Paper,http://papers.ssrn.com/sol3/papers.cfm?abstract_id=213628, accessedSeptember20,2009.

Jacob, B. andS. Levitt (2003), “RottenApples:An Investigationof thePrevalenceandPredictorsofTeacherheating”,Quarterly Journal of Economics,Vol.118,No.3,pp.843-877.

Jacob, B.andL. Lefgren (2005),“PrincipalsasAgents:SubjectivePerformanceMeasures inEducation”,KSG Working Paper No. RWP05-040,Cambridge,MA.

Jacobson, S. (1992), “Performance-Related Pay for the Teacher: The American Experience”, in H. Tomlinson (ed.),Performance-Related Pay in Education,Routledge,London.

Johnson, S.M.(2004).“ParalysisorPossibility:WhatDoTeacherUnionsandCollectiveBargainingBring”,inR.Henderson,W.Urban,andP.Wolman(eds.),Teacher Unions and Education Policy: Retrenchment or Reform?,Elsevier,Oxford,U.K.

Juerges, h., W. richterandK. Schneider (2004),“TeacherQualityandIncentives:TheoreticalandEmpiricalEffectsofStandardsonTeacherQuality”,CESifo Working Paper No. 1296,CESifo,Munich,

Kandel, e. ande. Lazear (1992), “PeerPressureandPartnerships”,The Journal of Political Economy,Vol.100,No.4,pp.801-817.

Kane, t.andD. Staiger (2001),“ImprovingSchoolAccountabilityMeasures”,NBER Working Paper 8156, NationalBureauofEconomicResearch,Cambridge,MA.

Kelley, c. (1999), “The Motivational Effects of School-Based Performance Awards”, Journal of Personnel Evaluation in Education,Vol.12,No.4,pp.54-78.

Kelley, c. (2002),“FinancialIncentivesinStateAccountabilitySystems:PerformancePayforTeachers”,CenteronReinventingPublicEducationandtheWashingtonStateAcademicAchievementandAccountabilityCommission,Seattle,WA.

Kimball, S., B. White, a. MilanowskiandG. Borman (2004),“ExaminingtheRelationshipBetweenTeacherEvaluationandStudentAssessmentResultsinWashoeCounty”,Peabody Journal of Education,Vol.79,No.4,pp.54-78.

Kingdon, G.andF. teal (2008),“TeacherUnions,TeacherPayandStudentAchievementinIndia”,CESifo Working Paper 2428,CESifo,Munich.

Kohn, a. (1999),Punished by Rewards: The Trouble with Gold Stars, Incentive Plans, A’s, Praise, and Other Bribes,HoughtonMifflin,Boston,MA.

Kole, S. (1997),“TheComplexityofCompensationContracts”,Journal of Financial Economics, Vol.43,pp.79-104.



98

© OECD 2009

Kremer, M.andD. chen (2001),“InterimReportonaTeacherIncentivePrograminKenya”,UniversityofChicago,Chicago,ILhttp://home.uchicago.edu/~dlc/papers/Preschool.pdf.

Ladd, h. (1999), “The Dallas School Accountability and Incentive Program: An Evaluation of Its Impacts on StudentOutcomes”,Economics of Education Review,Vol.18,No.1,pp.1-16.

Lavy, V. (2002),“EvaluatingtheEffectofTeachers’GroupPerformanceIncentivesonPupilAchievement”,Journal of Political Economy,Vol.110,No.6,pp.1286-1317.

Lavy, V. (2004), “Performance Pay andTeachers’ Effort, Productivity and Grading Ethics”, NBER Working Paper 10622,NationalBureauforEconomicResearch,Cambridge,MA.

Lavy, V. (2007),“UsingPerformance-BasedPaytoImprovetheQualityofTeachers”,The Future of Children,Vol.17,No.1,pp.87-109.

Lavy, V. (2008),“GenderDifferencesinMarketCompetitivenessinaRealWorkplace:EvidencefromPerformance-BasedTournamentsAmongTeachers”,NBER Working Paper 14338,NationalBureauofEconomicResearch,Cambridge,MA.

Lazear, e. (1995),Personnel Economics,MITPress,Cambridge,MA.

Lazear, e. (1998),Personnel Economics for Managers,Wiley,NewYork.

Lazear, e. (2000),“PerformancePayandProductivity”,American Economic Review,Vol.90,No.5,1346-1361.

Lazear, e. (2003),“TeacherIncentives”,Swedish Economic Policy Review,Vol.10,pp.197-213.

Lazear, e.andM. Gibbs (2009),Personnel Economics in Practice,Wiley,Hoboken,NJ.

Leventhal, t. and J. Brooks-Gunn (2004), “A Randomized Study of Neighborhood Effects on Low-Income Children’sEducationalOutcomes”,Developmental Psychology,Vol.40,No.4,pp.488-507.

Lewis, J.andM. Springer (2008),“PerformanceIncentivesinTexas:WhySchoolsChoseNottoParticipate”,Working Paper #2008-18,NationalCenteronPerformanceIncentives,Nashville,TN.

Liang, X. (1999),“TeacherPayin12LatinAmericanCountries:HowDoesTeacherPayComparetoOtherProfessions,WhatDeterminesTeacherPay,andWhoAretheTeachers?”, LCSHD Paper Series No. 49,WorldBank,Washington,D.C.

Locke, e.a. (1968),“TowardaTheoryofTaskMotivationandIncentives”,Organizational. Behavior and Human Performance, Vol. 3,pp.157-189.

Locke, e.andG. Latham (1990),A Theory of Goal Setting and Task Performance, Prentice-Hall,EnglewoodCliffs,NJ.

Locke, e., G. LathamandK. Smith (1990).A Theory of Goal Setting and Task Performance,Prentice-Hall,UpperSaddleRiver,NJ.

Locke, e., L. Saari, K. Shaw, G. Latham (1981),“GoalSettingandTaskPerformance:1968-1980”,Psychological Bulletin,Vol.90,No.1,pp.125-152.

Longenecker, c., D. Gioiaandh. Sims (1987), “Behind theMask:ThePoliticsofEmployeeAppraisal”,Academy of Management Executive,Vol.1,No.3,pp.183-193.

Lortie, D. (1979),Schoolteacher: A Sociological Study, UniversityofChicagoPress,Chicago,IL.

Malen, B. (1999), “On Rewards, Punishments, and Possibilities:Teacher Compensation as an Instrument for EducationReform”,Journal of Personnel Evaluation in Education,Vol.12,No.4,pp.387-394.

Marsden, D.andr. Belfield (2006),“PayforPerformanceWhereOutputisHardtoMeasure:TheCaseofPerformancePayforSchoolTeachers”,inB.Kaufman,E.BruceandD.Lewin(eds.),Advances in Industrial and Labor Relations,JAIPress,London.

Mcadams, J.ande. hawk (1992),Capitalizing on Human Assets,AmericanCompensationAssociation,Scottsdale,AZ.

Mccaffrey, D., B. hanandJ. Lockwood (2009),“IncentiveSystemDesignandMeasurement”,inM.G.Springer(ed.),Performance Incentives: Their Growing Impact on American K-12 Education,BrookingsInstitutionPress,Washington,D.C.

Mccollum, S. (2001),“HowMeritPayImprovesEducation”,Educational Leadership,Vol.58,No.5,pp.21-24.



Mcewan, p. (1999),“RecruitmentofRuralTeachersinDevelopingCountries:AnEconomicAnalysis”,Teaching and Teacher Education,Vol.15,No.8,pp.849-859.

Merchant, K.a. and J. Manzoni (1989), “The Achievability of BudgetTargets in Profit Centers: A Field Study”, The Accounting Review,Vol.64,pp.539-558.

Milgrom, p.andJ. roberts (1990),“Rationalizability,Learning,andEquilibriuminGameswithStrategicComplementarities”,Econometrica,Vol.58,No.6,pp.1255-1277.

Milgrom, p.andJ. roberts (1992),Economics, Organization & Management, PrenticeHall,EnglewoodCliffs,CA.

Milanowski, a. (2003),“TheVarietiesofKnowledgeandSkill-BasedPayDesign:AComparisonofSevenNewPaySystemsforK-12Teachers”,Education Policy Analysis Archives,Vol.11,No.4,http://epaa.asu.edu/epaa/v11n4/, accessedApril17,2009.

Milanowski, a. (2004), “The Relationship betweenTeacher Performance Evaluation Scores and Student Achievement:EvidencefromCincinnati”,Peabody Journal of Education,Vol.79,No.4,pp.33-53.

Mirrlees, J. (1999),“TheTheoryofMoralHazardandUnobservableBehaviour:PartI”,Review of Economic Studies,Vol.66,No.1,pp.3-21.

Mizala, a.andp. romaguera (2003),Scholastic Performance and Performance Awards,CentrodeEconomicaAplicada,Santiago,Chile.

Mizala, a.andp. romaguera (2004),“SchoolsandTeachersPerformanceIncentives:TheLatinAmericanExperience”,International Journal of Educational Development,Vol.24,No.6,pp.739-754.

Mullen, c. and K. Slagle (2007), “Teacher Perceptions of Systems-Driven Quality Teaching Initiatives”, Academic Leadership,Vol.3,No.1,pp.1-20.

Mullen, K., r. FrankandM. rosenthal (2009),“CanYouGetWhatYouPayFor?Pay-for-PerformanceandtheQualityofHealthcareProviders”,NBER Working Paper 14886,NationalBureauofEconomicResearch,Cambridge,MA.

Muralidharan, K. andV. Sundararaman (2008), “Teacher Incentives inDevelopingCountries:ExperimentalEvidencefromIndia”,HarvardUniversity,Cambridge,MA.

Murnane, r.andD. cohen(1985),“’Merit Pay’ and the Nature of the Employment Relationship: Lessons from Public School Systems. EducationResources InformationCenter,Washington,D.C.,http://www.eric.ed.gov/ERICDocs/data/ericdocs2sql/content_storage_01/0000019b/80/2f/4e/c7.pdf.

Murname, r. andD. cohen (1986), “MeritPayand theEvaluationProblem:WhyMostMeritPayPlansFailandFewSurvive”,Harvard Education Review,Vol.56,pp.1-17.

Murphy, K. (1999),“ExecutiveCompensation”,inHandbook of Labor Economics,Vol. III,Ashenfelter,O.andD.Card(eds.),NorthHolland,Oxford.

Murphy, K. (2001), “Performance Standards in Incentive Contracts”, Journal of Accounting and Economics,Vol. 30,pp.245-278.

Murphy, t. and J. Margulies (2004), “PerformanceAppraisals”.PaperpresentedatABAEmploymentandLawSectionConference,EqualOpportunityEmploymentCommittee,NeedhamHeights,MA,March24-27.

Nalbantian, h.anda. Schotter (1997),“ProductivityUnderGroupIncentives:AnExperimentalStudy”,The American Economic Review,Vol.87,No.3,pp.314-341.

Neal, D.(2009),“DesigningIncentiveSystemsforSchools”,inM.G.Springer(ed.),Performance Incentives: Their Growing Impact on American K-12 Education,BrookingsInstitutionPress,Washington,D.C.

Neal, D.andD. Schanzenbach (forthcoming),“LeftBehindByDesign:ProficiencyCountsandTest-BasedAccountability”,Review of Economics and Statistics.

Odden, a.andc. Kelley(1997),Paying Teachers for What They Know and Do: New and Smarter Compensation Strategies to Improve Schools,CorwinPress,ThousandOaks,CA.

http://epaa.asu.edu/epaa/v11n4/

http://ideas.repec.org/a/bla/restud/v66y1999i1p3-21.html


http://www.eric.ed.gov/ERICDocs/data/ericdocs2sql/content_storage_01/0000019b/80/2f/4e/c7.pdf

http://www.eric.ed.gov/ERICDocs/data/ericdocs2sql/content_storage_01/0000019b/80/2f/4e/c7.pdf



100

© OECD 2009

Odden, a., G. Borman, andM. Fermanich (2004),“AssessingTeacher,Classroom,andSchoolEffects,IncludingFiscalEffects”,Peabody Journal of Education,Vol.79,No.4,pp.4-32.

Oyer, p. (1998),“FiscalYearEndsandNonlinear IncentiveContracts:TheEffectonBusinessSeasonality”,The Quarterly Journal of Economics, Vol.113,No.1,pp.149-185.

paarsch, h.andB. Shearer (2000),“PieceRates,FixedWages,and IncentiveEffects:StatisticalEvidence fromPayrollRecords”,International Economic Review,Vol.41,No.1,pp.59-92.

perry, J., t. engbersandS. Jun (2009),“BacktotheFuture?Performance-RelatedPay,EmpiricalResearch,andthePerilsofPersistence”,Public Administration Review,Vol.69,No.1,pp.39-51.

perry, J.andL. porter (1982),“FactorsAffecting theContext forMotivation inPublicOrganizations”,The Academy of Management Review, Vol.7,No.1,pp.89-98.

pfeffer, J.(1995),Competitive Advantage Through People: Unleashing the Power of the Work Force,HarvardBusinessSchoolPress,Boston,MA.

pfeffer, J.andN. Langston(1993),“TheEffectofWageDispersiononSatisfaction,Productivity,andWorkingCollaboratively:EvidencefromCollegeandUniversityFaculty”,Administrative Science Quarterly,Vol.38,No.2,pp.382-407.

podgursky, M.andM. Springer(2007),“TeacherPerformancePay:AReview”,Journal of Policy Analysis and Management,Vol.26,No.4,pp.909-949.

prendergast, c. (1999),“TheProvisionofIncentivesinFirms”,Journal of Economic Literature,Vol.37,No.1,pp.7-63.

prendergast, c. and r. topel (1993), “Discretion and Bias in Performance Evaluation”, European Economic Review,Vol.37,No.2-3,pp.909-950.

rainey, h. (1982),“RewardPreferencesAmongPublicandPrivateManagers:InSearchoftheServiceEthic”,The American Review of Public Administration, Vol.16,No.4,pp.288-302.

rapple, B. (1990).PaymentbyEducationalResults:AnIdeaWhoseTimeHasGone?Thirty-FiveYearsofExperimentationwithEducationEfficiencyinEngland(1862-1897),MacArthur-Spencer Series Number 14,Chicago,IL.

reback, r. (2008),“TeachingtotheRating:SchoolAccountabilityandtheDistributionofStudentAchievement”,Journal of Public Economics,Vol.92,No.5-6,pp.1394-1415.

richardson, r. (1999), “Performance Related Pay in Schools:AnAssessment ofThe Green Papers”, London School ofEconomicsandPoliticalScience,London.

rivkin, S., e. hanushekand J. Kain (2005),“Teachers,Schools,andAcademicAchievement”,Econometrica,Vol.73,No.2,pp.417-458.

rosen, S. (1986), “TheTheory of Equalizing Differences”, in O.Ashenfelter and R. Layard (eds.), Handbook of Labor Economics,Vol. 1,North-Holland,Oxford.

rosenholtz, S.(1984),“Myths:PoliticalMythsAboutReformingTeaching”,NationalInstituteofEducation,Washington,D.C.

rothstein, r. (2009),“HowScholarshipandExperienceinOtherFieldsInformExplorationofPerformanceIncentivesinEducation”,inM.G.Springer(ed.),Performance Incentives: Their Growing Impact on American K-12 Education,BrookingsInstitutionPress,Washington,D.C.

Schacter, J.andY. thum (2004),“PayingforHigh-andLow-QualityTeaching”,Economics of Education Review,Vol.23,No.4,pp.411-430.

Schmitt, N., r. Noeandr. Gottschalk(1986),“UsingtheLensModeltoMagnifyRaters’Consistency,Matching,andSharedBias”,Academy of Management Journal,Vol.29,pp.130-139.

Schneider, M., p. teske andM. Marshall (2000),Choosing Schools: Consumer Choice and the Quality of American Schools,PrincetonUniversityPress,Princeton,NJ.

Skinner, B. (1981),“SelectionbyConsequences”,Science, Vol.213,No.4507,pp.501-504.



Springer, M. (2008), “The Influence of an NCLBAccountability Plan on the Distribution of StudentTest Score Gains.”Economics of Education Review,Vol.27,No.5,pp.556-563.

Springer, M. (2009), “Rethinking Teacher Compensation Policies: Why Now, Why Again?”, in M.G. Springer (ed.),Performance Incentives: Their Growing Impact on American K-12 Education,BrookingsInstitutionPress,Washington,D.C.

Springer, M., D. Ballouanda. peng(2008),Impact of the Teacher Advancement Program on Student Test Score Gains: Findings from an Independent Appraisal,NationalCenteronPerformanceIncentives,Nashville,TN.

Springer, M., M. podgursky, J. Lewis, M. ehlert, t. Gronberg, L. hamilton, D. Jansen, O. Lopez, a. peng, B. Stecher and L. taylor (2008), Texas Educator Excellence Grant (TEEG), Program: Year Two Evaluation Report, National Center onPerformanceIncentives,Nashville,TN.

Springer, M., M. podgursky, J. Lewis, M. ehlert, t. Gronberg, D. Jansen, O. Lopez, a. peng, B. Stecher and L. taylor (2009),Governor’s Educator Excellence Grant (GEEG), Program: Year Two Evaluation Report,NationalCenteronPerformanceIncentives,Nashville,TN.

Springer, M.andM. Winters(2009),New York City’s School-Wide Bonus Pay Program: Early Evidence From a Randomized Trial,NationalCenteronPerformanceIncentives,Nashville,TN.

Suen, h.andL. Yu(2006),“ChronicConsequencesofHigh-StakesTesting?LessonsfromtheChineseCivilServiceExam”,Comparative Education Review,Vol.50,No.1,pp.46-65.

taylor, L.andM. Springer (2009),“OptimalIncentivesforPublicSectorWorkers:TheCaseofTeacher-DesignedIncentivePayinTexas”,National Center on Performance Incentives Working Paper,Nashville,TN.

taylor, L., M. SpringerandM. ehlert(2009),“CharacteristicsandDeterminantsofTeacher-DesignedPayforPerformancePlans:EvidenceFromTexas’Governor’sEducatorExcellenceGrantProgram”,inM.G.Springer(ed.),Performance Incentives: Their Growing Impact on American K-12 Education,BrookingsInstitutionPress,Washington,D.C.

tomlinson, h. (2000),“ProposalsforPerformanceRelatedPayinEnglishSchools”,School Leadership and Management,Vol.20,No.3,pp.281-298.

Vroom, V. (1964),Work and Motivation,Wiley,NewYork,NY.

Weiss, a. (1987),“IncentivesandWorkerBehavior:SomeEvidence”,inH.Nalbantian(ed.),Incentives, Cooperation and Risk Sharing,Rowan&Littlefield,Latham,MD.

Weisberg, D., S. Sexton, J. MulhernandD. Keeling(2009),The Widget Effect: Our National Failure to Acknowledge and Act on Differences in Teacher Effectiveness, TheNewTeacherProject,Brooklyn,NY.

Weitzman, M. (1980),“TheRatchetPrincipleandPerformanceIncentives”,Bell Journal of Economics,Vol.11,No.1,pp.302-308.

West, M. andp. peterson (2006), “TheEfficacyofChoiceThreatsWithinSchoolAccountabilitySystems:Results fromLegislativelyInducedExperiments”,The Economic Journal,Vol.116,No.510,pp.C46-C62.

Wilson, W.(1987),The Truly Disadvantaged: The Inner City, the Underclass, and Public Policy,UniversityofChicagoPress,Chicago,IL.

Wise, S. (2004),“Work-lifeBalanceandCareersinNHSNursingandMidwifery”,EmploymentResearchInstitute,NapierUniversity,Edinburgh.

Zenger, t.(1992),“WhyDoEmployersOnlyRewardExtremePerformance?ExaminingtheRelationshipAmongPerformance,Pay,andTurnover”,Administrative Science Quarterly,Vol.37,No.2,pp.198-219.

evaluating and rewarding the quality of teachers

Documents