Photo Credit Goes Here

IAPRI-MSU Technical Training

IntrotoEconomicExperimentswithaFocusonFramedFieldExperiments

FacilitatedbyHambuloNgoma&NicoleMason

TrainingmaterialsdevelopedbyHambuloNgoma(IAPRI),NicoleMason(MSU),andStephenMorgan(MSU)

22June2018

IndabaAgriculturalPolicyResearchInstituteLusaka,Zambia

Why this training? •  Economicexperimentsofdifferent

typesarebecomingincreasinglypopularandareincreasinglyusedinagriculturaleconomicsanddevelopmenteconomicsresearch,incl.policyanalysis

–  NobelPrizeinEconomics(experimentalandbehavioraleconomistwinners)

•  2002:VernonSmith-"fortheuseoflaboratoryexperimentsasatoolinempiricaleconomicanalysis,inparticular,forthestudyofdifferentmarketmechanisms”.DanielKahneman-“fortheintroductionofinsightsfrompsychologicalresearchintoeconomics,inparticularwithregardtojudgementsanddecisionsunderuncertainty”

•  2017:RichardThaler-“forhiscontributionstobehavioraleconomics”

•  Tobroadenyourresearchmethodstoolkit

•  Toenableyoutobetterunderstandotherstudiesthatuseeconomicexperiments

Road map 1.  Briefintrotoeconomicexperiments-Nicky[10-11AM]

– Whatarethey,whydothem,differenttypes,examplesfromStephen&Nicky’swork

2.  Framedfieldexperiments(FFEs)–Hambulo[11:15-12:00]– Whatare,andwhyframedfieldexperiments?

3.  FFEapplicationsoncommonpoolresources[12-1PM]•  ExamplesfromColombiaandTanzania•  Hands-onpracticefromrecentFFEsinZambia

Brief intro to economic experiments – Learning goals

Bytheendofthisportionofthetraining,youshouldbeableto:1.  Explainwhataneconomicexperimentisandgive

someexamplesofthingstheycanbeusedtostudy2.  Describesomepros/consofeconomicexperiments3.  Distinguishbetweendifferenttypesofeconomic

experiments(e.g.,labexperiments,artefactualfieldexperiments,framedfieldexperiments,andnaturalfieldexperiments)

What is experimental economics?

Anempiricaltoolthateconomistsandinterdisciplinaryteamscanusetounderstandtheextenttowhichanindividual’s(orgroup’s)decisionsorbehaviorareaffectedbyvarious(testable)factorsinacontrolledenvironment.

Example:AlabexperimentbeingconductedattheLoyolaMarymountUniversityExperimentalEconomicsLab(Source:http://econlab.net/)

Example:AframedfieldexperimentbeingconductedinEthiopiabyresearchersfromtheUniversityofFrankfurt(Source:https://www.wiwi.uni-frankfurt.de/abteilungen/mm/flex/flex.html)

Source:AmereTeklayHailu.2018.NMBUPh.D.triallecture

A key advantage of economic experiments •  Givestheresearcher(more)controlover“treatment

status”ofstudyparticipants•  Recalltheimpactevaluation(IE)trainingswehavedone

overthelastyear.WhatisthemajorchallengeinIE,particularlywhenweareusingobservationaldata?

•  Inaneconomicexperiment,theresearcherrandomlyassignsparticipantstotreatmentandcontrolgroups– Whyisthishelpful?–  Enablescleaneridentificationofthetreatmenteffect– Makesanalysiseasier(e.g.,canoftenusesimpleOLSregressionofoutcomevariableontreatmentindicator)

Economic experiments can be used to: •  Testtheories•  Establishempiricalregularitiesasabasisfornewtheories•  Testinstitutions/rulesofthegame/markets•  Studypreferencesanddecision-making

–  E.g.,Riskandtimepreferences,preferencesforgoodsandservices,cooperation(publicgoods),etc.

•  Estimateparameters•  Replicatepreviouswork•  Teacheconomics•  …amongothers!

A taxonomy of economic experiments (Harrison & List 2004 - p. 1014)

Experimenttype Keyfeatures(emphasisadded)

Conventionallabexperiment ”employsastandardsubjectpoolofstudents,anabstractframing,andanimposedsetofrules”

Artefactualfieldexperiment “sameasaconventionallabexperimentbutwithanonstandardsubjectpool”(i.e.,notstudents)

Framedfieldexperiment “sameasanartefactualfieldexperimentbutwithfieldcontextineitherthecommodity,task,orinformationsetthatthesubjectscanuse”

Naturalfieldexperiment “sameasaframedfieldexperimentbutwheretheenvironmentisonewherethesubjectsnaturallyundertakethesetasksandwherethesubjectsdonotknowthattheyareinanexperiment”

Source:Harrison,G.W.,&List,J.A.(2004).Fieldexperiments.JournalofEconomicliterature,42(4),1009-1055.

Whataboutthingslikeexperimentalauctions&RCTs?

Have any of you been involved in an economic experiment?

•  Ifso,whatkindwasit?– Labexperiment– Artefactualfieldexperiment– Framedfieldexperiment– Naturalfieldexperiment– RCT

Example #1: A lab experiment Morgan,Mason,andShupp(2018).“Comments,contributionsandcompliance:Evidencefromapublicgoodsexperiment.”Manuscriptunderreview.•  Researchquestion:areeconomicagentsmorelikelytocomplywitha

ruleorregulationthatisultimatelychoseniftheyaregiventheopportunitytoprovideinputonwhatthatruleorregulationshouldlooklikebeforeitischosen?–  Also,anyinteractionwithenforcement?

•  Labexperimentsotheexperimentitselfwasnotframed(veryabstract)butthepolicy-relevance/contextthatmotivatedustothinkaboutthisquestionwasagri-environmentalpoliciesandopencommentperiodsintheUS.(Explain.)–  ZambiaConstitutionAmendmentBill&stakeholdercomments

Control Enforcementonly

Commentsonly Comments+enforcement

4treatmentgroups:

•  4”players”+1“policymaker”pergroup•  Playersgivenanendowmentof25pointsthattheymustdecidehowtoallocatetotheirprivate

accountvs.tothegroup’saccount•  Pointsallocatedtotheprivateaccountreturn1pointtotheindividualplayer•  Pointsallocatedtothegroupaccountreturn0.4pointstoALLplayersinthegroup(=1.6total)

•  Policymakerdecidesonaminimumcontributionrule(MCR,howmanypointsplayersshouldallocatetothegroupaccount)

•  Playersinthecomment-relatedtreatmentgroupshavetheopportunitytoprovideananonymoussuggestiontothepolicymakeronwhattheMCRshouldbe

•  Playersintheenforcement-relatedtreatmentgroupshavea50%chanceofbeing“caught”iftheycontributelessthantheMCRtothegroupaccount.Ifcaught,losetheirentireendowment.

•  Repeatmanytimes.Lookatcontributionstothegroupaccount(publicgood),compliancewiththeMCR.Testforstat.sig.differencesamongtreatmentandcontrolgroups.

Example #1: A lab experiment (cont’d)

Example:AlabexperimentbeingconductedattheLoyolaMarymountUniversityExperimentalEconomicsLab(Source:http://econlab.net/)

Example #1: A lab experiment (cont’d) •  PlayedinacomputerlabatMSUwithundergraduatestudents(oTree

software)

Example #2: A lab experiment & a related artefactual field experiment

Morgan,Mason,Shupp,andMyers.“Agriculturalinnovationandsustainabletechnologyadoption:Strategicdelayreconsidered.”Workinprogress.•  Researchquestion:areeconomicagentsmorelikelytoputoffadoptinga

newtechnologyiftherateofinnovationishighcomparedtowhenitislow?–  EX)ThinkaboutiPhonesorothermobilephones–  Thistypeofissuehasbeenexploredinothercontextsbutnotinagric.

•  Twoexperiments:

–  Labexperimentunderway(onlinew/MSUstudents)–  Artefactualfieldexperimentupcoming(onlinew/MIwheatproducers)

Lowrateofinnovation(20%probabilityofanewtechnologybecomingavailableinagivenperiod)

Highrateofinnovation(80%probabilityofanewtechnologybecomingavailableinagivenperiod)

2treatmentgroups:

•  Individualplay(notgroupplay)•  Given100pointendowment•  Startoutwithagiventechnology.Ifanewtechnologybecomesavailable,playerhastodecideifs/he

wantstostaywiththeircurrenttechnologyorswitchtothenewtechnology•  Ifswitchtoanewtechnology,pay50pointstoswitch

•  Asnewtechnologiesareadded,oldonesarestillavailable(canswitchbackifwantto–nocostifswitchingbacktoatechnologythey’vealreadyused)

•  Foreachtechnologythatisavailable,participantistoldtheminandmaxreturnspossiblewiththattechnology,andthenaftertheychoosewhichtechnologytouse,returnsarerandomlychosenfromwithinthatrange.Pointsaddedtotheiraccount.

•  Repeatmanytimes(26rounds)•  Playedonline

Example #2: A lab experiment & a related artefactual field experiment (cont’d)

Example #2: A lab experiment & a related artefactual field experiment (cont’d)

LABEXPERIMENTVERSION(playedbyMSU

undergraduatestudents)

Example #2: A lab experiment & a related artefactual field experiment (cont’d)

ARTEFACTUALFIELDEXPERIMENTVERSION

(tobeplayedbyMichiganwheatfarmers)

Another key benefit of economic experiments (esp. lab, artefactual field, and framed field):

•  CanoftenobtaindatamorequicklyandinexpensivelythanRCTsandpanelsurveys

•  EX)TwoofStephen’sPh.D.dissertationessays:–  Abletodowithsmallgrants(US$10,000&US$12,000).–  Inpersonlabexperimentcompletedin12one-hoursessions(N=215)

–  Plannedartefactualfieldexperimentbeingdoneon-line(participantsemailedalinktotheexperimentandsurvey)–anticipatedatawithinroughly2weeks(N=100-160)

•  ContrasttoRALScostsandtime(butnotethatRALSismuchlargerNandisusefulforMANYstudies,whereasaboveexperimentsaresmallNandonaverynarrowsetofresearchquestions)

Economic experiments: Internal validity is higher than studies based on observational data but external validity can be a concern

(depending on the type of experiment)

•  Why?

•  Whatdowemeanbyinternalvalidityandexternalvalidity?

Source:Roe,B.E.,&Just,D.R.(2009).Internalandexternalvalidityineconomicsresearch:Tradeoffsbetweenexperiments,fieldexperiments,naturalexperiments,andfielddata.AmericanJournalofAgriculturalEconomics,91(5),1266-1271.

Small group discussions •  Bouncearoundideaswitheachotherforlaborartefactualfieldexperimentsyoucouldpotentiallyrunaspartofyourresearchprogram– Whathypothesis(es)doyouwanttotest?– Howmightyoustructurealaborartefactualfieldexperimenttotestit?

Photo Credit Goes Here

FramedFieldExperiments:What,WhyandHow

22June2018IndabaAgriculturalPolicyResearchInstitute

Lusaka,Zambia

Roadmap

1.  Framedfieldexperiments(FFEs)[11:15AM-12pm]•  whatare,andwhyframedfieldexperiments?

2.  FFEapplicationsoncommonpoolresources[12-1pm]•  examplesfromColombiaandTanzania•  hands-onpracticefromrecentFFEsinZambia

LearningobjectivesBytheendofthissession,participantsshouldbeableto;

1.  defineframedfieldexperiments(FFEs)2.  differentiateFFEsfromotherfieldexperimentsin

economics&highlightsomemeritsandconsofFFEs

3.  setupabasicFFEgame,and4.  findrelevantliteratureonFFEs

Whatareframedfieldexperiments?FFEsarefieldexperimentsconductedwithasampleofrealsubjectsintheactualsettingswheretheymakereal–lifedecisionsrelatedtothestudyandusingacommodityasrealasispossible

FramedFieldExperimentinZambia:setting,commodityandtask.Source:IAPRI

CommodityEnvironmentTask

ThreekeyfeaturesdistinguishFFEs

•  Subjectpool:fieldsubjectsrecruitedfrompopulationofinterestforrealworldexperiences

•  Commodityandtasks:framedtobeasrealaspossible,e.g.,treebranchesandcuttingtreesandsubjectsplayforrealstakes

•  Environment:FFEsconductedinactualplaceswheresubjectsmakeeconomicdecisionsineverydaylives

FFEsversusothercontrolledexperiments

Sources:Harrison,G.W.,&List,J.A.(2004).Fieldexperiments.JournalofEconomicliterature,42(4),1009-1055Roe,B.E.,&Just,D.R.(2009).Internalandexternalvalidityineconomicsresearch:Tradeoffsbetween

experiments,fieldexperiments,naturalexperiments,andfielddata.AmericanJournalofAgriculturalEconomics,91(5),1266-1271.

Controlledexperiment

Lab Artefactual FramedField

Subjectpool Students Relevantsample Relevantsample

Commodity/task Abstract Abstract Asrealaspossible

Environment Laboratory Relevantlocation Relevantlocation

Internalvalidity High Mediumtohigh Mediumtohigh

Externalvalidity Low Medium Mediumtohigh

ThinkingaboutconductingFFEs?•  Researchquestion/hypothesis:whatareyoucuriousabout?

Whatdoyouwanttofindoutortest?•  Treatments:whataretherelevantinstrumentstochangein

theexperiment?•  Experimentaldesign:howaresubjectsallocatedtoexperiment

groups?–  Randomizationdesign,randomizationblockdesignetc

•  Sampling:howaresubjectsrecruited?•  Framing:whatistheframingforcommodityandtask?•  Stakes:whatarethestakesandpayofffunctions?•  Taketimeindesigningtheexperiment!

FramedFieldExperiments:Applicationstothemanagementofcommonpoolresources

Whatarecommonpoolresources?

Source:Ostrom,E.,Gardner,R.,&Walker,J.M.(1994).Rules,Games,andCommon-poolResources.AnnArbor:UniversityofMichiganPress.

Cardenas,J.C.,Stranlund,J.,&Willis,C.(2000).LocalEnvironmentalControlandInstitutionalCrowding-Out•  Researchquestion(s):howdoexternalregulationsaffecttimespentharvestingfirewoodinColombia?–  Collectingfirewoodaffectedwaterqualityduetosoilerosion

•  Treatments:–  commandandcontrol(governmentimposedquota)–  communityforestmanagement(communication,cheaptalk)

•  Experimentaldesign:14groups(of8subjects)played8–11initialroundsandanadditional9–12roundswithtreatment–  eachchosexiE[0,8]

Cardenas,J.C.,Stranlund,J.,&Willis,C.(2000).LocalEnvironmentalControlandInstitutionalCrowding-Out.WorldDevelopment•  Mainfindings:•  Regulationcrowdedoutother–

regarding

–  weaklyenforcedregulationledsubjectstobeself-centered

•  Communicationfosteredcooperationandhadbetterconservationoutcomes

Handberg,Ø.N.,&Angelsen,A.(2015).Experimentaltestsoftropicalforestconservationmeasures.JEBO•  Researchquestion(s):whataretheimpactsofCAC,CFMandPESonforeststockinTanzania?

–  atissueisdeforestation,leadingtoclimatechange–  REDD+triestoaddressdeforestationusingPES

•  Treatments:–  commandandcontrol(CAC)–  communityforestmanagement(CFM)–  paymentforenvironmentalservices(PES)

Handberg,Ø.N.,&Angelsen,A.(2015).Experimentaltestsoftropicalforestconservationmeasures.JEBO•  Experimentaldesign:36groups(of8subjects)played6pre–andpost–treatmentrounds–  payoffsbasedonharvestandstanding

treesfromastockof80tokens(papertrees)managedbyacommunityof8

–  participantsprivatelydecidedonharvest(withinlimit)ineachround,aggregateharvestannouncedandremoved,butreplacedbeforethenextround.Gamesplayedfor12rounds

Handberg,Ø.N.,&Angelsen,A.(2015).Experimentaltestsoftropicalforestconservationmeasures.JEBO

•  Mainfindings:•  CFMisaseffectiveasCACin

increasingprosocialforestuse

•  PESwasnoteffectiveinpromotingconservation

•  Moral,non–pecuniarymotivesimportantforconservation

Now,let’sconductaframedfieldexperiment

WhitherforestinZambia?Testingpolicyinstrumentsforcommunityforestmanagement

usingframedfieldexperiments(BasedonNgomaetal.,forthcoming)

Motivation

•  Zambiahasnecessarypolicyframeworkinsupportofsustainableforestmanagement(SFM)

–  ForestActof2015,NationalForestryPolicy,NationalClimateChangePolicy,REDD+strategy,2018CFMregulationsetc.

–  SFMinstrumentsaroundCFM,PESandtosomeextentCACareproposedandsometrialed

–  Deforestationremainsachallenge

–  Someknownquestionsremainunanswered:•  whataretheimpactsofCFM,CAC,PESandOAonforestconservation?•  canCFMoutperformothers(CAC&PES)?•  withinPES,ispayingindividualsbetterthatpayinggroups?

Motivation•  ItisdifficulttoaddressthesequestionsforZambia:

–  someofregulationsareyettobeimplemented–  often,policyinstrumentsareimplementedsingly,makingcrosscomparisondifficult,and

–  thereisamissingdataproblem:forestusedecisionsareonlyobservedunderonepolicyinstrumentatatime

•  Yet,weneedtoanswerthesequestionsex-antetoinformpolicy

•  Weusedframedfieldexperiments(FFEs)playedwithrealforestusersintheactuallocationswheretheymakeeverydayforestusedecisions,andusingactualtreebraches

Fieldworkandsampling

•  FieldworkconductedinMpikaandSerenjedistricts–  2villagesselectedineachdistrictbasedonhavingaforestinvicinityandwith>48households

•  Includedonevillageisforestreserve–  48householdsrandomlysampledineachvillageandeitherhusbandorwifeinvitedtoparticipateinthe‘study’

–  Eachparticipantwasrandomlyassignedtoapre-determinedgroupof8withaspecifictreatmenttoavoidkin–altruism

–  Intotal24groupsparticipatedforasampleof191and1,910observations*

Experimentaldesignandbasicgamestructure•  Eachgroupof8satinacirclewith60treebranchesandplayed

theexperimentover10roundsafterinitialpractice–  Stage1:pre–treatment(1-5rounds)–  Stage2:treated(6-10rounds)–  nocomminationwasallowedduringthesessions

•  Thisaone-shotgamerepeated10times

•  Detailedinstructionsgiveninlocallanguageatthestartofeverysession–  individualharvestsindicatedondecisionsheetsineveryround–  totalperroundannouncedandremovedbeforenextround–  stockreplenishedbeforenextround–  payofffunction:

( ) ⎡ ⎤⎣ ⎦∑ maxit it s it - it itπ = px + q N X - x - x , x < x

Treatments•  OpenAccess:4groupsplayedthebasedgamefor10rounds

•  CFM:5groupsplayedthesecondstagewitha3mincommunicationallowedbetweenrounds

•  CAC:5groupsplayedsecondstagewithsanctionsimposedforxi>3

•  PES,individualpay:5groupsplayedthesecondstagewithanadditionalincentiveof80%ofpasifxi<xRL

•  PES,individualpay:5groupsplayedthesecondstagewithanadditionalincentiveof80%ofpasifSum(xi)<xRL

Let’sgetworkingfolks…

Commodity Task Task

Trendsinharvestratesbytreatment

Harvestratespre–andpost–treatment

Harvestbytreatment,pre–andpost–treatment

Harvestratesbyvillage

Withintreatmentmeanharvestrates

Harvestrate(pre-treatment(1))

Harvestrate(post-treatment(2))

T-test(1)-(2)

N

OA 0.542(0.024) 0.492(0.027) 0.050 310

CAC 0.485(0.021) 0.431(0.019) 0.054* 400

CFM 0.488(0.023) 0.429(0.022) 0.059* 400

PES,individualpay 0.481(0.021) 0.312(0.017) 0.169*** 400

PES,grouppay 0.486(0.023) 0.483(0.024) 0.003 400

Overalleffects 0.494(0.010) 0.426(0.010) 0.068*** 1,910

Betweentreatmentmeanharvestrates

Econometricresults

(1) (2) (3) Paneldatafractionalresponsemodel POLS SE Treatments SE Full SECFM(yes=1) -0.084*** 0.026 -0.051 0.051 -0.083* 0.048CAC(yes=1) -0.025 0.027 -0.056 0.048 -0.021 0.052PES,individualpay(yes=1) -0.151*** 0.023 -0.123*** 0.046 -0.152*** 0.042

PES,grouppay(yes=1) -0.021 0.026 -0.031 0.053 -0.021 0.048

Numberoftripstotheforest -0.010 0.007 -0.009 0.014

Soldforestproductlastmonth(yes=1)

0.046** 0.019 0.050 0.039

Conclusion

•  Individualpayisbetterthangrouppayforconservation•  Theimpactsofcommunityforestmanagementaresmall

–  couldbecombinedwithmarket-basedincentivestoimproveforestconservation

–  Thus,clarifyingbenefitsharingmechanismsincommunityforestmanagementandtakingintoaccountindividuals’non-pecuniarymotiveswillbeimportanttoforconservation

Thank you for your attention & participation! HambuloNgoma(hambulo.ngoma@iapri.org.zm)ResearchFellowIndabaAgriculturalPolicyResearchInstituteNicoleMason(masonn@msu.edu)AssistantProfessorDepartmentofAgricultural,Food,&ResourceEconomics(AFRE)MichiganStateUniversity(MSU)StephenMorgan(snmorgan@msu.edu)Ph.D.CandidateAFRE/MSU

Acknowledgements

ThistrainingwasmadepossiblebythegeneroussupportoftheAmericanPeopleprovidedtotheFeedtheFutureInnovation Lab for Food Security Policy [grant numberAID-OAA-L-13-00001] through the United States AgencyforInternationalDevelopment(USAID).The contents are the responsibility of the trainingmaterialauthorsanddonotnecessarilyreflecttheviewsofUSAIDortheUnitedStatesGovernment.

www.feedthefuture.gov