what did we learn from impact evaluations in agriculture?

AADAPT Workshop South AsiaGoa, December 17-21, 2009

What did we learn from impact evaluations in agriculture?

Florence Kondylis

State of the knowledgeThere are very few rigorous impact evaluations of agricultural interventions. Why? Mostly, a few, scattered evaluations

do not respond to a larger learning agenda and needs on the ground

Not effective in affecting policy making! Methods often improvised ex-post

Harder to isolate causal links

Things are looking up It’s all getting better!

Increasing interest and innovative work: many of the papers in the talk are recent

We are doing it (cf. Mushfiq’s talk)2/4 years from now, we will be in a

different place

A quick tour of some of the evidence1. Technology adoption/diversification2. Information Technology & Rural

Markets Price Information Kiosks/Quality Hubs for

Soybeans in MP Cellphones and market performance in

Niger and Kerala3. Fertilizers in Kenya

1. How much?2. How to?

1. In Kenya, researchers asked:(Ashraf, Gine, Karlan 2008)

What is the impact of agricultural extension and export facilitation services provision (in various combinations) on:

Crop choice Technology adoption, and Market access?

What is the additional effect on all the above of providing in-kind credit for inputs (seeds, fertilizers)?

Used experimental design (2 treatment arms; farmers assoc) and found:

Farmers in program 19.2% more likely to grow export crops, have 4.3% more land devoted to cash crops

No overall impact on Input usage, HH income , Harvest value No significant additional impact of credit

2.1. Soybeans In Madhya Pradesh(Goyal, 2008)

What is the effect of price information provision, quality testing and direct private marketing facilitation on: Sale prices received by farmers in the mandis? Fraction of production sold through the mandis? Farmers’ subsequent planting decision?

Exploited the gradual phase-in of the intervention to find: Increase in the sale price by 1.6% for HHs in close proximity

to the mandis + decline in price dispersion across mandis Steep decline in volume sold through the mandis (14-20%,

increases with distance to the closest mandis) 3 percentage point increase area under soy cultivation

2.2. Cellphones and Market Performance in Kerala (Jensen 2007) and

Niger (Aker 2009)

What is the effect of an increase in cellphone coverage on Market performance (price dispersion) Traders’ behavior Consumer, producer and trader welfare?

Exploit regional variations in the timing of cellphones introduction and find: Large reductions in price dispersion (20-38 %), ▪ increase with transportation costs/distance to markets

Not attributable to collusive behavior Increase in farmer/fisherman’s and trader’s welfare

despite decrease in price (lower waste)

Fertilizer in Kenya: How much and How to?(Duflo, Kremer, Robinson 2008 & 2009)

FertilizerIntervention1 teaspoonper plant

½ teaspoonper plant

Fertilizer in Kenya: How much and How to?(Duflo, Kremer, Robinson 2008 & 2009)

FertilizerIntervention1 teaspoonper plant

½ teaspoonper plant

Fertilizer in Kenya: How much and How to?(Duflo, Kremer, Robinson 2008 & 2009)

FertilizerIntervention

1 teaspoonper plant

½ teaspoonper plant

Before / After harvestFree deliveryNo Subsidy

At “dressing”Free deliveryNo subsidy

At “dressing”Heavy SubsidyFree delivery

Fertilizer in Kenya: How much and How to?(Duflo, Kremer, Robinson 2008 & 2009)

FertilizerIntervention

1 teaspoonper plant

½ teaspoonper plant

Before / After harvestFree deliveryNo Subsidy

At “dressing”Free deliveryNo subsidy

At “dressing”Heavy SubsidyFree delivery

Fertilizer in Kenya: How much and How to?(Duflo, Kremer, Robinson 2008 & 2009) Free delivery after harvest

Impact did not persist past the 2 seasons the intervention lasted

Now need to learn how to achieve sustainable results??

So what did we learn? Some interesting things about a few programs in

some areas in a few countries Evaluations to date are often (but not always)

opportunistic: no overall strategy (Showed you the “best identified” ones but…) Often

methodologically challenged because they weren’t set up beforehand

Shift the focus on getting clear answers for key projects and areas

Start evaluations early on!