Advisory and Knowledge Services: Evidence for What

Works PIM Advisory Services Research Team 2014

PIM/IFPRI SEMINAR

17 September 2014, 12:00 – 2:00 pm

Washington, DC

How Did We Get Here?

IFPRI “Best Fit” Framework

GFRAS Evaluation

Guide

PIM Research

Area

Past Extension Work with PIM

• Building R4D learning platforms in Latin America, Africa Asia (CIAT)

• Evaluation of innovative extension approaches (ICRAF)

• Analyzing effects of decentralization and governance environment on policy processes and outcomes (IFPRI)

• Does agricultural training and female representation in extension foster investments among female farmers? Lessons from a policy experiment in Mozambique (IFPRI)

• Approaches for evaluating and increasing access to rural services by women and the poor (ILRI)

PIM’s “Advisory Services” Research

• October 2013 workshop

• 2014 work plan under PIM • IFPRI

• CIRAD

• GFRAS

• ICRAF

Overview of 2014 Work

Rationale

• Extension…advisory services…knowledge sharing…

• Critical institution for agricultural development

Goal

• Evidence & outreach to strengthen advisory services

Photo: Flickr, part of the image collection of the International Rice Research Institute (IRRI)

2014 Activities

1. Framework to assess performance of pluralistic extension systems

2. Evaluation of impact of extension on ag productivity

3. Systematic collection of “global good practices”

4. Assessments in Central Asia, Brazil, North Africa

5. Knowledge sharing platform

6. Proposal for 2015-2016 work

Impact of Agricultural Extension • AIM: Provide new set of evidence on impact of public investment • Past meta-reviews on economic impact and ROR (Birkhaueser et al.

1991; Alston et al. 2000; Evenson 2001): generally positive, but results widely varied, averages misleading, too much noise, suffer econometric deficiencies

• Recent studies addressed some econometric deficiencies (Dercon et al. 2013; Owens et al. 2003; Gautam 2000; Benin et al. 2011; Davis et al. 2012)

• We will use existing datasets with components on extension to provide new empirical evidence on impact of agricultural extension

• We will further disaggregate analysis and unpack extension input to explain reasons for low or high marginal impact and suggest ways to move forward

Malawi Case

• What is differentiated impact across regions, zones and groups of access to extension services?

• Do different sources of information or service providers matter in explaining differences in productivity?

• Does access to other service providers or other sources of information a substitute or complementary to public extension services?

• Do different types of advice or information provided matter in explaining productivity?

• Does the gender of the receiver of information matter in explaining productivity?

IHS3 2010/11 covering 31 districts; 560 rural EAs; 10,038 rural HH; and follow-up panel in 2013/14 covering 150 EAs; 2,400 rural HH

Assessments and Evaluations • How do we know RAS functions as supposed to?

• How to develop best-fit practices from wide range of contexts and policy regimes?

• Set of criteria for RAS program assessment and evaluations - Effectiveness, Efficiency, Relevance, Impact, Sustainability

• GFRAS evaluation indicators as starting point

• Currently engaged in applying methodology for Brazil, Central Asia, & North Africa

What does this mean for our team?

Which framework to assess the performance and impact of pluralistic extension systems ? • Objective : design and test a framework to support

national policy makers and top managers of service providers: • Which extension programs suit which needs under

what circumstances?

• How much public funding can be justified as a good investment?

• How performance and impact can be monitored?

Evaluations of Extension Performance: What do we have?

• Many evaluations of performance to assess the relevance of investments in extension (Neuchatel Initiative, World Bank, GFRAS, etc.)

• Mainly based on comparison between objectives of the project and achievements • Using OECD criteria (relevance, efficiency, effectiveness, sustainability) • Mix of surveys (SWOT, focus group, statistical surveys, etc.)

• More comprehensive studies about the evolution of RAS and the reforms undertaken: • Labarthe et al. on EU advisory systems, Babu in India, etc.

Evaluation of Extension Impact: What do we have?

• A few research to assess change of farmers’ skills (Cameron in Australia, de Romemont in Benin, etc.)

• A large number of evaluations to assess extension programs (T&V, FFS, and other) (see GFRAS documents) • Impacts on adoption of technologies, yields, and incomes,

• Some research to compare impacts of • different methods (video, farmer to farmer, etc.)

• type of service provider (NGO, private provider)

• A few research to assess spill-over

Conclusions about Evaluations • Positive impacts but variability and over-estimation

with regards to the real evolution of agriculture

• Methodological questions (the quality of the data, the design of samples, the attribution)

• Such methods consider the service providers as a black box (not very useful to provide recommendations)

• There is no study about the impacts of an extension system (with a set of providers providing different type of

advice) at territorial level/national level

Proposal to Assess Complex Pluralistic Extension Services

• Holistic approach based on IFPRI framework (Birner el al. 2009) • Need to operationalize taking into account the

national level and the local level • Improving the impact pathway approach (Douthwaite

2003) • Designing a mix of qualitative and quantitative

methods

Context-ual

factors

Policy, farming systems, access to market)

Perfor-mance

efficiency

effectiveness inclusion

sustainability

Impact

Economic Social

Environmental

Governance structure (funding)

Advisory methods

Service provider

capacities

Farm house-

hold Representation

Decision

Change of practices

Methodological Proposal • Mapping of RAS at national level (governance, funding,

capacities, methods) • Case studies to represent the diversity of service

providers and diversity of innovation processes • Impact pathway linking key outputs (information, training,

platform, etc.), outcomes (change at farm level) and impacts (economic, social, environmental, etc.)

• Two levels to assess impacts : direct impacts, indirect impacts

• Impact at national level = sum of impacts at case studies level

Tools to be Designed • Timeline diagram describing actions undertaken by

service provider and external events influencing advisory services

• Impact pathway diagram showing place where interactions advisors/farmers really take place, and changes at different levels

• Performance diagram is aiming to quantify an visualize the performances (effectiveness, efficiency, equity (targeting), sustainability)

• Impacts diagram to quantify and visualize different impacts

Key Methodological Questions to be Addressed

• How to select case studies to have relevant results?

• How to select performance indicators that make sense for actors and in line with national public policies on extension?

• How to calibrate a method to quantify the impacts (using existing surveys, carrying out specific surveys) (see Wellard 2014)?

• Next step : test the method in one or two countries

Global Best-fit Practices The problem:

• Much available about how a particular practice works in a particular context: “case studies”

• But little synthetic material available - how a particular practice works in different contexts

• E.g. farmer to farmer extension: dozens of case studies available about a particular experience but only one study found that cuts across countries and facilitating organizations: Selener et al 1997 – covering only Latin America

• High demand from practitioners for simple materials that present how particular practices perform in different contexts and key lessons for implementation

• Our proposed solution:

• Commission preparation of syntheses of best-fit practices and produce reports and training materials useful for policy, rural advisory services managers and practitioners (different products likely needed for different target groups)

• Syntheses will be sought from 5 key dimensions of advisory services, using a structured framework for case selection:

• Governance structures (e.g., level and sources of financing)

• Policy (e.g., experiences with national advisory services policies)

• Capacity and management (e.g., staff numbers, expertise)

• Advisory methods (e.g., the use of ICTs)

• Cross cutting (e.g. gender, nutrition)

5 Dimensions of extension

Theme Case Country Typology

Inter-country Intra-country

Socio-economic

Political Organiza-tional

Ecological

1.Governance Decentrali-zed extension systems

Ethiopia Agrarian Federal (decentrali-zed)

Government providers

Low natural resource base, subsistence, medium density

2. Policies

3. Capacity & management

4. Methods

5. Cross-cutting

23

Implementation plan • Fund raising. GIZ already on board

• Establish advisory committee composed mainly of potential users of the information

• The committee, in consultation with regional ag. advisory system networks, selects potential topics

• Issue call for proposals to conduct syntheses

• Award grants; develop and complete studies

• Develop informational materials and implement communication and capacity strengthening strategy to promote their use

Where Do We Go from Here?

• Team meetings, 18 Sep, 6-7 Nov

• Develop common framework and proposal

• Feedback from experts*

• Engage partners, find funding

*Mark Lundy (CIAT), Cheryl Doss (Yale), Arame Tall (CCAFS), Aden Aw-Hassan (ICARDA), Laurens Klerkx (Wageningen), Cathy Colverson (University of Florida), Ismail Moumouni (University of Parakou, Benin), Victor Manyong (IITA), Pierre Labarthe (INRA), Andrea Knierim (Hohenheim University), Evelyne Kiptot and Anne Degrande (ICRAF) and Katarina Kosec, David Spielman, Ephraim Nkonya, Tewodaj Mogues (IFPRI).

Response by the Panel

• Cheryl Doss, Yale

• David Spielman, IFPRI

• Valerie Mueller, IFPRI

• Pierre Labarthe, INRA

Questions? Discussion & Conclusions