INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

Qualitative Methods for Monitoring, Evaluation and Impact Assessment

Jemimah Njuki

INTERNATIONAL LIVESTOCK RESEARCH INSTITUTE

Open-ended questions to identify types of indicators or impacts

• May feed into survey questions

Provides interpretive power

• What as well as why

Exploring topics less amenable to survey questions

Express local voice and perceptions

Uses of qualitative approaches

Advantages of qualitative methods

Qualitative methods are effective at capturing these issues because

• Elaboration, and sometimes several tries and lengthy follow-up and probing, are needed to convey the questions adequately and understand answers

• People do not always tell the truth the first time: Getting candid information often requires time, trust, rapport, triangulation, observation

Discover unanticipated issues

Solicit local solutions to problems

Qualitative as complementary to quantitative tools

Identifies issues and questions for surveys and hypotheses for testing

Identifies response options for survey questions

Clarifies terms/language for use in surveys

Confirms validity of constructs and proxies

Triangulation, confirmation, contradiction

Explanation/interpretation of survey findings

• Depth, texture, context

Quantitative as a complementary to qualitative

Identifies stratification strategy

Provides community and household characteristics for sampling

Identifies issues for investigation

Triangulation, confirmation, contradiction

Determines prevalence of qualitative findings in wider population

Reveals representation of qualitative sample

Some misconceptions about qualitative approaches

It is easy to do and cheap, and anyone can do it.

“If you can talk to farmers, you can use qualitative approaches and tools”

With qualitative tools, all you need is to document the results. No analysis is needed—

”just write up the report, its basic stuff”

“You don‟t need a research design! Just talk to a few farmers and do a few focus group discussions”

A lack of understanding of the tools and approaches

“We are using a PRA tool”

Different types of tools

Mapping tools

PRA tools Interviews Ethnographic tools

Participatory impact diagrams

Diffusion Maps

Before and After resource /asset maps

Social network analysis

Transect Walks

Trend lines

Venn diagrams

Seasonal calendars

Focus Group Discussions

Community meetings

Ranking /Rating/Scoring

Community surveys

Semi-structured interviews

Unstructured d interviews

Key informant interviews

Organizational assessment

Case studies

Innovation Histories

Life Histories

Personal diaries

Useful for identifying important traits / criteria for organising issues, items by preference

Used mainly for technology evaluation, ranking of priority options e.g priority household income options, asset preferences

Can be used to compare preferences across groups e.g men and women,

Scoring /Ranking /Rating methods

Ranking Priority Assets for Men and Women

Scoring /ranking /Rating methods

Advantages

Can be used with symbols and counters especially with groups with low literacy levels

Can be done individually or in groups

Allows for group contribution of list to be ranked /rated and the criteria to use

Can be easily quantified

Disadvantages

Takes time especially when community groups identify their own lists and criteria for evaluation

Mapping Tools-Participatory Impact Diagrams

Mapping Tools: Before and After Maps

Before After

friend

DENGORE

Sister’s

husband

Mhaber

Brother

WOLENCHETI

TOWN

MAP LEGEND

Methods of Seed Dissemination

Gift (3kg or less)

Gift (3kg or more)

Exchange

Sale

Neighboring

Village

brother

mother brother

friend

friend

Wife’s father

DONI

10 km

6 km

MERKO

4 km

Mekanajo

Neighbor-

struggling widow

MOME

Mekanajo/

Mhaber

7 km

Mekanajo

75 km

Iddir

neighbor

Appendix 3b. Technology Diffusion / Social Network Mapping: Simplified Sample from Worka Village

Iddir

Iddir

Mekanajo

/Mhaber

Mapping Tools: Diffusion

Mapping tools

Advantages

Can be used with groups that have low literacy levels

Very engaging

Easy visual presentation

Can provide massive amounts of information that combine qualitative as well as simple numbers

Can be done on paper or on the ground

Disadvantages

Requires close facilitation

Time consuming

Can be dominated by those that can write (who holds the pen /chalk, stick?)

Focus Group Discussions: Description

A group discussion of approximately 6 - 12 persons guided by a facilitator, during which group members talk freely and spontaneously about a certain topic.

Its purpose is to obtain in-depth information on concepts, perceptions and ideas of a group

Can be useful to:

• Focus research and develop relevant research hypotheses by exploring in greater depth the problem to be investigated and its possible causes

• Formulate appropriate questions for more structured, larger scale surveys

• Help understand and solve unexpected problems in interventions

• Explore controversial topics

FGDs; Advantages and Disadvantages

Advantages

Cost- and time-efficient—cover more people

Can be a safe space for discussion of sensitive issues

Early identification of important issues

Discussions trigger ideas, recollections, opinions

Disadvantages

Less time to explore and probe

Inability to triangulate data on individuals/households

Louder and quieter voices, peer pressure

No very appropriate for sensitive topics

Page

16

FGDs Analysis: Analysis

Text Analysis

• QDA Miner

• Relational themes

• Identifying Key themes in FGDs.docx

Conducted with a fairly open framework which allow for focused, conversational, two-way communication

Starts with more general questions or topics. Followed by more specific probing questions

Not all questions are designed and phrased ahead of time

Uses an interview guide rather than a set of questions

Used to:

• Obtain specific quantitative and qualitative information from a sample of the population

• Obtain general information relevant to specific issues, (ie: to probe for what is not known)

• Gain a range of insights on specific issues

Semi-structured interviews; Description

Semi-structured interviews: Advantages and disadvantages

Advantages

Depth and detail

Rapport

Analytic power: Ability to relate data to other data at individual, household, and community levels

No peer pressure

Disadvantages

More time consuming and costly, so smaller sample sizes

No trigger and interaction effects of group

Pag

e

19

Most Significant Change

It is a story-based, qualitative and participatory approach to monitoring and evaluation

Involves the collection of significant change (SC) stories emanating from the field level, and the systematic selection of the most significant of these..

Can be used for different domains of change and provides case studies

„Looking back over the last 1 year, what do you think was the most

significant change in [particular domain of change]?‟

Can used for:

• program evaluation

• organizational review and evaluation.

• building community ownership through participatory evaluation

Most Significant Change

Advantages

Participatory, involves multiple stakeholders

Does not use pre-set indicators and therefore can capture unexpected /unanticipated changes

Disadvantages

Time consuming especially in selecting most significant change stories

Network Analysis

Set of integrated techniques to depict relations among actors and to analyze the social structures that emerge from the recurrence of these relations

Conducted by collecting relational data organized in matrix form.

Actors are depicted as nodes, and their relations as lines among pairs of nodes

Network Analysis: Advantages and Disadvantages

Advantages

Can be done individually or in groups

Data /information can be quantified (UCINET)

Can be done through visual tools (Venn diagrams) or short questionnaire survey

Disadvantages

Limited to evaluation of interactions

Requires multiple software (UCINET for analysis, Netdraw for network maps)

Analysis

Three stages of analysis

• During and post interview analysis (“follow your nose!”

• Field analysis

• Formal analysis

Communication

• Language

• Technical expertise

Field implementation issues

Sample selection

• For groups, sample stratification based on key variables such as gender, socio-economic status

Representation

• For semi-structured and case study interviews, representation of key interest groups

Triangulation

• Most qualitative tools not enough for evaluation by themselves, use of other tools for triangulation

Examples of use of Qualitative tools in evaluation Physical set up

• For group discussions, sitting arrangements, venues can influence discussion outcomes

Discussion guides /interview guides

• Semi-structured or unstructured guides with key issues /questions

Skills

• Good facilitation skills required for qualitative data collection

• Analytic skills

Recording /documentation

INTERNATIONAL FOOD POLICY RESEARCH INSTITUTE

Thank You

INTERNATIONAL LIVESTOCK RESEARCH INSTITUTE