11 feb05 quant-rmd
TRANSCRIPT
Quantitative Methods
Quantitative Methods• Attempts to generalize across some
“population”• Techniques for testing strength of
relationship, probability that it is “true”• Tradeoffs between sample size (which
helps in generalizing, testing), depth, skills, and cost of study
Sampling• Essential for statistical testing, generalizing• Guard against biases• Need to identify “population”• Sampling methods
– Select randomly– Stratify by hypothesized key variables (H/M/T in irrigation)– Multi-stage: e.g. select communities then households– NOT “convenience sample”!!
• These principles are also useful for qualitative studies!! (How representative are your findings?)
What Level of Analysis?• Potential Levels of Interest
– Intra-household (do outcomes differ for different members of the family)
– Household– Particular group(s) within the community?– Community-level outcomes– Groups that function above level of
community?
What Level Data Collection?
• Even if level of analysis is household or intra-household, often very important to collect some data at the community level:– Agro-ecological Data– Distance to Market– Community Infrastructure– Social “Cohesion”– Cooperative Capacity
Alternatively, Even if Community, Might Want Household-Level Data
• Heterogeneity within the community: distribution of wealth (land or livestock holdings), how many with higher education, proportion of different ethnic groups etc.– Potential Difficulty: If you are collecting
household and community-level data, some people are tempted to use “household averages” within a community, but… very few studies include enough households per community so that the sampled households are representative; averages of sampled households can be very misleading.
Data Collection Methods• Individual or household survey• Key informant interview
– NGOs, officials, local leaders, poor women, etc.
• Focus groups
Consider what you can get from each– Private or public knowledge– Norm or exceptions
Mapping Data to MethodsData Key
informantFocus group
Secondary
HH Survey
Water scarcity
Castes Landhold Income WUA particip
WUA structure
Complications• Usually, we have “abstract” concepts
(e.g. social cohesiveness) that require operationalization – How will it be measured, concretely?
• Even if variable at first seems easy… may not be! e.g. – size of landholdings in areas without
titles– crop yield and how much of crops sold – often different and/or non uniform
measures
What to Collect• Preparation is key…Preparation is key… Think through
variables you want to explain AND variables you think are explanatory
• Draw out diagram—what is likely to cause or influence other variables?
• Distinguish between:– Exogenous: externally generated, not
influenced by other variables in the set– Endogenous: those that are influenced
by other variables that you are studying
Exogenous and Endogenous
Resource scarcity
Group size
Castes
Market development
Collective action
Education
User group
Resource outcome
Exogenous and Endogenous• Can’t use an endogenous variable as an
independent variable in a regression– e.g. user association in regression on collective
action• Option 1: Instrumental variables
– Find a proxy for the endogenous variable, that is not influenced by other variables in the system.
• Option 2: Multi-stage analysis– User association as function of exogenous variables– CA (work days) as function of predicted value of
user association & other exogenous variables (not same set of independent variables)
What to Collect: Household Decisions
• Lists are excellent!Endogenous: HH participation in Planting Trees on Common LandMeasure: Number of Hours Contributed (Actual)Exogenous: Opportunity Cost of Adult LaborMeasure(s): Wage rates for Adult Males and Females (Actual), or
Dependency Ratio (Proxy, Adult Females), orTotal Number of Adults (Proxy, Family Labor,
which assumes that labor markets completely
missing)Education (Proxy, Household Data by Adult)
What to Collect, Household Instruments
• After going through the list, go back through the exogenous variables, and critically assess: Is it really exogenous?
• If you have reason to believe that it is not, then must come up with potential variables to use as instruments
• Example: Endogenous: Household Participation in planting trees on common lands/pasturesExogenous: Number of livestock (often used as a measure of “wealth”). Is it exogenous? Probably not. Why? Decision on participation and livestock likely to be a joint – same variables will affect both decisions.Potential Instruments: Stock (number or value) of consumer durables; whether household is from traditionally “livestock-owning” ethnic group, total number of livestock held at formation of household.
What to Collect, CommunityEndogenous: Maintenance of Community IrrigationMeasure(s): Engineer/Other Expert Assessment – Quality Ranking (Actual, outside expert), orNumber of Hours and/or Money Contributed (Actual,Survey of Person In-Charge of Maintenance, perhaps corroborated with household surveys?), orMeasure of number of months functioned in last 12 months (Actual, Survey of Person In-Charge, or Focus Groups? What if information differs across sources?)Exogenous 1: Benefits of MaintenanceMeasures(s): Might assume benefits same across all communities (water for humans and/or animals, the same), so no variable, orAlternative water sources (own tubewells? % of community members, either from group interviews or key informant interview); price of purchased water (either from group interviews, or from shop that sells locally, etc.)
What to CollectEndogenous: Maintenance of Community BoreholesExogenous 2: Heterogeneity within the communityMeasure(s)Ethnic Differentiation (diversity index, or number of groups) Actual, Key Informant and/or Focus GroupsWealth Differentiation (diversity in landholdings, % communitymembers with vehicles or other expensive consumer durables) Actual, Key Informant and/or Focus Groups, possible corroboration with household level data (if enough household data))Difference in Education (proportion of households with at least one member with primary, secondary schooling; Focus Groups but still difficult to get if large community
Complications Specific to Property Rights
• Property Rights…– To a specific resource, or specific land/water
area?– Of a specific person over certain resources
• If so, are the rights complete (to all resources, without condition?) and/or unconditional (always have rights, or does strength of right depend on other factors (e.g. relative rainfall)
• If incomplete or conditional, can characterize in any way that is “quantifiable” or easily comparable across individuals, across communities and regions?
Complications Specific to Collective Action• Measuring “Capacity” to Undertake Collective
Action at the group or community level: Potential Measures: Trust, Social Cohesion,
Agency, History of Success and/or Failure– How do you measure “trust” amongst members and
get an indicator of trust at the level required (the group or the community). Some have tried the following:Question at Household Level: Do you trust immediate family members to watch your children? family friends? Neighbors in the village?Potential Problems: If not enough households sampled, then may not be representative; what if a household has no small children?
Complications Specific to Collective Action
Potential Measures: Trust, Social Cohesion, Agency, History of Success and/or Failure– How do you get a quantifiable, comparable (across
communities) measure social cohesion? Social differentiation often used, but there can be tensions even in communities where members all of same ethnic group and speak same language
– Agency (A. Krishna): even when there is the “will” there is often not the “way”. How do you measure the capacity to transform “social capital” into collective action?•Possibilities: Characteristics of Leadership and/or Entreprenuership.
Complications Specific to Collective Action
Potential Measures: Trust, Social Cohesion, Agency, History of Success and/or Failure
– How do you get a quantifiable, comparable (across communities) measure of the historical experience with collective action, particularly in large, heterogeneous communities?Potential Difficulties with simply “counting” successes and failures:1. Greater capacity may well lead a group to try more things, and more difficult things – so number of failed (or less successful) activities should be greater! Also, implies endogeneity!
2. In practice, people prefer to speak of successes, so data collected likely to be biased
Recall Data• Subjective: Successes and failures; the
human tendency:•The past was always better, generically, or•Difficulty discussing negatives/failures
• Objective: Historical recall data on assets (livestock), rainfall, climate, other shocks
•Nearly impossible to get good data for more than 5 years, even 5 years is very difficult (e.g. livestock holdings),
•Rainfall, as with positive and negative subjective events, people tend to forget how “bad” it was in past. Borana “forgot” 1991-92 drought, which was quite devastating.
•Shocks (pest infestation, human disease, drought): Worst shocks will be those most recently experienced, irrespective of whether more likely to happen again.
• Possible Solution: Wherever possible, get secondary corroborating data, or do very extensive job of triangulating information from multiple sources
• Preferences and Subjective Assessments:– Need to make sure that everyone has the
same “reference point”; e.g.•Those in communities with generally bad soil might rank their own plot as favorable… relative to the generally poor soils in the area; if ask someone in different area, might rank quality of his/her plot as “average”, when in fact is much better because soils generally more favorable
•Same with questions of leadership, for instance. If community used to fairly poor leadership, current leader may be relatively good, but in fact may still be worse than in other community where leadership ranked as poor – since that community may have a different experience and expectations
• Open-Ended, Semi-Structured Interviews: – Enumerator Bias:
•In Burkina Faso, information on waterpoint management perfectly correlated with 1 of 3 enumerators. Enumerator had previous water project implementation experience; his 18 communities reflected his bias in how should be, so no information on how is
•In India, information on individuals’ risk preferences were nearly completely correlated with the enumerator…
Final ThoughtsSome things simply not amenable to
quantification:• Processes (how decisions are made, enforced)• Variables with many “dimensions”; e.g. certain
tenure situations where many multiple, overlapping claims over same resource
• Many subjective assessments – this does NOT mean are not important, it means difficult to construct questions that allow comparable information to be generated across households in many different regions