harmonize or perish! the living standards measurement study

Harmonize or Perish!The Living Standards Measurement Study

Gero CarlettoDevelopment Research Group

WORLD BANK GOALSJim Yong Kim announces new goals End extreme poverty: the percentage of people living with

less than US$ 1.25 a day to fall to 3 percent by 2030

Promote shared prosperity: foster income growth of the bottom 40 percent of the population in every country

Recognition of large data gap75 percent of countries with “updated” household survey

WB GOALS,IMPLICATIONS Some key current data gaps:

Household income or consumption distribution data is up-to-date in about 50% of developing countries

The stated goal: halve the poverty data gaps in developing countries by 2017, ensuring up-to-date data for 75% of all countries (containing virtually all the poor)

WB GOALS & LSMSMore data, but just as important …quality datapolicy-relevant datacomparable data

cross-country and over timeNot only averages, but distributions (shared prosperity)

Comparability of existing consumption data …

Less than one week One week Two weeks One month Greater than one month

Less than or equal to two weeks

0102030405060708090

100

41.0

24.0

5.0 7.0

23.0

70.0

Percent

Reference period

Source: “Assessment of the reliability and relevance of food data …”, Smith et al.

Food purchases Home-produced food consumed In-kind receipts of food All three sources0

102030405060708090

100100.0 96.0

86.0 85.0

Percent

Mode of acquisition

Instrument design & implicationsfor poverty

Beegle, Kathleen, Joachim De Weerdt, Jed Friedman, and John Gibson. 2012. “Methods of Household Consumption Measurement through Surveys: Experimental Results from Tanzania.” J of Development Economics 98: 3-18

Type of consumption module in questionnaire

Mean consumption per cap (Tsh)

Poverty headcount

($1.25/day) Long 7 day recall 520,850 54.9

HH diary infrequent 425,298 55.6

Personal diary 510,616 47.5 Note: Test compared 8 different instruments, varying recall period and method

Heterogeneity in Surveys

• Initial purpose of the survey drives the way survey is designed and implemented

– Different agenda Different instrument

• An increasingly crowded field…

Instrument Sponsor

Censuses UNFPA

Income Expenditure /Budget Surveys (IES/HBS) Central Banks, IMF, NSOs

Labor Force Surveys (LFS) ILO

Demographic and Health Surveys (DHS) USAID

Multiple Indicator Cluster Surveys (MICS) UNICEF

Core Welfare Indicator Questionnaires (CWIQ) UNDP, DfIDWB Africa Reg.

Welfare Monitoring Survey (WMS) Stat Norway

Statistics on Income and Living Conditions (SILC) Eurostat

Comprehensive Food Security and Vulnerability Analysis 9CFSVA)

WFP

Integrated, Multi-Topic Surveys[Living Standards MeasurementStudy (LSMS), Integrated Surveys (IS), Family Life Surveys (FLS)]

World BankRANDNSOs

• The “McNamara Anecdote”• Need to understand living standards, poverty,

inequality and the correlates and determinants of these- not just monitor.

• Unit of analysis is the household, as both a consuming and producing unit

• One survey collecting data on a range of topics is a more powerful tool for policy formulation than a series of single purpose surveys: the sum is greater than the parts– Farmers are diversified– Poverty and FS are multidimensional

The thinking behind the LSMS survey

The thinking behind the LSMS survey (cont’d)

• Demand driven and country-owned• Priority often given to meeting the policy needs of

each country, but with an eye to x-country comparability and accepted standards

• Implications– no standard set of LSMS questionnaires: content, length

and complexity varies by country– Questionnaire development- lengthy process linking data

users, stakeholders and data producers – Capacity building, sustainability

A “typical” LSMS• Consumption-based welfare measure

– Multi-dimensional poverty• Multiple instruments

– HH, Agriculture, Community, Price, Facilities• Strict data quality control

– “Intelligent” data entry, CAPI– Small sample– Pre-coded, closed-ended questions– Training, supervision

• Documentation and Dissemination– Basic Information Document– Consumption aggregate .do files– Publicly available microdata

The LSMS today• Goal: ensure that the LSMS meets new demands for

data and remains at the forefront of survey methodology

• New demand -- new topics• Old topics with new focus (agriculture)• New technologies • Increased standardization• Four areas of focus

– Data collection– Methodological Work– Tools, Resources for researchers/survey practitioners– Training and Dissemination

DATA RESEARCH

AG POVERTY/FS OTHER

SURVEY METHODS ANALYSIS

*Gender*AgNut*Facts+Myths*Migration*Subsidies*Tracking poverty

LSMS-ISATanzaniaUgandaEthiopiaMalawiNigeriaNigerMaliBurkina Faso

Non-LSMS-ISASerbiaHaitiTajikistan …

ADVOCACY/DISSEMINATION

LSMS

OUTREACH

Training (LSMS course, e-learning)

Survey Clinics

Sourcebooks

Technical Assistance

Tools(CLSP, ADePT)

LandSoilInputsSkillsCropsLvstck

*Mig*Labor*Income*Credit

SHWALITASubj. Pov.

Lack of standards result in poor comparability!• Take Food Consumption …

– Diary vs. recall– Household vs. individual– Reference period– Nomenclature (COICOP)– Bulk purchases– Non-standard units of measure– Food consumed away from home (FCAH)– Valuation of consumed own-production

Take diary vs. recall …• Diary often considered ….

– Unfeasible (low literacy rate)– Too onerous for respondents– Too costly– Often, diary converts into short (2-3 day) recall … but no metadata!

• Recall considered imprecise (telescoping, recall bias)• 7-day recall most frequent. Most feasible?

Can we improve on 7-day recall?• Creating a continuum between diary and

recall: “SHWALITA, the sequel”• Bounding reference period• Assisting households to recall• Accounting for bulk purchases (annualization)

Can we improve on 7-day recall?• Creating a continuum between diary and

recall: “SHWALITA, the sequel”• Bounding reference period• Assisting households to recall• Accounting for bulk purchases (annualization)• Food Consumed Away from Home

– Increasing share of total food consumption

Can we improve on 7-day recall?• Creating a continuum between diary and

recall: “SHWALITA, the sequel”• Bounding reference period• Assisting households to recall• Accounting for bulk purchases (annualization)• Food Consumed Away from Home

– Increasing share of total food consumption• Non-standard Units of Measure (CAPI)

20

Non-Standard Units

What about non-food?

• Lack of consistency even in number of components– Imputed rents– User value of durables– Health expenditures– List of 12-month items

What about income?

“The practical and conceptual difficulties of collecting good income data are severe enough to raise doubts about the value of trying”

A. Deaton (1997), p. 30

Income and Consumption• Income (Y) information important for other uses

(besides poverty & inequality)– Livelihood strategies (income shares)– Productivity/efficiency analysis– Net buyer/net sellers, impact of high food prices

• Consumption (C) preferred welfare measure in developing countries– More stable (short-term fluctuations)– Income harder to measure (self-employment)– Less incentive to mis-report

• Y relatively neglected• Some components more troubling than others!

A closer look at Y componentsWage earners

A closer look at Y components (cont’d)

Farm households

Measuring crop production

Measuring crop production• Farmers/HHs don’t keep records• Crops often harvested in small quantities over several months• Mostly consumed• Recall widely used but does not always work• Measured in non-standard units of varying size• Different units along the value chain, different states• Standards do not exist or not feasible

– Need validation

#whatwillittake … to harmonize?• Friction bet/w country ownership/temporal

comparability and x-country harmonization?• Not a DHS but more standardization is possible• Start with inventory of surveys (Olivier)• Mapping and influencing “pipeline”• Agreement on current standards

– Consumption vs. Income– Food consumption

• Method• Reference period• Disaggregation• Nomenclature (COICOP)

– Non-food expenditure components– Imputations

#whatwillittake … to harmonize?

• Enhanced coordination– Some “unflattering” examples …– Clearly defined mandates and responsibilities– Establish forum (IHSN, UNSC,…)

• Methodological research to establish future, improved standards

“If you want to make enemies, try to change

something”  Woodrow Wilson