The Role of Over-Sampling of the Wealthy in the SCF

Arthur B. KennickellFederal Reserve Board

Opinions are those of the presenter alone and do not necessarily reflect the views of the Federal Reserve Board or its staff.

Thanks!

• Organizers, esp. Andrea Brandolini

• FRB, SOI and NORC colleagues

• Interviewers

• Respondents

LWS Process

• Create conceptual categories

• Create “harmonized” (procrustean?) variables for a number of countries– Not everything is measured everywhere– Some things are measured differently

Apples vs. Oranges

Kumkwats vs. Tangerines

Harmonize Samples?

• Samples are bedrock of a survey– Determines what a survey represents– Basis of scientific inference– For surveys with a common population

definition, representational differences should be ones of efficiency

• What if definition is different?

• Sample vs. participants– Nonresponse probably not purely random– Compensating adjustments to weights differ

Fruit Baskets

Understand and Document

• Differences may be due to structural differences in economies– Institutions– Behavior

• Differences may be artifacts of different types of measurement– Questions in surveys (instruments)– Survey samples and nonresponse

Theorem

• There is never an acceptable substitute for thinking about what you are doing.

• Corollary: Don’t count on remembering the details: document, document, document.

Survey of Consumer Finances

• Designed to measure wealth– Detail on the “Primary Economic Unit”– Summary information on other units in the household– Framed approach to measurement– Disaggregated information on assets and liabilities,

with supporting detail– Strong emphasis on instrument development, training

and quality control

• Leave it to other exercises to evaluate the equivalizing of variables in LWS– Focus on sample

SCF Sample

• Dual-frame design– Area-probability and List Samples

• Area-probability sample– Geographically based sample of addresses– Key elements of stratification to achieve balance– Multi-stage, with clusters (tracts) as penultimate stage– Postal address sequences usually used to order the final stage– All units selected with equal probability

– Robust coverage of the nation and good representation of behaviors that are broadly distributed

List Sample

• List sample is a sample of named units (individuals/couples) used to identify households

• Based on and selected from statistical records derived from tax returns by Statistics of Income (IRS)

• Two-stages of selection– Accepts high-level geographic selections of AP sample

• Otherwise, entirely independent– Selected using “wealth index” derived from blend of two models

using income and associated data• One, a simple grossing-up of capita income• Other based on modeling of measured wealth in preceding survey• Multiple years of income data to smooth fluctuations

• Over-samples wealthy units• Robust coverage of the wealthy, but not whole

population

Role of Over-Sampling

• Increases efficiency of estimates affected by upper tail of the wealth distribution– Makes possible study of relationships that

would be too thinly represented in an AP sample alone

• Means of detecting and correcting for nonresponse bias

Evaluating Over-Sampling

• Compare AP sample alone with AP + list sample• Use AP weights calibrated to key population

margins• Use combined sample weights with full

calibrations in the separate and combined samples– LS determines the shape of the upper tail of the

wealth distribution

• Allows direct evaluation of the role of type of over-sampling used in the SCF

LS as % Combined Samples, 2004

0

10

20

30

40

50

60

70

80

90

100

All >=$100K >=$500K >=$1M >=$5MWealth group

% o

f g

rou

p

Narrowly Held Assets

• Of approximately 400 observations with direct holdings of bonds, only ~10% were AP cases

• Of approximately 1,500 observations with direct holdings of publicly traded stocks, only ~37% were AP cases– ~25% of weighted total value attributable to

AP sample

Shares of Total Net Worth, 2004

0

5

10

15

20

25

30

35

40

45

50

0-50 50-90 90-95 95-99 99-100

Percentiles of net worth

Pe

rce

nt

AP only

AP+LS

Nonresponse

• Present in virtually all surveys• If distribution of a variable for nonrespondents

differs from that for participants bias• Wealth surveys cover a particularly sensitive

topic: respondent resistance• Systematic components to nonresponse

– E.g., NR increases with capital income and decreases with age and charitable contributions

• Central area of research for SCF– LS provides means of correction

(AP+LS)-AP, 2004

NET WORTH

$250/1.9%

$3,500/3.9%

$13,600/4.3%

$2.7M/74%

Effect on Wealth Distribution

• High-wealth list sample cases “displace” top of the AP wealth distribution downward

• Similar pattern if add synthetic case to AP sample with weight of 1% of population and wealth at 99 percentile of combined distribution

• Actual effects more subtle

Special Difficulties

• More complex interviews– Requires more sophisticated instrument– Training more important

• More difficult to contact and gain cooperation– “Gatekeepers”– Interviewers often have an incentive to avoid difficult

cases– More expensive

• Same problems without an oversample, but much less visible

Thanks