the development of a successful household panel survey: the hilda experience mark wooden project...

The Development of a Successful Household Panel Survey:

The HILDA Experience

Mark WoodenProject Director, HILDA Survey

www.melbourneinstitute.com

About HILDA: Introduction

Funded and owned by Australian Government Multi-purpose survey

– Modelled on other household panels – BHPS, SOEP

Survey manager = Melbourne Institute of Applied Economic & Social Research (University of Melbourne)

Fieldwork subcontractor = Roy Morgan Research Unit record data available (under license) Want to know more?

Articles in The Economic Record, June 2007 and Australian Economic Review, September 2010

www.melbourneinstitute.com/hilda/


About HILDA: Key Design Features

Commenced (in 2001) with national probability sample of households– Area-based clustered / stratified sample design

Annual survey waves Follow all original hh members and offspring indefinitely Sample augmented with hh joiners Interview all “adults”

– Face-to-face where possible– CAPI / CATI technology

Refreshment (top-up) sample added in wave 11 Cash incentives paid


Survey Instruments

Household Form– Key identifiers / Changing HH membership / HH relationships /

Reasons for non-response

Household Questionnaire– Collects hh level data from relevant HH member

Continuing Person Questionnaire– All persons 15+ who have previously been interviewed

New Person Questionnaire– All persons 15+ who have never previously been interviewed

Self-completion Questionnaire– All interview respondents; 16 pp, expanded to 20 from W5


What’s In It? HQ / CPQ Core:

– Child care– Housing– Education– Employment status– Job characteristics– Job search– Calendar– Income– Family formation– Partnering & relationships– Living in Australia

• Disability, Life satisfaction, Spatial mobility, Caring

– Tracking– Interview situation

Special “modules”:– W1 (+NPQ) = Personal history– W2 = Wealth– W3 = Retirement– W4 = Youth issues;

Private health insurance– W5 = Family formation– W6 = Wealth– W7 = Retirement; Lifestyle– W8 = Family formation; Non-

cores. relationships– W9 = Health– W10 = Wealth– W11 = Family formation;

Retirement– W12 = Skills & abilities; Non-

cores. relationships


What’s In It? SCQ Health and well-being (SF36, Kessler 10, serious health conditions)

Health behaviours (smoking, drinking, exercise, height / weight, diet)

Social capital / relationships (satisfaction with family, social support, community participation, religion)

Neighbourhood characteristics Life events Time use Finances (stressful financial events, savings habits, risk preference, h’hold expend)

Job attributes Parenting (parenting stress / work family gains and strains)

Attitudes to work / gender roles / marriage Personality


Indicators of Success

We are still going!– Funded renewed until wave 16– And total funding has increased

Response / attrition rates are good to excellent

Data usage is high Strong evidence of validity


Response in 2001 was good

HH response– In-scope sample = 11,693– 7682 cooperating households = 66% RR

Individual response– W1 individual sample = 15,127 persons – 13,969 respondents = 92% RR

Sample reasonably representative, but …– Sydney residents under-represented– People from a NESB under-represented– Males less likely to complete a PQ

(but no less likely to be a CSM)


Response in 2011 was better

HILDA Wave 1 (2001)

HILDA Top-up (2011)

USoc: UKHLS (2009-10)

SOEP H (2006)0

10

20

30

40

50

60

70

80


Retention is High(Annual Re-interview Rates: HILDA, BHPS & GSOEP)

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 2575

80

85

90

95

100

BHPS*

GSOEP AB

Wave

%


Retention is High(Annual Re-interview Rates: HILDA, BHPS & GSOEP)

2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 2575

80

85

90

95

100

HILDA

BHPS*

GSOEP AB

Wave

%


Fieldwork Outcomes: W1 Adults

W2 W3 W4 W5 W6 W7 W8 W9 W100%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Dead

Overseas

Lost

NR - not issued

NR - non-contact

NR - contact

Respondent


Tracking Movers

22-23% of all hh’s change address b/w each survey wave

Pre-field office activity– Notifications (1800#, change of address card, email)– Matching to Australia Post– Returns to sender– Move indicator variable

Other household members Contact information collected at previous ivw Neighbours Other community resources Online White Pages


Minimising Refusals

MARKETING / RESPONDENT ENGAGEMENT PAL and brochure, newsletter / Stat report 1800 numberPERSISTENCE 2-3 stage fieldwork NRs re-issued in later wavesGOOD PEOPLE Selection and continuity of interviewers Training / interviewer engagementRESPONDENT INCENTIVES


Data User Numbers

Release Total data orders New users Cumulative total

1 204 204 202

2 265 169 373

3 279 157 530

4 329 176 706

5 387 196 902

6 401 176 1078

7 455 199 1277

8 431 125 1402

9 500 141 1543

10 (@19 July) 426 132 1675


Publication Count

YearJournalarticles

Books /book chapters

Other publications

Workingpapers

2002 5 0 0 3

2003 6 2 2 8

2004 24 4 8 15

2005 24 3 8 21

2006 25 1 19 23

2007 35 0 11 35

2008 38 0 23 35

2009 47 7 27 35

2010 52 6 20 30

2011 64 0 36 42

2012 / forthcoming 52 0 17 14

TOTAL 372 23 171 261


Promoting Data Use

Well-documented, user-friendly data sets User Manual Other on-line tools (e.g., PanelWhiz) Discussion Papers / Technical Papers User training and panel data analysis courses Biennial research conference Membership of CNEF Presentations to different stakeholders Annual Statistical Report Study-specific web site User email list


Research Uses: Key Features

Topic coverage extremely broad


Research Topics: Just a Few Examples!

Income and wealth– Poverty dynamics– Distribution of household wealth– Retirement savings

Labour supply / Unemployment

– LFP and health– Family policy and couples LS– Impact of child care costs– Forgone earnings of mothers

Employment– Working hours mismatch– Casual employment transitions– Part-time employment and wages– Job insecurity– Responses to long hours– Gender inequity

Marriage and family– Patterns of cohabitation– Children’s living arrangements– Post-separation contact with children– Childlessness– Predictors of marital separation

Subjective well-being– Adaptation to life events– Predictors / correlates of life

satisfaction Mental health and:

– welfare reliance– retirement– housing affordability– joblessness– job quality


Research Uses: Key Features

Topic coverage extremely broad Three key types of studies

i. Innovative content / questions

ii. Unobserved heterogeneity

iii. Dynamics of change (and persistence)

Still many cross-sectional analyses


Policy Impacts: Examples

Key input into Government’s Pension Review Annual Wage Reviews RBA

– Household debt and risk– Effect of the superannuation guarantee on

household saving Productivity Commission – Paid Parental

Leave report


Keys to Success: Response

Expectations of fieldwork agency Motivated interviewer workforce Long fieldwork period Persistence Cash incentives


Keys to Success: Other Ingredients

Champions (and lots of them) Money (and lots of it) Imitation Good people Many users Luck


Other Issues To Think About (I)

Sample– Population, dwellings, households– Clustered / stratified– Dealing with future immigration

What mode?– Interviewer administered vs self-administered– Single mode vs mixed mode or multi-mode

Respondent burden How to reach non-English speakers?


Other Issues To Think About (II)

Making use of technology– Dependent data / On-line options

How much value adding?– Data cleaning / Weights / Imputation / Derived

variables Confidentiality vs data access Linkages to admin. data Scientific stewardship / Stakeholder

involvement

The Development of a Successful Household Panel Survey:

The HILDA Experience

Mark WoodenProject Director, HILDA Survey

the development of a successful household panel survey: the hilda experience mark wooden project...

Documents