Methodology of LongitudinalSurveys

Peter LynnDept of Survey Methodology,Institute for Social and Economic Research,University of Essex, UK.

A John Wiley and Sons, Ltd, Publication

Methodology of Longitudinal Surveys

WILEY SERIES IN SURVEY METHODOLOGY

Established in Part by WALTER A. SHEWHART and SAMUEL S. WILKS

Editors: Robert M. Groves, Graham Kalton, J. N. K. Rao, Norbert Schwarz,Christopher Skinner

A complete list of the titles in this series appears at the end of this volume.

Methodology of LongitudinalSurveys

Peter LynnDept of Survey Methodology,Institute for Social and Economic Research,University of Essex, UK.

A John Wiley and Sons, Ltd, Publication

This edition first published 2009

� 2009 John Wiley & Sons, Ltd

Registered office

John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex,

PO19 8SQ, United Kingdom

For details of our global editorial offices, for customer services and for information about how to apply for

permission to reuse the copyright material in this book please see our website at www.wiley.com.

The right of the author to be identified as the author of this work has been asserted in accordance with the

Copyright, Designs and Patents Act 1988.

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or

transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise,

except as permitted by the UK Copyright, Designs and Patents Act 1988, without the prior permission

of the publisher.

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be

available in electronic books.

Designations used by companies to distinguish their products are often claimed as trademarks. All brand

names and product names used in this book are trade names, service marks, trademarks or registered

trademarks of their respective owners. The publisher is not associated with any product or vendormentioned in

this book. This publication is designed to provide accurate and authoritative information in regard to the

subject matter covered. It is sold on the understanding that the publisher is not engaged in rendering

professional services. If professional advice or other expert assistance is required, the services of a competent

professional should be sought

A catalogue record for this book is available from the British Library.

ISBN: 978-0-470-01871-2

Set in 10/12pt Times by Integra Software Services Pvt. Ltd, Pondicherry, India

Printed in the UK by Antony Rowe Ltd, Chippenham, Wilts.

Contents

Preface xv

1 Methods for Longitudinal Surveys 1

Peter Lynn

1.1 Introduction 11.2 Types of Longitudinal Surveys 21.3 Strengths of Longitudinal Surveys 4

1.3.1 Analysis Advantages 41.3.2 Data Collection Advantages 6

1.4 Weaknesses of Longitudinal Surveys 81.4.1 Analysis Disadvantages 81.4.2 Data Collection Disadvantages 9

1.5 Design Features Specific to Longitudinal Surveys 111.5.1 Population, Sampling and Weighting 111.5.2 Other Design Issues 13

1.6 Quality in Longitudinal Surveys 151.6.1 Coverage Error 151.6.2 Sampling Error 161.6.3 Nonresponse Error 161.6.4 Measurement Error 16

1.7 Conclusions 17References 18

2 Sample Design for Longitudinal Surveys 21

Paul Smith, Peter Lynn and Dave Elliot

2.1 Introduction 212.2 Types of Longitudinal Sample Design 212.3 Fundamental Aspects of Sample Design 23

2.3.1 Defining the Longitudinal Population 23

v

2.3.2 Target Variables 242.3.3 Sample Size 252.3.4 Clustering 262.3.5 Treatment of Movers 262.3.6 Stratification 272.3.7 Variances and Design Effects 272.3.8 Selection Probabilities 28

2.4 Other Aspects of Design and Implementation 282.4.1 Choice of Rotation Period and Pattern 282.4.2 Dealing with Births (and Deaths) 292.4.3 Sample Overlap 302.4.4 Stability of Units and Hierarchies 31

2.5 Conclusion 32References 32

3 Ethical Issues in Longitudinal Surveys 35

Carli Lessof

3.1 Introduction 353.2 History of Research Ethics 353.3 Informed Consent 38

3.3.1 Initial Consent 383.3.2 Continuing Consent 393.3.3 Consent to Trace Respondents 393.3.4 Consent for Unanticipated Activities or Analyses 403.3.5 Implications for Consent of Changing Circumstances

of Sample Members 403.3.6 Consent for Linkage to Administrative Data 413.3.7 Using Administrative Data without Full Consent 423.3.8 Can Fully Informed Consent be Realised? 43

3.4 Free Choice Regarding Participation 433.5 Avoiding Harm 463.6 Participant Confidentiality and Data Protection 49

3.6.1 Dependent Interviewing 493.6.2 The Treatment of Research Data 50

3.7 Independent Ethical Overview and ParticipantInvolvement 52Acknowledgements 53References 53

4 Enhancing Longitudinal Surveys by Linking to Administrative Data 55

Lisa Calderwood and Carli Lessof

4.1 Introduction 554.2 Administrative Data as a Research Resource 564.3 Record Linkage Methodology 584.4 Linking Survey Data with Administrative Data at Individual Level 61

4.4.1 Sampling, Sample Maintenance and Sample Evaluation 61

vi CONTENTS

4.4.2 Evaluation Methodology 624.4.3 Supplementing and Validating Survey Data 63

4.5 Ethical and Legal Issues 674.5.1 Ethical Issues 684.5.2 Legal Issues 684.5.3 Disclosure Control 68

4.6 Conclusion 69References 69

5 Tackling Seam Bias Through Questionnaire Design 73

Jeffrey Moore, Nancy Bates, Joanne Pascale andAniekan Okon

5.1 Introduction 735.2 Previous Research on Seam Bias 745.3 SIPP and its Dependent Interviewing Procedures 75

5.3.1 SIPP’s Pre-2004 Use of DI 765.3.2 Development of New DI Procedures 765.3.3 Testing and Refining the New Procedures 78

5.4 Seam Bias Comparison – SIPP 2001 and SIPP 2004 795.4.1 Seam Bias Analysis for Programme Participation

and Other ‘Spell’ Characteristics 795.4.2 Seam Bias Evaluation for Income Amount

Transitions 875.5 Conclusions and Discussion 89

Acknowledgements 90References 90

6 Dependent Interviewing: A Framework and Application

to Current Research 93

Annette Jackle

6.1 Introduction 936.2 Dependent Interviewing – What and Why? 94

6.2.1 Data Quality 946.2.2 Survey Processes 95

6.3 Design Options and their Effects 956.3.1 Reactive Dependent Interviewing 966.3.2 Proactive Dependent Interviewing 97

6.4 Empirical Evidence 996.4.1 Income Sources 1006.4.2 Current Earnings 1046.4.3 Current Employment 1056.4.4 Labour Market Activity Histories 1056.4.5 School-Based Qualifications 106

6.5 Effects of Dependent Interviewing on Data QualityAcross Surveys 107

CONTENTS vii

6.6 Open Issues 109Acknowledgements 109References 110

7 Attitudes Over Time: The Psychology of Panel Conditioning 113

Patrick Sturgis, Nick Allum and Ian Brunton-Smith

7.1 Introduction 1137.2 Panel Conditioning 1147.3 The Cognitive Stimulus Hypothesis 1167.4 Data and Measures 1177.5 Analysis 1187.6 Discussion 123

References 124

8 Some Consequences of Survey Mode Changes in Longitudinal Surveys 127

Don A. Dillman

8.1 Introduction 1278.2 Why Change Survey Modes in Longitudinal Surveys? 1288.3 Why Changing Survey Mode Presents a Problem 130

8.3.1 Changes in Question Structure 1308.3.2 Effects of Visual vs. Aural Communication Channels 1328.3.3 Interviewer Presence 1358.3.4 How Answers to Scalar Questions are Affected by

Visual vs. Aural Communication 1368.4 Conclusions 137

References 137

9 Using Auxiliary Data for Adjustment in Longitudinal Research 141

Dirk Sikkel, Joop Hox and Edith de Leeuw

9.1 Introduction 1419.2 Missing Data 1429.3 Calibration 1449.4 Calibrating Multiple Waves 1479.5 Differences Between Waves 1499.6 Single Imputation 1509.7 Multiple Imputation 1519.8 Conclusion and Discussion 153

References 155

10 Identifying Factors Affecting Longitudinal Survey Response 157

Nicole Watson and Mark Wooden

10.1 Introduction 15710.2 Factors Affecting Response and Attrition 159

viii CONTENTS

10.2.1 Locating the Sample Member 15910.2.2 Contacting the Sample Member 16010.2.3 Obtaining the Cooperation of the Sample Member 16210.2.4 The Role of Respondent Characteristics 164

10.3 Predicting Response in the HILDA Survey 16710.3.1 The HILDA Survey Data 16810.3.2 Estimation Approach 16910.3.3 Explanatory Variables 16910.3.4 Results 171

10.4 Conclusion 178References 179

11 Keeping in Contact with Mobile Sample Members 183

Mick P. Couper and Mary Beth Ofstedal

11.1 Introduction 18311.2 The Location Problem in Panel Surveys 184

11.2.1 The Likelihood of Moving 18511.2.2 The Likelihood of Being Located, Given a Move 187

11.3 Case Study 1: Panel Study of Income Dynamics 19011.4 Case Study 2: Health and Retirement Study 19611.5 Discussion 200

Acknowledgements 201References 202

12 The Use of Respondent Incentives on Longitudinal

Surveys 205

Heather Laurie and Peter Lynn

12.1 Introduction 20512.2 Respondent Incentives on Cross-Sectional Surveys 206

12.2.1 Effects of Incentives on Response Rates on MailSurveys 206

12.2.2 Effects of Incentives on Response Rates onInterviewer-Administered Surveys 207

12.2.3 Effects of Incentives on Sample Composition and Bias 20712.2.4 Effects of Incentives on Data Quality 20812.2.5 Summary: Effects of Incentives 208

12.3 Respondent Incentives on Longitudinal Surveys 20812.4 Current Practice on Longitudinal Surveys 21112.5 Experimental Evidence on Longitudinal Surveys 218

12.5.1 Previous Experiments on UK Longitudinal Surveys 22112.5.2 British Household Panel Survey Incentive Experiment 222

12.6 Conclusion 229Acknowledgements 230References 231

CONTENTS ix

13 Attrition in Consumer Panels 235

Robert D. Tortora

13.1 Introduction 23513.2 The Gallup Poll Panel 23713.3 Attrition on the Gallup Poll Panel 241

13.3.1 Descriptive Analysis 24113.3.2 Experiments 24413.3.3 Logistic Regression 24613.3.4 A Serendipitous Finding: The Relationship Between

Type of Survey and Attrition 24613.4 Summary 248

References 248

14 Joint Treatment of Nonignorable Dropout and Informative Sampling

for Longitudinal Survey Data 251

Abdulhakeem A. H. Eideh and Gad Nathan

14.1 Introduction 25114.2 Population Model 25514.3 Sampling Design and Sample Distribution 256

14.3.1 Theorem 1 25614.3.2 Theorem 2 257

14.4 Sample Distribution Under Informative Sampling and InformativeDropout 257

14.5 Sample Likelihood and Estimation 25814.5.1 Two-Step Estimation 25914.5.2 Pseudo Likelihood Approach 259

14.6 Empirical Example: British Labour Force Survey 26014.7 Conclusions 262

References 263

15 Weighting and Calibration for Household Panels 265

Ulrich Rendtel and Torsten Harms

15.1 Introduction 26515.2 Follow-up Rules 266

15.2.1 Population Definitions 26615.2.2 Samples and Follow-up 268

15.3 Design-Based Estimation 26915.3.1 The Horvitz–Thompson Estimator 26915.3.2 Link Functions 27015.3.3 Convexity and Variance of the Weighted Estimator 273

15.4 Calibration 27415.4.1 Types of Calibration within Panels 27515.4.2 Bias and Variance 276

15.5 Nonresponse and Attrition 278

x CONTENTS

15.5.1 Empirical Evidence Regarding Nonresponse and Attrition 27815.5.2 Treatment via Model-Based Prediction 28115.5.3 Treatment via Estimation of Response Probabilities 281

15.6 Summary 285References 285

16 Statistical Modelling for Structured Longitudinal Designs 287

Ian Plewis

16.1 Introduction 28716.2 Methodological Framework 28816.3 The Data 29016.4 Modelling One Response from One Cohort 29216.5 Modelling One Response from More Than One Cohort 29516.6 Modelling More Than One Response from One Cohort 29716.7 Modelling Variation Between Generations 29816.8 Conclusion 300

References 301

17 Using Longitudinal Surveys to Evaluate Interventions 303

Andrea Piesse, David Judkins and Graham Kalton

17.1 Introduction 30317.2 Interventions, Outcomes and Longitudinal Data 304

17.2.1 Form of the Intervention 30417.2.2 Types of Effects 30517.2.3 Conditions for the Evaluation 30517.2.4 Controlling for Confounders in the Analysis 30617.2.5 Value of Longitudinal Surveys 307

17.3 Youth Media Campaign Longitudinal Survey 30917.4 National Survey of Parents and Youth 31117.5 Gaining Early Awareness and Readiness for

Undergraduate Programs (GEAR UP) 31417.6 Concluding Remarks 315

References 315

18 Robust Likelihood-Based Analysis of Longitudinal Survey Data

with Missing Values 317

Roderick Little and Guangyu Zhang

18.1 Introduction 31718.2 Multiple Imputation for Repeated-Measures Data 31818.3 Robust MAR Inference with a Single Missing Outcome 32018.4 Extensions of PSPP to Monotone and General Patterns 32318.5 Extensions to Inferences Other than Means 32418.6 Example 325

CONTENTS xi

18.7 Discussion 328Acknowledgements 330References 330

19 Assessing the Temporal Association of Events Using Longitudinal

Complex Survey Data 333

Norberto Pantoja-Galicia, Mary E. Thompson and MiloradS. Kovacevic

19.1 Introduction 33319.2 Temporal Order 334

19.2.1 Close Precursor 33419.2.2 Nonparametric Test for Close Precursor 335

19.3 Nonparametric Density Estimation 33519.3.1 Kernel Density Estimation 33619.3.2 Local Likelihood Approach 337

19.4 Survey Weights 34019.4.1 Assessing the Standard Error 341

19.5 Application: The National Population Health Survey 34119.5.1 Pregnancy and Smoking Cessation 34219.5.2 Subsample 34219.5.3 Interval-Censored Times 34219.5.4 Results 343

19.6 Application: The Survey of Labour and Income Dynamics 34419.6.1 Job Loss and Separation or Divorce 34519.6.2 Subsample 34519.6.3 Interval-Censored Times 34519.6.4 Results 346

19.7 Discussion 347Acknowledgements 348References 348

20 Using Marginal Mean Models for Data from Longitudinal Surveys

with a Complex Design: Some Advances in Methods 351

Georgia Roberts, Qunshu Ren and J.N.K. Rao

20.1 Introduction 35120.2 Survey-Weighted GEE and Odds Ratio Approach 35420.3 Variance Estimation: One-Step EF–Bootstrap 35620.4 Goodness-of-Fit Tests 357

20.4.1 Construction of Groups 35820.4.2 Quasi-Score Test 35820.4.3 Adjusted Hosmer–Lemeshow Test 360

20.5 Illustration Using NPHS Data 36120.5.1 Parameter Estimates and Standard Errors 36220.5.2 Goodness-of-Fit Tests 363

20.6 Summary 364References 365

xii CONTENTS

21 A Latent Class Approach for Estimating Gross Flows in the Presence

of Correlated Classification Errors 367

Francesca Bassi and Ugo Trivellato

21.1 Introduction 36721.2 Correlated Classification Errors and Latent Class Modelling 36821.3 The Data and Preliminary Analysis 37121.4 A Model for Correlated Classification Errors in Retrospective

Surveys 37321.5 Concluding Remarks 378

Acknowledgements 379References 379

22 A Comparison of Graphical Models and Structural Equation Models

for the Analysis of Longitudinal Survey Data 381

Peter W. F. Smith, Ann Berrington and Patrick Sturgis

22.1 Introduction 38122.2 Conceptual Framework 38222.3 Graphical Chain Modelling Approach 38322.4 Structural Equation Modelling Approach 38422.5 Model Fitting 38522.6 Results 38622.7 Conclusions 390

Acknowledgements 391References 391

Index 393

CONTENTS xiii

Preface

Longitudinal surveys have become highly valued by researchers and policy makers fortheir ability to provide insights that cannot be obtained by any other means. Thecontribution of longitudinal surveys to social justice agendas, for example, is now wellestablished. In this era when the realities of anthropogenic climate change have finallyachieved broad acknowledgement, study of the determinants and impacts of individual-level behavioural change has become of crucial significance to our future survival.Longitudinal surveys should come into their own in this and other arenas.

While the successful implementation of a longitudinal survey can be extremelyrewarding, there are considerable complexities involved in designing and carrying outa longitudinal survey over and above those that apply to other surveys. Survey metho-dologists have been studying these complexities and developing better methods for years,but the results are scattered about throughout the general survey methods, statistics andindeed social science literature. A notable exception – a book devoted to methods forlongitudinal surveys – is the Wiley book Panel Surveys, which was edited by DanielKasprzyk, Greg Duncan, Graham Kalton and the late M. P. Singh and published in1989. That book contained monograph papers presented at the InternationalSymposium on Panel Surveys held in November 1986 in Washington, DC. While thevolume has remained helpful to survey researchers, there have been important changesand developments in the field in the past two decades.

In 2004 I proposed the organisation of a conference that would in some sense be asuccessor to the 1986 Symposium. The idea was that the conference would bring togetherresearchers interested in the methodology of longitudinal surveys from around the worldand that a series of invited monograph papers would form the edited volume that younow have in front of you. The proposal was considered by the Steering Group of the UKLongitudinal Studies Centre (ULSC), a centre located at the Institute for Social andEconomic Research (ISER), University of Essex. The remit of the ULSC includes theadvancement of methodology and the promotion of best practice for longitudinalsurveys, so the conference and book seemed a good fit. The Steering group agreed andthe decision was taken to adopt the conference and the book as a project under theauspices of the ULSC.

The next steps were to assemble both a scientific committee to pull together the pro-gramme for the conference and a local organising committee to make all the arrangementsnecessary for a successful international conference. The scientific committee consisted ofthe following people: Roeland Beerten (Office for National Statistics, UK), Paul Biemer(Research Triangle Institute and University of North Carolina at Chapel Hill, USA),

xv

Nick Buck (ISER, University of Essex, UK), Marco Francesconi (Dept of Economics,University of Essex, UK), Stephen Jenkins (ISER, University of Essex, UK), GrahamKalton (Westat, USA), Femke De Keulenaer (University of Antwerp, Belgium – nowGallup Europe, Belgium), Peter Lynn (Chair: ISER, University of Essex, UK), GadNathan (Hebrew University of Jerusalem, Israel), Cheti Nicoletti (ISER, University ofEssex, UK), Randy Olsen (Ohio State University, USA), Ian Plewis (Institute ofEducation, University of London, UK – now University of Manchester, UK), UlrichRendtel (Freie Universitat Berlin, Germany), Joerg-Peter Schraepler (DIW Berlin,Germany) and Mark Wooden (University of Melbourne, Australia). I am grateful to allof them for their role in screening submitted abstracts and shaping the programmes of bothmonograph and contributed papers for what was to become known as the InternationalConference on the Methodology of Longitudinal Surveys, orMOLS 2006 for short.

The conference took place on the campus of the University of Essex in July 2006 andattracted over 200 delegates from 26 countries, with 85 papers presented. The localorganising committee consisted of five members of the ISER: Randy Banks, AnnFarncombe, Peter Lynn (Chair), Emanuela Sala and Janice Webb. Randy was respon-sible for creating and maintaining the conference website, which was well-received byparticipants and continues to provide a valuable archive of presented papers (www.iser.essex.ac.uk/ulsc/mols2006). Janice took charge of bookings, accommodationarrangements, meals and a host more besides. She will be well known to many of theconference participants, having helped many of them with tasks as diverse as travelarrangements, payments, dietary requirements and finding their way across theUniversity of Essex campus. Janice’s professionalism and dedication to the job wasinstrumental to the success of the conference.

The role of these ISER staff in organising and administering the conference was madepossible by the support of the ULSC. The ULSC is funded by the Economic and SocialResearch Council with support from the University of Essex and I am deeply grateful toboth organisations for ultimately making this whole thing possible.

Gratitude is also due to many others: the University of Essex Conferences Office fortheir efficient role in the local arrangements on campus; the then Vice-chancellor ofthe University, Professor Sir Ivor Crewe, who generously hosted a welcome reception ina marquee in the campus grounds; the deputy mayor of Colchester, councillorRay Gamble, who hosted a civic reception in Colchester Town Hall and attended theconference dinner in Colchester’s historic Moot Hall; and Colchester Borough CouncilMuseums Service who organised guided tours of Colchester Castle Museum for con-ference participants.

On the day before the conference commenced, two associated short courses were held,each of which was attended by more than 40 participants. The courses were ‘Handlingincomplete data in longitudinal surveys’, presented by Joop Hox and Edith de Leeuw,and ‘Multilevel modelling for longitudinal survey data’, presented by Sophia Rabe-Hesketh and Anders Skrondal. The courses benefited from funding by the ESRCNational Centre for Research Methods (www.ncrm.ac.uk).

With the conference over, attention turned to producing this monograph volume. Thestrength of the book lies in the quality of the chapters, which in turn reflects theexperience, knowledge and energy of the chapter authors. I am grateful to the thirty orso researchers around the world who reviewed the draft chapters. All the chapters weresubsequently presented in monograph sessions at MOLS 2006 and further benefited

xvi PREFACE

from the insightful comments of knowledgeable discussants. The Editor’s role has beento mould the chapters into a coherent whole, a task made easier by the responsivenessand cooperation of the authors and, above all, by the professionalism of the staff atWiley, particularly Kathryn Sharples, the commissioning editor who showed faith in theproject, Susan Barclay, project editor, who showed commendable patience and restraintin her regular reminders regarding my tardy delivery, and Beth Dufour, content editor,who skillfully guided the book through its final stages. Any weaknesses of the bookremain solely the responsibility of the Editor. In addition to supporting the productionof the book, Wiley sponsored a prize for best student paper at MOLS 2006. This wasawarded to Mario Callegaro of the University of Nebraska-Lincoln (now at KnowledgeNetworks Inc.), with runner-up Hari Lohano of the University of Bath, UK.

The task of editing this book inevitably impinged considerably on family life andI wish to acknowledge the forbearance of Elisabeth, Henry and Adrian who had toooften to put up with me being inseparable from my laptop.

The result of this effort I hope is worthwhile. The book covers the range of issuesinvolved in designing, carrying out and analysing a longitudinal survey. In discussingways to improve longitudinal surveys, chapters draw upon theories and ideas frompsychology, sociology, statistics and econometrics. As well as this multidisciplinarity,an international flavour can be detected in the volume. Though the book’s UK rootsmay be apparent, the branches reach out to provide glimpses of survey practice inCanada, USA, Australia, Germany and The Netherlands. The community of surveymethodology researchers is truly global and the community specialising in methods forlongitudinal surveys is pretty small in most countries, making international communica-tion and collaboration particularly valuable. Such collaboration is now continuingthrough the biennial International Workshop on Panel Survey Methods, which was insome sense spawned by MOLS 2006. These workshops are relatively small, informalgatherings of like-minded methods researchers willing to share and discuss research. Thefirst took place at the University of Essex in July 2008 (www.iser.essex.ac.uk/seminars/occasional/psmw2008) and the second is planned for Spring 2010 in Mannheim,Germany. I hope that the friendly international collaboration and spirit of commonpurpose that I have experienced in putting together this book and in the conference andworkshops will continue in future research and publication ventures.

This volume aims to be of particular value to researchers involved in commissioning,designing or carrying out longitudinal surveys, as well as to those planning the analysisof longitudinal survey data. It may also serve as a useful secondary text for students ofresearch methods. I hope that it might additionally help to chart out the knownunknowns and hence to stimulate further methodological research.

Peter Lynn

Colchester, October 2008

PREFACE xvii

CHAPTER 1

Methods for Longitudinal Surveys

Peter LynnUniversity of Essex, UK

1.1 INTRODUCTION

A longitudinal survey is one that collects data from the same sample elements on multipleoccasions over time. Such surveys are carried out in a wide variety of contexts and for awide variety of purposes, but in many situations they have considerable analyticaladvantages over one-time, or cross-sectional, surveys. These advantages have beenincreasingly recognised and appreciated in recent years, with the result that the numberof longitudinal surveys carried out has multiplied. This growth of interest has occurredin the government, academic and private sectors.

For example, in the UK the Economic and Social Research Council (ESRC) hasinvested heavily in enhancing the country’s already impressive portfolio of academiclongitudinal surveys. The three existing long-term birth cohort studies that began life in1946, 1958 and 1970 (Ferri et al., 2003) have been joined by a fourth, the MilleniumCohort Study (Dex and Joshi, 2005), which started collecting data in 2001 – and maysoon be joined by a fifth. And an ambitious new household panel survey, UnderstandingSociety: the UK Household Longitudinal Study, was commissioned in 2007 and beginsdata collection in January 2009 (www.understandingsociety.org.uk). With a targetsample size of 40 000 households and a large boost of ethnic minority households thissurvey represents the largest single investment in academic social research resources everlaunched in the UK. Meanwhile, the UK government has also initiated major newlongitudinal surveys. The Longitudinal Study of Young People in England (LSYPE)started in 2004, involving annual interviews with a sample of over 15 000 young peopleborn in 1989–1990 and their parents (http://lsype.notlong.com). The Wealth and AssetsSurvey entered the field in 2006 and plans to have biennial waves with a sample of over30 000 persons. TheOffending, Crime and Justice Survey (OCJS) began in 2003, with themain component being a longitudinal survey of young people aged 10–25, with annualinterviews (Roe and Ashe, 2008).

1

Methodology of Longitudinal Surveys P. Lynn

� 2009 John Wiley & Sons, Ltd

This expansion in longitudinal surveys has also been visible in the InternationalArena. The Survey of Health, Ageing and Retirement in Europe (SHARE) began in2004 with a first wave of data collection with a sample of over 30 000 individuals aged50 or over in 11 countries (Borsch-Supan and Jurges, 2005). Further waves took place in2006 and 2008. The European Union Statistics on Income and Living Conditions (EU-SILC) is a panel survey of households in all countries of the EU that began in 2004 andfeatures a four-wave sample rotation pattern with annual interviews (http://eu-silc.notlong.com). The international suite of broadly comparable household panel surveys,which for years comprised only the UK, USA, Germany and Canada, has more recentlybeen joined by Switzerland (1999), Australia (2001), New Zealand (2002) and SouthAfrica (2008).

Reflecting this burgeoning interest in longitudinal surveys, and recognising that suchsurveys have distinct methodological features, this book aims to review the current stateof the art in the design and implementation of longitudinal surveys. All aspects of thesurvey process are considered, from sample design through to data analysis and includ-ing overarching issues such as ethics and data linkage. This chapter aims to provide anoverview of the features of longitudinal surveys that are distinct from other surveys and anomenclature and framework within which to consider the other chapters.

1.2 TYPES OF LONGITUDINAL SURVEYS

The definition of a longitudinal survey presented in the first sentence of this chapterencompasses a multitude of different survey designs that are used in practice.Longitudinal surveys can vary on a number of key dimensions that tend to haveimplications for various aspects of methodology. These include features of the surveyobjectives, such as the study topics and the study population of interest (Binder, 1998),and features of the design such as the interval between waves, the mode of data collec-tion, the treatment of new entrants to the population, and so on (Kalton and Citro,1993). Some of the different types of longitudinal surveys are outlined below. This is notintended as a comprehensive typology, but merely to give a flavour of the variety thatexists:

� Surveys of businesses carried out by national or regional statistical offices. Thesesurveys tend to collect a limited range of information, typically restricted to keyeconomic indicators. Data may be collected at frequent intervals, such as monthlyor quarterly, and the main objectives are usually to publish regular series of statisticson totals, means and net change between periods, often for cross-cutting domainssuch as regions and industries. The rationale for a panel design is often to improve theprecision of estimates of net change rather than an interest in estimates of grosschange or any other micro-level longitudinal measures;

� Surveys of school-leavers, graduates or trainees. Institutions offering education ortraining, such as universities, or government agencies responsible for related policy,often wish to assess the outcomes of such education or training at a micro (student)level. These outcomes are often medium term or long term and consequently it isnecessary to keep in touch with students/trainees for some time after they havecompleted their study. Longitudinal surveys are often used for this purpose, collecting

2 METHODS FOR LONGITUDINAL SURVEYS

data from students/trainees on several occasions, perhaps beginning while they arestill students/trainees and for up to several years after they have completed the course.The information collected is often quite complex, including perhaps full employmentand activity histories between each survey wave and maybe also reasons for changesand decisions made. Sometimes, one or more ‘control’ groups may be included in thesurvey in an attempt to assess the impact of the education/training;

� Household panel surveys. In several countries, long-term panel surveys of the generalhousehold population are carried out. The oldest, the Panel Survey of IncomeDynamics (PSID) in the USA, has been interviewing the same people since 1968(Duncan et al., 2004). These surveys are multitopic and general purpose, collectingbehavioural, attitudinal and circumstantial data on a range of social and economicissues. The main objective is to provide a rich data resource to be used by a wide rangeof users for a broad set of purposes. The data structure is complex, involving inter-views with each person in the household of each sample member at each wave, inaddition to household-level information and, often, additional survey instrumentssuch as self-completion questionnaires or health measurements;

� Birth cohort studies. As already referred to in Section 1.1 above, theUKhas a suite of fourlong-term birth cohort studies. Similar studies are carried out in many other countries.For some, the sample consists of all the births in a particular location (often a country)over a particular short time period such as a week. For others, such as the MilleniumCohort Study, a sample is taken of all births over a longer period, such as a year.Many ofthese studies beganwith a focus on eithermaternal health or child development (or both),but were later continued into the sample members’ adult life when the value of doing sobecame apparent. Data about the sample members are typically collected from themother in the first years of the study and later on from the sample members themselves;

� Epidemiological studies. Longitudinal medical studies often follow up samples ofpeople over periods of time in order to study the progression of a disease or otherchanges in health, wellness and illness. The samples often consist of patients who havepresented with a particular condition, either at a particular hospital or across abroader area such as the area of jurisdiction of a health authority. Some of thesestudies collect data through interviews or self-completion questionnaires while otherscollect only medical data via tests and diagnoses;

� Repeated-measures studies. This is a term found in the literature referring to studiesthat repeatedly measure the same variables in the same way on each sample unit. Inpractice, most longitudinal surveys of the types outlined above include some repeatedmeasures, though they may vary in the proportion of the questionnaire or interviewitems that are repeated measures. For example, most items collected on householdpanel surveys are repeated (though not necessarily at every wave) in order to build uphistories of economic activity, health status and so on (though an interesting devel-opment over the last decade or so is the recognition that simply repeating an item inidentical fashion at every wave may not be the best way to obtain accurate measuresof change or stability – see Chapters 5 and 6). Similarly, epidemiological studies takethe same measurements repeatedly in order to chart disease progress. On the otherhand, birth cohort studies often have very few repeated items, at least during thesample members’ childhood years, as the questions that are appropriate to ask andthe ways of testing development are somewhat age-specific. Surveys of school leaversor graduates tend to contain a mix of repeated items and wave-specific items;

TYPES OF LONGITUDINAL SURVEYS 3

� Panels and cohorts. The description above makes a distinction between householdpanel surveys and birth cohort studies. In fact the distinction is somewhat artificialand in practice consists only of differences of emphasis both in design and insubstantive focus. A household panel contains samples of every birth cohort – butof course the sample of each cohort tends to be much smaller than would be foundin a birth cohort study that concentrates on just a single cohort. And samples from aseries of birth cohort studies can be combined to provide a picture of the situation ofpeople of different ages at any one point in time. At the extreme, if a new birthcohort study were started every year, say, with a sample of births over a one-yearperiod, then the combination of the studies would have the same population coverageas a household panel survey. The design focus of birth cohort studies, with a samplewho are all very close to one another in age, tends to be associated with substantiveresearch topics that are age-specific, such as growth and development. But thereis no reason in principle why such topics could not be addressed with a householdpanel survey design, with age-specific survey instruments. The advantage would bethat every age cohort is covered, providing some kind of population representation.The disadvantage is that in practice it would be impossible for any narrow age cohortto be afforded the kinds of sample sizes that are typically extant in birth cohortstudies.

The connections between the two kinds of surveys may be even more obvious if weconsider the case of surveys of graduates or school leavers. These typically involvesamples from a one-year age group or stage group and therefore constitute anotherkind of cohort survey. Some of these surveys are carried out only once every n years,where nmay take a value like 5 or 10. But some do indeed take place every year and thus,over time, they in principle cover the entire population of persons who have experiencedthe particular school system or educational establishment that is being sampled.

1.3 STRENGTHS OF LONGITUDINAL SURVEYS

Longitudinal surveys present a number of options, both for data collection and foranalysis, that are either simply impossible with cross-sectional surveys or cannot beachieved in a satisfactorily accurate or reliable way. Often these features are key elementsof the rationale for carrying out a longitudinal survey. However, it is important todistinguish between longitudinal surveys and longitudinal data. Longitudinal surveysare a source of longitudinal data, as the resultant data include items that refer to differentpoints in time. But there are also other ways of obtaining longitudinal data, includingdiary methods and the use of retrospective recall within a single survey instrument. Webriefly discuss here the strengths of longitudinal data, but our particular focus is on thestrengths of longitudinal surveys.

1.3.1 Analysis Advantages

It is of course artificial to separate analysis advantages of longitudinal surveys from datacollection advantages, as the reason for collecting a certain type of data is in order to beable to carry out certain types of analyses. The key advantages of longitudinal data

4 METHODS FOR LONGITUDINAL SURVEYS

(which in most cases can only be accurately or adequately collected by longitudinalsurveys) are analytical:

� Analysis of gross change. Analysis of gross change is perhaps one of the most commonobjectives of longitudinal surveys. Repeated cross-sectional surveys can be used toestimate net change, for example change in the proportion of employees who reg-ularly cycle to work. But only a longitudinal survey can identify the extent to whichthis is composed of different elements of gross change. For example, suppose that theproportion of employees who regularly cycle to work is estimated to be the same attwo points in time. It would be of interest to know whether it is exactly the same set ofemployees who cycle on both occasions or whether there are equal and opposite flowsinto and out of regular cycling (and, if so, how large they are and what the individualcharacteristics associated with each flow are, etc). These are the kinds of questionsthat longitudinal surveys can address.

� Analysis of unit-level change. Individual-level change can be of interest independentlyof interest in population-level net change. For example, analysts may wish to under-stand the characteristics and circumstances surrounding divorce regardless ofwhether there is net change in the proportion of people who are married.

� Aggregate measures. There may be a desire to derive measures that combine observa-tions frommultiple time points. An example might be combining 12 measurements ofmonthly expenditure to obtain an estimate of annual expenditure.

� Measures of stability or instability. Another reason for combining observations frommultiple time points is to providemeasures of stability or instability. Often individual-level change can only be well interpreted in the context of changes over a considerableperiod of time.With amultiwave panel that collects measures of income at each wave,it should be possible to identify people with different patterns of change, such assteady growth, fluctuation around a low level, sudden decline followed by stability,etc. The characteristics associated with such patterns are likely to be of considerableinterest to policy makers. Patterns such as these can only be identified using accuratedata referring to lengthy time periods. For example, poverty analysts have usedhousehold panel data to demonstrate that there is considerable instability over timein the poverty status of many individuals and households in Westernised countries(Jenkins and Rigg, 2001). While the proportion of households in poverty may remainrelatively stable over time, there may be many entrants to and exits from poverty.A large proportion of households may experience at least one spell of poverty over along period of time, while very few households may remain continuously in povertythroughout the period. This insight provided by longitudinal surveys may haveshifted the policy focus from (stable) characteristics associated with poverty propen-sity at a point in time to better understanding poverty dynamics and the factorsassociated with falling into poverty or persistently failing to exit from poverty.

� Time-related characteristics of events or circumstances. Often characteristics such asfrequency, timing or duration are of central interest to researchers. For example,understanding the duration of spells in a particular state and the factors associatedwith exiting from the state (sometimes referred to as ‘persistence’) are important forpoverty, unemployment, marital and partnership status, participation in educationand training, company profitability and many other topics. Hazard modelling andsurvival analysis are techniques used to better understand the propensity for change

STRENGTHS OF LONGITUDINAL SURVEYS 5

(in any status of substantive interest) and the factors associated with such change.These techniques require longitudinal data.

� Identifying causality. Policy makers are ultimately interested in understanding whatcauses certain changes to occur. Most policies are designed to bring about change ofsome kind or other, but often informed by only limited knowledge of how a changedpolicy might have an impact on the desired outcomes. Thus, a key objective ofresearchers is to shed light on causality rather than mere association. Analysis ofthe ordinal nature of events, which requires longitudinal data, often sheds light onissues of causality. For example, a cross-sectional survey can establish an associationbetween factors A and B. But a longitudinal survey might establish that, for mostpopulation units that have experienced bothA and B, A happened before B, making itrather more likely that A caused B than vice versa (though of course a third factor, C,may have caused both A and B and this and other possibilities must always beconsidered). Chapter 17 discusses ways in which longitudinal surveys can help tounderstand causality.

A number of recent advances in methods for analysing longitudinal survey data arethe topics of Chapters 16, 18, 19, 20, 21 and 22.

1.3.2 Data Collection Advantages

There are several ways in which longitudinal surveys provide advantages in terms of datacollection. These are mostly connected with either the quantity or quality of data thatcan be collected compared with alternatives such as the use of retrospective recall. Inmany situations, the advantages are so clear that most researchers would conclude that alongitudinal survey is the only worthwhile option. In other situations, the advantagesmay be marginal. Additionally, there are certain specialised situations in which a long-itudinal survey may have cost or logistical advantages over repeated cross-sectionalsurveys, even though the latter may in principle also be able to provide adequate datato address the research objectives.

Length of HistoriesIt is possible to collect much longer continuous histories of events and transitions with alongitudinal survey than could be collected retrospectively in a single interview, simplydue to the volume of data involved (and hence the length of the interview orquestionnaire).

Accuracy of RecallLongitudinal surveys are often able to obtain more accurate data than could be collectedin a single interview with retrospective recall if the data might be subject to recall error.This advantage accrues primarily because of the ability to limit the reference period overwhich respondents must recall information to a relatively short recent period – such asthe past month, quarter or year – while ultimately obtaining data that relate to muchlonger periods. The reference period often equals the interval between waves, if the aim isto build up continuous histories. Recall accuracy tends to be associated with events andcircumstances that are rare, salient and recent (Tourangeau et al., 2000). For events andcircumstances that do not have these characteristics, data recalled over long time periods

6 METHODS FOR LONGITUDINAL SURVEYS

may be subject to severe recall bias. Recall error can take a number of forms. Events canbe forgotten completely, they can be misplaced in time or misclassified, or associatedcharacteristics may be misremembered. For example, most survey respondents are likelyto be able to recall with reasonable accuracy the date(s) when they got married or hadchildren, even if those events took place decades ago, but they are rather less likely toremember accurately when or how many times they visited a doctor, even if only askedabout the past year.

BoundingFormany purposes, accurate dating of events is at least as important as accurate recall ofthe details of the event. But respondents may not be able to recall accurately the date of aspecific event, even if they can recall the event itself. Consequently, retrospective recallquestions asked in a single interview may produce biased estimates of frequencies andassociated measures. A commonly reported phenomenon is ‘telescoping’, wherebysurvey respondents report events as having taken place within a reference period whenin fact they took place longer ago. This is the result of errors in dating increasing thelonger ago the recalled event took place (Rubin and Baddeley, 1989). Panel surveys havean extra advantage when collecting dates of events. Each interview after the first isbounded by the previous interview, so any events reported previously can be discountedfrom the reports in the current interview in order to avoid telescoping. This of courseassumes that it can be unambiguously concluded whether or not reports in two con-secutive interviews refer to the same event. Sometimes this is difficult, particularly whena respondent has a tendency to experience frequent events of a similar nature, but formany types of survey data it can usually be achieved.

Between-respondent VariationThe difficulty of any recall task will vary between respondents depending on the number,nature and timing of events they have experienced. This variation can be very consider-able in the case of economic activity histories for example, causing a dilemma for surveydesigners. If a survey aims to collect complete activity histories over several years for asample of people who are likely to vary greatly in their experiences, such as a cross-section of the general population, the ideal interval between survey waves will be verydifferent for different sample members. But it is rarely possible to predict this in advance,nor is it often practical to have different between-wave intervals for different individuals.Instead, a standard interval is chosen. Interviews at annual intervals may be inefficientfor persons whose circumstances change little (e.g. retired persons or those who remainin the same job for many years). The marginal amount of information collected in eachinterview, relative to the considerable cost, will be small. But annual interviews mightpresent a considerable recall challenge to persons who experience large numbers of shortspells of employment, perhaps interspersed with spells of unemployment or otheractivities. The advantage of longitudinal surveys in dealing with this challenge is that alot of relevant information is known about sample members in advance of each wave(apart from wave 1). This information can be used to predict the likely complexity ofrecall tasks in various domains and tailored questioning strategies can be adopted. Forexample, those respondents who experience the highest frequency of changes in aparticular domain may be administered a short telephone interview between waves inorder to update information and thereby shorten the reference period. Or a group for

STRENGTHS OF LONGITUDINAL SURVEYS 7

which particularly low levels of change are observed might be administered an extramodule of questions on a different topic in order to maximise the value of the datacollection exercise. Such tailored approaches can have disadvantages and will not alwaysbe advisable, but they are options that are not available to cross-sectional surveys. Thus,to gainmaximum benefit from the ability of longitudinal surveys to collect longer and/ormore accurate histories of events, the survey designer needs to understand the recall andreporting task being asked of respondents, how it relates to between-wave interval andhow this relationship might vary over sample members.

Expectations and IntentionsSubjective measures such as expectations, intentions, attitudes and reasons for makingparticular choices may be particularly prone to recall error. Though some researchershave retrospectively collected attitudes (Jaspers et al., 2008) or reasons for makingdecisions (De Graaf and Kalmijn, 2006), others believe that attempting such measure-ment is simply not worthwhile, in particular because respondents’ memory of such thingsis likely to be tainted by subsequent experiences and post hoc rationalisation may leadrespondents to reassess their earlier motivations or expectations (e.g. Smith, 1984).Longitudinal surveys are able to collect information about expectations and reasonsfor choices untainted by subsequent events and outcomes. Later waves can of course alsocollect information about the relevant subsequent events and outcomes. This enablesanalysis of associations between, say, expectations and outcomes, that would simply notbe possible with any other data source.

Maintenance of Samples of Rare PopulationsFor certain rare study populations, the cost of assembling a representative sample maybe very high relative to the cost of collecting survey data from the assembled sample. Thismay be the case for populations that are not identified on any available sampling frameand for which it is therefore necessary to carry out a large-scale population screeningexercise (Kalton and Anderson, 1986). In such cases, it may be cost-efficient to carry outrepeated surveys with the same sample rather than select a new sample for each newstudy of the same population. The motivation in this case, therefore, is not a need forlongitudinal data but the result is nevertheless the use of longitudinal data collectionmethods.

1.4 WEAKNESSES OF LONGITUDINAL SURVEYS

1.4.1 Analysis Disadvantages

Longitudinal surveys are often not as good as cross-sectional surveys at providing cross-sectional estimates. This may be perceived as a weakness, but it is simply not somethingthat longitudinal surveys are designed for. Compared with estimates from a cross-sectional survey, cross-sectional estimates from a longitudinal survey (from wave 2onwards) may bemore likely to suffer from coverage error if the sample does not include,or under-represents, additions to the population since the sample was selected. Thiscoverage error may increase over waves as the time since sample selection, and hence theundercoverage rate, increases. The design of a longitudinal survey can often be adjusted

8 METHODS FOR LONGITUDINAL SURVEYS

to improve the quality of cross-sectional estimates that can be made, though this may beresource-intensive and may detract from the central longitudinal aims of the survey.Such design options are discussed in Chapter 2. Also, a longitudinal survey sample maysuffer from a lower net response rate than a cross-sectional survey on top of the lowercoverage rate (though lower coverage and response rates do not necessarily imply greatererror: this depends on the association between the coverage and response propensitiesand key survey estimates).

A difficulty with longitudinal analysis may be the analysis-specific nature of thepopulation to which estimates refer. This can create problems in understanding andcommunicating effectively the nature of the reference population and consequentdifficulties of inference. Definitions of study populations are discussed further inSection 1.5 below.

1.4.2 Data Collection Disadvantages

There are some aspects of survey data collection that are unique to longitudinal surveysand potentially detrimental or problematic.

Panel ConditioningPanel conditioning refers to the possibility that survey responses given by a person whohas already taken part in the survey previously may differ from the responses that theperson would have given if they were taking part for the first time. In other words, theresponse may be conditioned by the previous experience of taking part in the survey. Thistherefore relates to all data collected by longitudinal surveys other than those collected atthe first wave. There are two ways in which conditioning can take place. The way inwhich respondents report events, behaviour or characteristics might change; or theactual behaviour might change.

For example, a two-wave survey of unemployed persons might find that more peoplereport a particular type of job search activity at the second wave than at the first wave.This might reflect a genuine increase in the extent to which that activity takes place(independent of taking part in the survey) but it could also be affected by panelconditioning. This could be because the first wave interviewmade some sample membersaware of possible job search activities in which they were not currently engaged, so theysubsequently started doing those things. Thus, there was a genuine increase in the extentof the activity, but only amongst sample members – not amongst the population as awhole. The behaviour of the sample members has been conditioned by the experience ofthe first interview. Alternatively, the experience of the first interviews may have affectedthe way in which some sample members respond to the questions in the second interview,even though their actual job search behaviour may not have changed. Perhaps in the firstinterview they discovered that reporting no activity of a particular type led to them beingasked a series of questions about why they did not participate in that activity. So, tomake the second interview shorter, or to avoid embarassing questions, they now reportthat they have participated in this particular activity. In this case, the reporting of thesample members has been conditioned by the experience of the first interview.

In the above example the conditioning was caused by having been asked the samequestion previously, but in other cases the conditioning could be caused merely by havingbeen interviewed previously. Questions that were not asked previously can therefore also

WEAKNESSES OF LONGITUDINAL SURVEYS 9

be affected. One process by which this might happen is that respondents could build up agreater sense of trust of the interviewer and the survey organisation as time passes. If askedsomequestions on a sensitive topic at wave 3, say, theymight then bemorewilling to revealtruthful answers than they would have been at wave 1. This example also illustrates thatsome forms of conditioning can result in improvements to data quality. It is not the casethat conditioning always damages quality. Another way in which quality might improveover waves of a panel is if respondents learn that it is useful to prepare for the interview byassembling certain documents. Examples of such documents might be pay slips, utilitybills, health records or examination results, depending on the survey topics.

Panel conditioning is more likely to occur for certain topics and certain types ofquestions than for others, meaning that the extent to which the researcher needs to beconcerned about panel conditioning – and the measures that might be introduced tominimise any negative consequences of it – should depend on the survey content. Panelconditioning in the context of attitudinal measures is discussed in Chapter 7.

Sample AttritionSample attrition (also referred to as panel attrition) refers to the continued loss ofrespondents from the sample due to nonresponse at each wave of a longitudinal survey.The term attrition is usually used to refer to a monotone process whereby samplemembers can change from being respondents to nonrespondents but not vice versa. Infact, many longitudinal surveys continue to attempt to collect data from sample mem-bers after they have been nonrespondents at one or more waves, and are often successfulin doing so. Generally, then, the issue is that many sample members may have beennonrespondents at one or more wave, whether or not their response pattern is one ofmonotone attrition. The response rate at any one wave of a longitudinal survey may bejust as good as that for any other survey but after, say, five waves the proportion ofsample units that have responded at every wave may be quite low. Thus, the effectiveresponse rate for longitudinal analysis – for which data from every wave is required –may be lower than the response rates that we are used to having on cross-sectionalsurveys. After several waves there is also a risk that responding sample sizes can becomeunacceptably small. Minimising sample attrition is consequently a major preoccupationof researchers responsible for longitudinal surveys. The factors affecting sample attritionare discussed in Chapter 10. Chapters 11, 12 and 13 discuss various strategies forminimising attrition while Chapters 14, 15 and 16 address issues of how to deal appro-priately with attrition at the analysis stage.

Initial and Ongoing CostsThe time dimension of a longitudinal survey brings with it a set of extra tasks for thesurvey organisation. In particular, the design and planning work that must take placeprior to wave 1 is considerably greater than that required prior to a cross-sectionalsurvey. The wave 1 instrument(s) cannot be designed independently of the instrumentsfor later waves as it is the combination of data from each wave that will provide thenecessary survey estimates. Sample design and fieldwork planning must take intoaccount the need to follow sample members throughout the life of the survey. Anintegrated survey administration system is needed so that both field outcomes andsurvey data from each wave can determine the actions to be taken at later dates (thesemay include things like keeping-in-touch exercises – see Chapter 11 – and dependent

10 METHODS FOR LONGITUDINAL SURVEYS