1 share pretest 2010 feedback & results. 2 schedule

1

SHARE Pretest 2010

Feedback & Results

2

ScheduleDate 1. Generic

QuestionnaireDesign,Programming, Testing

2. SMS anddata transfer

3. Country-SpecificQuestionnaireDesign,TranslationProgramming, Testing

4. Agencies,fieldworkprep,contracts

5. Fieldwork 7.Conferences,Meetings

8. Deadlines

20 – 21J ul 2010

Post-pretestmeeting inOdense

01 – 15Aug 2010

Area Teams,Country Teamsrevise genericand country-specificinstrument onpaper (->version 5)

MEA,CentERdatarevise SMS (->Version 5)

Area Teams,Country Teamsrevise genericand country-specificinstrument onpaper

15 – 31Aug 2010

CentERdataprogramsGeneric Version5

CentERdataprograms SMSVersion 5

01 – 15Sep 2010

MEA, AreaTeams,Country Teamstest GenericVersion 5

MEA, AreaTeams,Country Teamstest SMSVersion 5 (->Version 6)

MEA finishespreloaddatabase formain survey

16 – 30Sep 2010

CentERdataprograms CAPIFinal GenericVersion 6

CentERdataprograms SMSFinal GenericVersion 6

Surveyagencies testpreloaddatabase formain survey

Generic Version6 frozen forMain SurveyTraining

3

Schedule01 – 07Oct 2010

Country teamstranslate finalchanges toCAPI & SMS inLMU

08 – 15Oct 2010

CentERdatacompilestranslatedinstruments

16 – 22Oct 2010

Country-teamstest,implement finalchanges inLMU;CentERdatacompiles finaltranslatedinstruments

22 Oct2010

FinalTranslatedCAPI and SMSVersion 6

27 – 28Oct 2010

TTT inGermany

02 Nov2010 - 30J un 2011

Fourth wave ofSHARE

-- Everybodyworks hard

4

What we've done since Paris

• Further testing, revision and one (major) change of the instrument– Sample Management System– Translation

• IT meeting for agencies (Apr 23)– Concepts, ex ante v ex post harmonization

• TTT for new and old countries (May 25-26)– Organized and conducted by MEA– Mock interviews of new modules/questions– SMS– Preload– Gaining respondent cooperation– Proxy etc.

5

Pretest goals

• 100 realized household interviews– at least 50 refresher households

• Test logistics in all phases of the fieldwork– includes proxy interviews if necessary– includes exit interviews where necessary– included interviews in institutions / nursing homes

where necessary– have drop-off for each respondent

6

Pretest goals

• Testing the CAPI instrument– Do the structural changes to the CAPI work?– In particular new/much changed sections SN and CH– Is the translation/adaptation correct?– Where are we in terms of length?

• Testing the functionality:– SMS Client CAPI instrument– SMS Client SMS Sample Distributor– SMS SD CentERdata

• Get ready for the main survey!

7

Qualitative Feedback

• All feedback received until July 18 (Sunday)

• 7 agencies, 6 countries– DK, ES, EE, FR, BN, IT– also received by now: CH, BF, NL, HU

• Going to forward to area teams– please deal with detailed comments

8

Qualitative Feedback

• Gaining R cooperation– getting harder and harder to convince people to

participate– length of interview does not help– reluctance to answer personal questions to people

living in their community (IT)

9

Qualitative Feedback

• Particular problems to gain cooperation– older R (afraid, easily tired)– working R (too busy)– R living in appartment buildings (difficulty getting to

door)– earlier refusals (i.e. who did refuse SHARELIFE) hard

to convince to be interviewed again– couples (two interviews are just too much in one

evening session)– older Rs often first asked children whether they

should participate --> special info aimed at children?

10

Qualitative Feedback

• General complaints about length, repetitiveness and detail– tiredness– lack of interest– anger

11

Qualitative Feedback

• Specific length problems identified– Introductions are too long, too much blablabla (sic!) -

time robbers (DK, FR)– Smoother way to do "all proxy interview" needed– Mental health module too long– second person refused because of length

• Brochures– helpful but should be more neutral (i.e. not linking low

education and obesity)– third wave results needed

12

Qualitative Feedback

• Preload errors– year of last interview

• What was lacking– questions about personal interests and activities

• Unfolding bracket entry points were unrealistic (FR)

• A comprehensive QxQ should be available before the data collection

13

Qualitative Feedback

• SN:– too early in the interview, resp do not feel comfortable– reluctance to give names/details of friends and relatives without

their permission (interview should be about the R)– people seem to forget people they live with– better shift after CF because it makes it easier to separate

couple Rs– hard to separate couples right at beginning of the interview when

insisted both should be present

• CH:– becomes confusing if too many children with similar

names/nicknames (ES)

14

Qualitative Feedback

• DO– respondents, especially old did not like filling in (ES) – between 10 and 30 minutes (DK)– favorable reaction to have health care questions

(costs) on paper rather than CAPI– people got annoyed, questions seemed to be already

administered– people did not understand why they had to fill in DO

after such a long interview– illiterate respondents

15

Qualitative Feedback

• Measurements:– are generally liked because they bring variation to the

interview (BN)– 3 test are too many (IT)– why can't interviewers choose when to perform test

(like in SHARELIFE)– The preferred section amongst several respondents,

they were very cooperative and interested during the physical tests

16

Qualitative Feedback

• Financial questions:– too difficult, too detailed, too nosy!!! (sic!)– questions are too long; easier to understand when

you read them than when you hear them– respondents do not understand what is the purpose of

questions

17

Data on which numbers are based

• All data received until July 20 (yesterday)

• 12 countries

– no data yet from DE, GR, HU, PT, LU(?)

• 844 household interviews

• Target was 100 hh i'views per country(reached in only 4 countries so far)

18

Number of interviews by country

0

20

40

60

80

100

120

140

AT BE CH CZ DK EE ES FR IT NL SE SI

Single Couple

19

Length

• Excluding all interviews shorter than 20 minutes and longer than 5 hours, the overall average and median length are:– 80 and 76 minutes for single interview, 83 and 79 if

weighted– 130 and 125 minutes for couple interview, 130 and

125 if weighted

• Note: this is pure interview time including interviewer observations but excluding general setup and coverscreen

20

Length

• The averages do not tell us much about the variance– 35% of the total variance is between countries– as always, there are "long" and "short" countries

21

Length by country, single hh

050

100

150

200

min

utes

CH NL AT FR EE BE SE DK ES CZ IT SIexcludes outside values

22

Length by country, couple hh

5010

015

020

025

030

0m

inut

es

CH DK SE AT NL BE FR EE ES CZ SI ITexcludes outside values

23

Length (Pretest v Main)

• Comparison of wave 2 timings between pretest and main survey– Single HH took 1.3 minutes longer in main

than pretest– Couple HH took 4.5 minutes longer in main

than pretest• Changes between main and pretest?

– Added Chair Stand and Peak Flow!– Added CASP and CES-D to AC module– Various minor cuts and additions

24

Length (Pretest v Main)

• Accounting for CS, PF, AC additions– Single HH: 5.9 minutes shorter– Couple HH: 8.3 minutes shorter

• Yet again: huge between-country variance

25

Pretest v Main(single HH - wave 2)

-40

-20

02

04

0M

ain

- P

rete

st

bf at de dk pl it bn es ch nl se grCountry

26

Length by area

• Single / couple hh– Health: 34.8 / 65.6 min– Econ: 23.6 / 42.6 min– Social: 19.8 / 32.0 min

• Note: allocation of modules to areas does not allow for overlaps– e.g. CF, DN, HO

27

Length by module0

510

1520

min

utes

CF EP PH EX HC HO AC MH SN CH BR AS WS GS DN CO PF CS SP FT IV HHexcludes outside values

28

If to cut, where to cut?

• Expecting a too long interview, we have discussed in May with area teams a contigency plan– two round of discussion– designed to take on average 10 minutes off the CAPI

questionnaire

• achieved reasonable degree of agreement on what to cut

• scheduled some of the dropped item for next wave / in-between data collection

29

If to cut, where to cut?

• Health– Drop Walking Speed/Chair stand– Skip HC including end of life care – with the exception of

• HC002 (seen a doctor in last 12 months), • HC003 (seen a GP in last 12 months), • HC004 (seen a specialist in last 12 months), • HC005 (which specialist), HC012 (been in Hospital in the last 12 months), • HC013 (How often in Hospital in last 12 months), • HC014 (total nights in hospital in last 6 months), • HC029 (been in nursing home last 12 months), and the

– Health insurance questions in drop-off?

30

If to cut, where to cut?

• Social– Drop year of birth from SN loop– FT: drop FT004, FT006, FT011, FT013, FT015 etc.

large gifts or inheritances (amount and reason for transfer)

– HO071, HO072, HO073: Quality of neighborhood– SP: return to the single solicitation of help

given/received outside the household (i.e. do not distinguish between personal care and practical help)

31

If to cut, where to cut?

• Econ– EP021, EP022, EP057, EP058: Responsibility for

supervising employees; how many– EP059: Opportunities to work after retirement age– EP065: Retirement been a relief or concern– HO035: Years living in community– HO038, HO039, HO040: Apart from vacations or brief visits,

do you regularly spend part of the year in another residence? Where is this residence located? In Which country

– HO011, HO012: ask only at baseline or if moved house since last wave...

32

If to cut, where to cut?

• Econ– Drop all asset income question (AS005, AS009,

AS015, AS058) – replace by summary question– AS061 reason no bank account only at

baseline– Keep expectations module only for baseline– Drop financial literacy questions from EX

module– Drop detailed ISCO questions

33

If to cut, where to cut?

• Scheduled for next wave / in-between wave data collection:– End of life decisions– Quality of living environment– Prevention– Financial literacy– Walking speed / Chair Stand / Other

measurement?

34

Questions?

?