11

ESCAP/WHO Pilot Study Overview

Wei Liu

Statistics Division, ESCAPliuw@un.org

Sixth Meeting of the Washington Group on Disability Statistics, Kampala, Oct 10-13, 2006

22

•

1.1. ObjectiveObjective2. Background and methodology2. Background and methodology3. Questions tested3. Questions tested4. Analysis and results4. Analysis and results5. Moving forward5. Moving forward

Outline Outline

33

ObjectivesObjectives

Comparing the Washington Group 6-Comparing the Washington Group 6-question “short set” with a WHO “long question “short set” with a WHO “long set” of questionsset” of questions

Studying the relative importance of Studying the relative importance of different domainsdifferent domains

Considering differences in underlying Considering differences in underlying concepts and phrasings of questionsconcepts and phrasings of questions

44

BackgroundBackground

Pilot countries: Pilot countries: Fiji, Indonesia, India, Mongolia, Fiji, Indonesia, India, Mongolia, Philippines Philippines

Test Design:Test Design:

Study One: Testing sensitivity and specificity Study One: Testing sensitivity and specificity

Study Two: Test-Retest ReliabilityStudy Two: Test-Retest Reliability

Study Three: Linguistic Evaluation & Cognitive Study Three: Linguistic Evaluation & Cognitive interviewing interviewing

Other Partners:Other Partners: Australian Bureau of Statistics (ABS Australian Bureau of Statistics (ABS))

55

Sample SelectionSample Selection

India:India: Meerut District, Uttar Pradesh State Meerut District, Uttar Pradesh State

Indonesia:Indonesia: Central Jakarta Central Jakarta

MongoliaMongolia: Ulaanbaatar: Ulaanbaatar

Philippines:Philippines: A selection of urban and rural A selection of urban and rural

geographic “clusters”geographic “clusters”

Fiji:Fiji: 2 rural and 8 urban enumeration areas in 2 rural and 8 urban enumeration areas in Central Division (capital city & surrounding Central Division (capital city & surrounding areas)areas)

66

Sample SizesSample Sizes

FijiFiji IndiaIndia IndonesiaIndonesia MongoliaMongolia PhilippinesPhilippines

Study 1Study 1

10021002 10001000 800800 10621062 10571057

Study 2Study 2

100100 100100 100100 100100 100100

Study 3Study 3

100100 100100 100100 100100 100100

77

Sample Summary: Sample Summary: Age StructureAge Structure0

10

20

30

40

01

02

03

04

0

20 40 60 80 100 20 40 60 80 100 20 40 60 80 100

India Indonesia Philippines

Fiji Mongolia Total

Pe

rcen

t

ageGraphs by country names

88

Pilot Study PacketsPilot Study Packets

Study Study InstructionsInstructions

Questionnaire Questionnaire Version AVersion A Version BVersion B

Cognitive Cognitive Interview Interview ScheduleSchedule

99

Support MaterialsSupport Materials TranslationTranslation

GuidelinesGuidelines List of key termsList of key terms Linguistic evaluation Linguistic evaluation

formform

Interview Guidelines & Q-Interview Guidelines & Q-by-Q specificationsby-Q specifications

Disability Statistics Disability Statistics Training Manual - DraftTraining Manual - Draft

Quality assurance Quality assurance standards & guidelinesstandards & guidelines

Data entry programData entry program

1010

Questions TestedQuestions Tested

Question set 1: Question set 1: Washington Group’s general disability measures, 6 QsWashington Group’s general disability measures, 6 Qs

Question set 2:Question set 2: -World Health Survey body structure/functions -World Health Survey body structure/functions

domains, 17 Qs; domains, 17 Qs;

-WHO Disability Assessment Schedule II (WHO DAS -WHO Disability Assessment Schedule II (WHO DAS II), activity and participation domains, 36 Qs; II), activity and participation domains, 36 Qs;

Australian Bureau of Statistics, need-for-assistance Australian Bureau of Statistics, need-for-assistance domains, 4 Qsdomains, 4 Qs

http://www.unescap.org/stat/meet/widsm4/disabilty_questionnaire_Ver.A.pdf

1111

Study 1: Sensitivity and Study 1: Sensitivity and Specificity of WG Q’sSpecificity of WG Q’s

n=4512n=4512 SensitivitySensitivity Specificity Specificity

““Difficulty…seeing, even if wearing Difficulty…seeing, even if wearing glasses” (W1), compared to:glasses” (W1), compared to:

“…“…Seeing a person across the road” (B1.6)Seeing a person across the road” (B1.6)59%59% 90%90%

“…“…Seeing an object at arm's length” (B1.7)Seeing an object at arm's length” (B1.7)

62%62% 90%90%““Difficulty…hearing, even if using a Difficulty…hearing, even if using a hearing aid” (W2) , compared to:hearing aid” (W2) , compared to:

“…“…Hearing someone talking on the other Hearing someone talking on the other side of the room in a normal voice?side of the room in a normal voice? ” (B1.8)” (B1.8)

39%39% 98%98%

“…“…Hearing what is said in quiet room” Hearing what is said in quiet room” (B1.9)(B1.9) 38%38% 96%96%

1212

Study 1 (cont.)Study 1 (cont.)

Walking or climbing steps (W3) * Walking a Walking or climbing steps (W3) * Walking a long distance >1km (D2.5)long distance >1km (D2.5) 51%51% 91%91%Remembering or concentrating (W4) * Remembering or concentrating (W4) * Remembering to do important things (D1.2)Remembering to do important things (D1.2) 48%48% 92%92%

Self care (W5) * Washing whole body Self care (W5) * Washing whole body (D3.1)(D3.1) 50%50% 98%98%Self care (W5) * Getting dressed (D3.2)Self care (W5) * Getting dressed (D3.2)

56%56% 98%98%Self care (W5) * Need for assistance: Self Self care (W5) * Need for assistance: Self Care (E1)Care (E1) 84%84% 98%98%Communicating (W6) * Understanding what Communicating (W6) * Understanding what people say (D1.5)people say (D1.5) 32%32% 95%95%

1313

Study 2: Kappa Statistics for Study 2: Kappa Statistics for Test-retest ReliabilityTest-retest Reliability

W1 SeeingW1 Seeing 0.570.57

W2 HearingW2 Hearing 0.540.54

W3 Walking & climbingW3 Walking & climbing 0.640.64

W4 Remembering & concentratingW4 Remembering & concentrating 0.590.59

W5 Self CareW5 Self Care 0.710.71

W6 CommunicatingW6 Communicating 0.530.53

B1.1 PainB1.1 Pain 0.600.60

B1.2 DiscomfortB1.2 Discomfort 0.530.53

B1.5 Using hands & fingersB1.5 Using hands & fingers 0.580.58

B1.6 Seeing (short sighted)B1.6 Seeing (short sighted) 0.650.65

B1.7 Seeing (long sighted)B1.7 Seeing (long sighted) 0.530.53

B1.8 Hearing (far)B1.8 Hearing (far) 0.580.58

B1.9 Hearing (close)B1.9 Hearing (close) 0.380.38

B1.12 Short of breath (rest)B1.12 Short of breath (rest) 0.480.48

B1.13 Short of breath (exercise)B1.13 Short of breath (exercise) 0.520.52

1414

Study 2 (cont.)Study 2 (cont.)

B1.14 CoughingB1.14 Coughing 0.470.47

B1.15 SleepingB1.15 Sleeping 0.490.49

B1.17 Worry or anxietyB1.17 Worry or anxiety 0.500.50

D1.1 ConcentratingD1.1 Concentrating 0.550.55

D1.2 RememberingD1.2 Remembering 0.550.55

D1.5 Understanding what people sayD1.5 Understanding what people say 0.420.42

D1.6 maintaining conversationD1.6 maintaining conversation 0.420.42

D2.1 Standing 30min.D2.1 Standing 30min. 0.600.60

D2.2 Standing upD2.2 Standing up 0.580.58

D2.3 Moving insideD2.3 Moving inside 0.430.43

D2.4 Getting out D2.4 Getting out 0.600.60

D2.5 Walking >1kmD2.5 Walking >1km 0.590.59

D3.1 WashingD3.1 Washing 0.550.55

D3.2 DressingD3.2 Dressing 0.620.62

1515

Study 3: Cognitive DebriefingStudy 3: Cognitive Debriefing

Most questions are understood wellMost questions are understood well 95% of the respondents had no difficulties 95% of the respondents had no difficulties

responding to questionsresponding to questions

Minor difficulties with questions on: Minor difficulties with questions on: Mental disability (i.e. cognition, anxiety, Mental disability (i.e. cognition, anxiety,

getting along with people, etc.)getting along with people, etc.) Need-for-assistance questions Need-for-assistance questions W6 communication questionW6 communication question

TranslationTranslation

1616

Identifying “Core Domains”Identifying “Core Domains”

ESCAP/WHO 4ESCAP/WHO 4thth Workshop on Workshop on Improvement of Disability Improvement of Disability Statistics and Measurement Statistics and Measurement (Bangkok, June, 2006)(Bangkok, June, 2006)

Factor analysis researchFactor analysis research

1717

Factor LoadingsFactor Loadings

DisabilityDisability

Factor 1Factor 1 Factor 2Factor 2 Factor 3Factor 3 Factor 4Factor 4 Factor 5Factor 5 Factor 6Factor 6

Life Life ActivityActivity HearingHearing Self CareSelf Care ParticipationParticipation MobilityMobility VisionVision

1818

Domain SelectionsDomain Selections

Factor AnalysisFactor Analysis

Life ActivitiesLife Activities HearingHearing Self CareSelf Care ParticipationParticipation MobilityMobility VisionVision

44thth Bangkok Workshop Bangkok Workshop RecommendationRecommendation

Life Activities Life Activities HearingHearing Self CareSelf Care CommunicationCommunication MobilityMobility VisionVision

1919

Moving ForwardMoving Forward

Further analysis of “core” domains Further analysis of “core” domains To select/reconsider the phrasings of To select/reconsider the phrasings of

the questions for the selected domainsthe questions for the selected domains Follow-up pilot studies for census Follow-up pilot studies for census

recommendations and for an recommendations and for an “extended” question set for use in “extended” question set for use in surveys surveys

Regional training and advisory Regional training and advisory missionsmissions

2020

Thank You!Thank You!

www.unescap.org/stat/