methods to improve the efficiency of screening for...

Methods to improve the efficiency of screening for multiple mental disorders

Phil Batterham Centre for Mental Health Research

The Australian National University

Hierarchical screening

2

General psychological distress

Internalizing disorders

Depression Anxiety

Social phobia Generalized

anxiety Panic disorder

Externalizing disorders

Alcohol abuse Substance abuse Adult ADHD


3

Psychological distress (e.g. K6)

Depression (PHQ-9)

GAD (GAD-7)

Social phobia (SOPHS-5)

Panic disorder

(PHQPAN-5)

Finish 4 disorders, 26 items


4

First-phase screening approaches 1. No hierarchy (control)

2. K6 score

3. Psychological distress decision tree

4. Disorder-specific decision tree

5. Gating items

Method 1: No hierarchy (control)

5 5

Depression (PHQ-9)

GAD (GAD-7)


Panic disorder (PHQ-Panic-5)

Method 2: K6 hierarchy

6 6

K6 score

<5

STOP

≥5

Full screen

Depression (PHQ-9)

GAD (GAD-7)


Panic disorder (PHQ-Panic-5)

Method 3: Distress decision tree

7 7

• Choose distress items that best discriminate absence of disorder

• Subgroups least likely to have disorder escape further screening

• Chi-Square Automatic Interaction Detection (treedisc macro in SAS)


8 8

Chi-Square Automatic Interaction Detection (CHAID) • Divides the sample into subsamples with

different risks of outcome

• Diagram with leaves and branches

• Categorical items

• Branching based on item that best differentiates on the basis of the outcome

• Smallest p-value from a chi-square statistic


9 9

Chi-Square Automatic Interaction Detection (CHAID) • Splitting stops when:

– There is a small number of observations in a leaf (20 observations)

– No split would result in a significant 2 value (=.2)

– A specified level of branching is reached (6 levels)


10 10

K7 (depressed)

MWBS9 (close to others)

K18 (tense/shaky)

FULL SCREEN

0 1 3,4

K18 (tense/shaky)

2

K11 (good mood)

K11 (good mood)

K18 (tense/shaky)

MWBS11 (decisive)

K8 (restless)

FULL SCREEN

FULL SCREEN

STOP (n=23)

STOP (n=95)

FULL SCREEN

STOP (n=14)

FULL SCREEN

K2 (nervous)

FULL SCREEN

K19 (angry)

FULL SCREEN

FULL SCREEN

STOP (n=37)

0,1

2,3,4 2,3,4 0,1

0

3,4 0,1,2

0-2 3,4

4 0-3 0 1-4

0,1 2,3,4

0 1-4

MWBS2 (useful)

FULL SCREEN

1-4

2,3,4

0,1

K1 (so sad...)

STOP (n=11)

K3 (restless)

FULL SCREEN

STOP (n=10)

0 1 2,3,4

2,3,4 0,1

Method 4: Disorder decision tree

11 11

PHQ-2 (down/

depressed)

SOCPH-4 (suffered social

occasions)

SOCPH-5 (social fear disruption)

PHQPAN-1 (anxiety attack)


GAD7-1 (nervous/ anxious)

GAD7-1 (nervous/ anxious)

PHQ-1 (little

interest)

FULL SCREEN

FULL SCREEN

FULL SCREEN

STOP (n=425)

FULL SCREEN

STOP (n=52)

FULL SCREEN

GAD7-3 (worrying)

FULL SCREEN

FULL SCREEN

0 1 2,3

1,2 3,4,5

1

2,3

Yes No

No 3,4

4

Yes

0,1 2,3

0,1 2,3

0,1

3,4

FULL SCREEN

FULL SCREEN

PHQ-1 (little

interest)

FULL SCREEN

2


STOP (n=189)

FULL SCREEN

No Yes

0,1 2,3

0,1

Method 5: Gating items

12 12

PHQ-9 Items 1 & 2

PHQ-9 Items 3-9

GAD-7 Item 1

GAD-7 Items 2-7

PHQ-panic Item 1

PHQ-panic Items 2-5

SOPHS Item 1

SOPHS Items 2-5

END

PHQ1 ≥ 2 or PHQ2 ≥ 2 PHQ1<2 and PHQ2<2

GAD1 > 2 GAD1 < 2

PHQ-pan1 = “No” PHQ-pan1 = “Yes”

SOPHS1 ≤ 2 SOPHS1 > 2

Testing the hierarchies

• Efficiency

– Mean number of items presented

• Precision – Sensitivity relative to control

13

Validation samples

• Two community-based samples

• N1 = 1360; N2 = 668

• Recruited through Facebook ads

• Australia-wide, 18+

14

Sample 1 (N=1360)

15

0% 10% 20% 30% 40%

18-24

25-29

30-39

40-49

50-59

60 and over

Refused

M

F

O/R

0%

10%

20%

30%

40%

50%

60%

Metropolitan Regional Rural Remote Refused

Sample 2 (N=668)

16

0% 10% 20% 30% 40%

18-24

25-29

30-30

40-49

50-59

60 and over

Refused

M

F

O/R

0%

10%

20%

30%

40%

50%

60%

Metropolitan Regional Rural Remote Refused

Samples: Psychopathology

17

MDD 34%

GAD 26%

PD 22%

SP 7%

MDD 25%

GAD 20%

PD 14%

SP 16%

None: 63% None: 53%

N = 1360 N = 668

(Not to scale)

Results: Efficiency gains

18

22.0 21.8

18.4

14.6

11.8

0

5

10

15

20

25

No hierarchy K6 hierarchy Psychologicaldistress items

Disorder-baseditems

Gating items

Me

an it

em

s p

rese

nte

d

1% 16% 34% 46%

Results: Projected efficiency gains

19

22.0

18.2

16.2

13.3

10.1

0

5

10

15

20

25


Disorder-baseditems

Gating items

Me

an it

em

s p

rese

nte

d

17% 26% 40% 54%

Results: Summary

20

95.7%

99.7% 99.9% 100.0%

90%

92%

94%

96%

98%

100%


Disorder-baseditems

Gating items

Sen

siti

vity

Results: Summary

• Two-phase hierarchical screening was efficient and precise

• Using gating items had most efficiency gain (up to 54%)

• Using decision trees also had large efficiency gains (up to 40%)

• K6 did not improve screening efficiency

21

Considerations

• The K6/K10 were designed to “rule in” not “rule out”

• Hierarchical screening works better for:

– Low rates of psychopathology

– Longer screening scales (60% fewer items)

• Tested with other disorders/outcomes

– PTSD, adult ADHD, alcohol abuse, suicidality

22

Considerations

• Purpose of screening

• Brevity vs. need for data

• Ease of implementation vs. efficiency

– Gating only works for scales with gated scoring criteria

– Pencil and paper vs. computer-based

23

Future research: Adaptive screening

• Fully adaptive measures

– Each response determines next item presented

• PROMIS measures

– IRT-calibrated item banks

– PROMIS-depression 5-item adaptive screener more precise than 20-item CES-D

24


25

From: Pilkonis PA, et al. Item Banks for Measuring Emotional Distress From the Patient-Reported Outcomes Measurement Information System (PROMIS®): Depression, Anxiety, and Anger. Assessment 2011 18: 263-283


26

From: Pilkonis PA, et al. Item Banks for Measuring Emotional Distress From the Patient-Reported Outcomes Measurement Information System (PROMIS®): Depression, Anxiety, and Anger. Assessment 2011 18: 263-283


27

From: Choi SW et al. Efficiency of static and computer adaptive short forms compared to full-length measures of depressive symptoms. Qual Life Res, 2010, 19: 125-136.


• Assess severity level, not clinical criteria

→ Test against DSM criteria using decision tree approach

• Validated in US

→ Validate internationally

• Limited array of mental health problems

→ Develop item banks for other disorders

28


29

Psychological distress

Psychosis Internalising

Depression Generalised

anxiety

Panic / agoraphobia

Social phobia

PTSD OCD

Externalising

Alcohol use/abuse

Substance use/abuse

Adult ADHD Anger

Suicidality


30

Emergency referral

Referral to MH professional

Referral to phone service

Referral to primary care

Low intensity online CBT

Psychoeducation

Watchful waiting

Item bank

Adaptive screening tools: •Anxiety •Depression •Suicidality, etc

Assessment: •Resources •Context •Preferences

Care decision

Ongoing monitoring

VIRTUAL CLINIC

STEPPED CARE SERVICE

Conclusions

• Hierarchical screening for multiple disorders can result in large efficiency gains without sacrificing accuracy

• Disorder-specific items more useful than general distress items

• Much promise in adaptive screening

31

Conclusions

• May be applied to – Virtual clinics

– Primary care screening

– Research

– School-based screening

– Other service contexts

32

33

Collaborators

Prof Helen Christensen Black Dog Institute, The University of New South Wales, Sydney Australia

Dr Alison Calear Centre for Mental Health Research, The Australian National University, Canberra Australia

Dr Matthew Sunderland and Dr Natacha Carragher National Drug and Alcohol Research Centre, The University of New South Wales, Sydney Australia

Prof Andrew Mackinnon Orygen Youth Health Research Centre, The University of Melbourne, Melbourne Australia

34

Acknowledgements

The College of Medicine, Biology and Environment, The Australian National University

Australasian Society for Psychiatric Research

National Health and Medical Research Council (Australia)

[email protected]

methods to improve the efficiency of screening for...

Documents