using routine data on needs and outcomes to improve clinical practice michael dennis, ph.d. chestnut...

Using Routine Data on Needs and Outcomes to Improve Clinical Practice

Michael Dennis, Ph.D.Chestnut Health Systems, Normal, IL

Presentation at “Serving EAP Clients: The Roles of Mental Health Practitioners in Managing Workplace Mental Health.”, Weaver Ridge, IL, October 1, 2010. Baltimore, MD, August 24-26, 2010.. This presentation

reports on treatment & research funded by SAMHSA contract 270-07-0191, as well as several individual CSAT, NIAAA, NIDA and private foundation

grants. The opinions are those of the author and do not reflect official positions of the consortium or government. Available on line at

www.chestnut.org/LI/Posters or by contacting Joan Unsicker at 448 Wylie Drive, Normal, IL 61761, phone: (309) 451-7801, Fax: (309) 451-7763, e-

mail: [email protected]

1. Examine the strengths and weakness of common performance measures

2. Explore epidemiological and research data on what we should expect

3. Illustrate how to use routinely collected data to improve the identification of client needs, target services, and improve outcomes in private and agency practices

Goals of this Presentation are to

While I will draw many examples from substance abuse treatment & recovery research (my field), they easily generalize to mental health

While I will use data from the Global Appraisal of Individual Needs (GAIN) (Chestnut’s instrument) the points are generic and apply to other measures as well.

Two Key Qualifiers..

Examples of Common Record Based Performance Measures

* NQF: National Quality Forum; WCG: Washington Circle Group; CSAT: Center for Substance Abuse Treatment evaluations; NOMS: National Outcome Monitoring System; NIATX: Network for the Improvement of Addiction Treatment; PFP: Pay for Performance evaluations

NQ

F

WC

G

CS

AT

NO

MS

NIA

TX P

FP

Initiation: Treatment within 2 weeks of diagnosis X X X X X

Engagement: 2 additional sessions within 30 days X X X X X

Continuing Care: Any treatment 90-180 days out X X X

Detox Transfer: Starting treatment within 2 weeks X X

Residential Step Down: Starting OP Tx w/in 2wks X

Evidenced Based Practice: From NREP/Other lists X X X X

Within Cost Bands: see French et al 2009 X X

Evaluation of these Existing Measures

Strengths:– Easy to collect/ calculate in electronic health records– Give broad overview of where problems– Useful for program evaluation and pay for

performance

Weaknesses:– Doesn’t lead to specific changes or intervention at

the individual level– Doesn’t address comorbidity or case mix– Doesn’t easily lead to specific improvement at the

program level – Doesn’t address relationships with other gaps in the

macro system

Examples of Additional Standards of Care Being Considered by NQF

Annual screening for tobacco, alcohol and other drugs using systematic methods

Referral for further multidimensional assessment to guide patient-centered treatment planning

Brief intervention, referral to treatment and supportive services where needed

Pharmacotherapy to help manage withdrawal, tobacco, alcohol and opioid dependence

Provision of empirically validated psychosocial interventions

Monitoring and the provision of continuing careSource: www.tresearch.org/centers/nqf_docs/NQF_Crosswalk.pdf

Source: 2008 CSAT AAFT Summary Analytic Dataset

553/771=72%unmet need

218/224=97% to targeted

771/982=79% in need

With electronic health records we can also focus on more substantive measures

Size of the Problem

Extent to which services are currently being targeted

Extent to which services are not reaching those in most need

Treatment Received in the first 3 months

Mental Health Need at Intake

No/Low Mod/High Total

Any Treatment 6 218 224

No Treatment 205 553 758

Total 211 771 982

Mental Health Problem (at intake) vs. Any MH Treatment by 3 months

79%

97%

72%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

% of Clients WithMod/High Need

(n=771/982)*

% w Need but No ServiceAfter 3 months

(n=553/771)

% of Services Going toThose in Need

(n=218/224)


Why Do We Care About Unmet Need?

If we subset to those in need, getting mental health services predicts reduced mental health problems

Both psychosocial and medication interventions are associated with reduced problems

If we subset to those NOT in need, getting mental health services does NOT predict change in mental health problems

Conversely, we also care about services being poorly targeted to those in need.

Residential Treatment need (at intake) vs. 7+ Residential days at 3 months

36%

52%

90%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

% of Clients WithMod/High Need

(n=349/980)*

% w Need but NoService After 3 months

(n=315/349)

% of Services Going toThose in Need (n=34/66)

Opportunity to redirect

existing funds through better

targeting


Prevalence of Lifetime Disorders and Past Year Remission in the US Household Population

Source: Dennis, Scott, Funk & Chan forthcoming; National Co morbidity Study Replication

47%

15%

8% 13% 25

%

10%

10%

8% 8%

37%

20%

19%

4% 2%

31%

7% 8% 7% 12%

5% 2%

13%

0%10%20%30%40%50%60%70%80%90%

100%

Any

Dis

orde

r

Any

Sub

stan

ce D

isor

der

Dru

g D

isor

der

Alc

ohol

Dis

orde

r

Ext

erna

lizi

ng D

isor

der

Con

duct

Dis

orde

r

Opp

osit

iona

l Def

iant

AD

HD

Inte

rmit

tent

Exp

losi

ve

Inte

rnal

izin

g D

isor

der

Any

Moo

d D

isor

der:

Maj

or D

epre

ssiv

e E

pi.

Dys

thym

ia

Bi-

Pola

r I

or I

I

Any

Anx

iety

Dis

orde

r:

Adu

lt S

epar

atio

n A

nxie

ty

Gen

eral

ized

Anx

iety

Dis

.

Post

trau

mat

ic S

tres

s D

is.

Soci

al P

hobi

a

Pani

c D

isor

der

Ago

raph

obia

Oth

er S

peci

fic

Phob

ia

Lifetime Disorder

Past Year Remission

SUD EXT INT

Data can help give our clients “HOPE”Recovery “Rates” (Remission/Lifetime)


44%

66% 83

%

77%

58%

89%

89%

50%

45%

41% 56

%

57%

43%

31% 39

%

71%

48%

48%

44%

42%

41%

30%

0%10%20%30%40%50%60%70%80%90%

100%

Any

Dis

orde

r

Any

Sub

stan

ce D

isor

der

Dru

g D

isor

der

Alc

ohol

Dis

orde

r

Ext

erna

lizi

ng D

isor

der

Con

duct

Dis

orde

r

Opp

osit

iona

l Def

iant

AD

HD

Inte

rmit

tent

Exp

losi

ve

Inte

rnal

izin

g D

isor

der

Any

Moo

d D

isor

der:

Maj

or D

epre

ssiv

e E

pi.

Dys

thym

ia

Bi-

Pola

r I

or I

I

Any

Anx

iety

Dis

orde

r:

Adu

lt S

epar

atio

n A

nxie

ty

Gen

eral

ized

Anx

iety

Dis

.

Post

trau

mat

ic S

tres

s D

is.

Soci

al P

hobi

a

Pani

c D

isor

der

Ago

raph

obia

Oth

er S

peci

fic

Phob

ia

Past Year Recovery Rate

SUD EXT INT

Data Teaches us that Comorbidity is the NORM


Lifetime Number of Disorders

Lifetime Pattern of Disorders

None54%

1 Disorder18%

2 Disorders10%

3 to 16 Disorders

18%

Substance Only3%

None48%

Sub.+Int4%

Ext.+Int.10%

Sub. + Ext. + Int. 8%

Sub.+Ext1%

Internalizing Only21%

Externalizing Only5%

(28%/46% Any)=61% Co-occurring

(13%/16% SUD)=81% Co-occurring

(19%/24% Ext)=79% Co-occurring

(22%/43% Int.)=51% Co-occurring

Comorbidity is also related who enters treatment..


Number of Disorders Pattern of Disorders

5%

39%

54%

75%

4%

29%

19%

50%

49%

64%

60%

79%

0%10%20%30%40%50%60%70%80%90%

100%

Non

e

1 D

isor

der

2 D

isor

ders

3 to

16

Dis

orde

rs

Non

e

Sub

stan

ce O

nly

Ext

erna

lizi

ng O

nly

Inte

rnal

izin

g O

nly

Sub

stan

ce+

Ext

erna

lizi

ng

Sub

stan

ce+

Inte

rnal

izin

g

Ext

erna

lizi

ng+

Inte

rnal

izin

g

Sub

. + E

xt.

+ I

nt.

Any Behavioral Health TxAny Mental Health TxAny Substance Disorder Tx

..And the likelihood of Recovery


Number of Disorders Pattern of Disorders

64%

50%

19%

68%

65%

41% 51

%

26%

24%

16%

0%10%20%30%40%50%60%70%80%90%

100%

Non

e

1 D

isor

der

2 D

isor

ders

3 to

16

Dis

orde

rs

Non

e

Subs

tanc

e O

nly

Ext

erna

lizi

ng O

nly

Inte

rnal

izin

g O

nly

Subs

tanc

e+E

xter

nali

zing

Subs

tanc

e+In

tern

aliz

ing

Ext

erna

lizi

ng+

Inte

rnal

izin

g

Sub.

+ E

xt.

+ I

nt.

Past YearRecovery Rate

Patterns of Comorbidity change with Age

Source: Chan, YF; Dennis, M L.; Funk, RR. (2008). Prevalence and comorbidity of major internalizing and externalizing problems among adolescents and adults presenting to substance abuse treatment. Journal of Substance Abuse Treatment, 34(1) 14-24 .

Internalizing Disorders go up

with age

Externalizing Disorders go down

with age (but do NOT go away)

Substance Use & Disorders Also Vary by Age

Source: 2002 NSDUH and Dennis & Scott 2007

0

10

20

30

40

50

60

70

80

90

100

12-13

14-15

16-17

18-20

21-29

30-34

35-49

50-64

65+

No Alcohol or Drug Use

Light Alcohol Use Only

Any Infrequent Drug Use

Regular AOD Use

Abuse

Dependence

NSDUH Age Groups

Severity Category

Over 90% of use and

problems start between the ages of

12-20

It takes decades before most recover or die

People with drug dependence die an

average of 22.5 years sooner than those

without a diagnosis

Higher Severity is Associated with Higher Annual Cost to Society Per Person

Source: 2002 NSDUH

$0$231 $231

$725$406

$0$0

$500

$1,000

$1,500

$2,000

$2,500

$3,000

$3,500

$4,000

No Alcohol orDrug Use

Light Alcohol

Use Only

AnyInfrequentDrug Use

Regular AODUse

Abuse Dependence

Median (50th percentile)

$948

$1,613

$1,078$1,309

$1,528

$3,058Mean (95% CI)

This includes people who are in recovery, elderly, or do not use

because of health problems Higher Costs

Photo courtesy of the NIDA Web site. From A Slide Teaching Packet: The Brain and the Actions of Cocaine, Opiates, and Marijuana.

pain

Adolescent Brain Development Occurs from the

Inside to Out and from Back to Front

There 41.4 Million Under Age or Problem Drinkers in the U.S.

54%

31%

34%

58%

3%

4%

9%

7%

0% 20% 40% 60% 80% 100%

Age 12 to 20(38.1mil)

Age 21+(207.9mil)

No use in past yearOnly light alcohol use in the past yearHeavy alcohol use in the past monthAlcohol abuse or dependence in the past year

17.6 Million under age

drinkers (46% of 38.1 Mil)

28.4 Million (12%) Problem Drinkers

(4.6m/12% of youth, 23.8m/11% of adult)

Source: SAMHSA 2006. National Survey On Drug Use And Health, 2006 [Computer file]

NOTE: Not asked about work if under age 15 in NSDUH

Potential Screening/ Intervention Sites: Age 12 to 20 (38.1 million)


5% 8%

0%

30%

52%

90%

6% 10%

2%

36%

75%

95%

7% 9% 5%

38%

89% 96

%

7%

15%

10%

41%

81%

95%

0%

20%

40%

60%

80%

100%

Hosptial MentalHealth Tx

SubstanceAbuse Tx

EmergencyRoom

Workplace School

% A

ny C

onta

ct


Key potential of Workplace (e.g., EAP, Wellness, HRA) and School (e.g., SAP,

EI, Prevention) Programs

Potential Screening/ Intervention Sites: Age 21+ (207.9 million)


16%

12%

1%

32%

58%

10%

13%

1%

27%

80%

7% 8%

1%

26%

87%

8%

21%

8%

34%

89%

0%

20%

40%

60%

80%

100%

Hosptial Mental HealthTx

SubstanceAbuse Tx

EmergencyRoom

Workplace

% A

ny C

onta

ct


Key potential of Workplace Programs

NOTE: Not asked about School if over age 18 in NSDUH

How does data related to the move towards Evidence Based Practice (EBP)?

EBP means introducing explicit intervention protocols – Targeted at specific problems/subgroups and outcomes– Having explicit quality assurance procedures to cause

adherence at the individual level and implementation at the program level

Reliable and valid assessment is needed that can be used to – Immediately guide clinical judgments about

diagnosis/severity, placement, treatment planning, and the response to treatment at the individual level

– Drive longer term program evaluation, needs assessment, performance monitoring and program planning

– Allow evaluation of the same person or program over time– Allow comparisons with other people or interventions

Major Predictors of Bigger Effects Found in Multiple Meta Analyses

1. Triage to focus on the highest severity subgroup

2. A strong intervention protocol based on prior evidence

3. Quality assurance to ensure protocol adherence and project implementation

4. Proactive case supervision of individual

Impact of the numbers of these Favorable features on Recidivism in 509 Juvenile Justice Studies in Lipsey Meta Analysis

Source: Adapted from Lipsey, 1997, 2005

Average Practice

The more features, the lower

the recidivism

Impact of Intake Severity on Outcome

Source: ATM Main Findings data set

SPSM groupings

Dot/Lines show Means

0 6

Wave

8

10S

ub

stan

ce P

rob

lem

Sca

le

(0-1

6 P

ast

Mon

th S

ymp

tom

s)

No problems (0-25%ile)

1-3 problems (25-50%ile)

4-8 problems (50-75%ile)

9+ problems (75-100%ile)

OVERALL

6

4

2

0

Intake Severity Correlated -.66 with amount of

change

• Programs with low severity look better with absolute outcomes (e.g. abstinence)

• Programs with high severity look better with amount of change

Example of Generic vs. Targeted Effects

-0.0

3

-0.1

0 -0.0

2

-0.80

-0.60

-0.40

-0.20

0.00

0.20

0.40

A. Low RiskW/O Trauma

B. Mod. RiskW/O Trauma

C. Mod. RiskWith Trauma

D. High RiskWith Trauma

Total

Coh

en's

Eff

ect S

ize

d

Unprotected Sex Acts (f=.14)

Days of Victimization (f=.22)

Days of Needle Use (f=1.19)

-0.3

9

0.20

-0.0

4

-0.0

8

0.00

0.15

-0.2

9

0.01

0.10

0.27

0.00

-0.6

9

Source: Lloyd et al 2007

GenericTargeted

Evidenced Based Treatment (EBT) that Typically do Better than Usual Practice in Reducing Juvenile Recidivism (29% vs. 40%)

Aggression Replacement Training Reasoning & Rehabilitation Moral Reconation Therapy Thinking for a Change Interpersonal Social Problem Solving MET/CBT combinations and Other manualized CBT Multisystemic Therapy (MST) Functional Family Therapy (FFT) Multidimensional Family Therapy (MDFT) Adolescent Community Reinforcement Approach (ACRA) Assertive Continuing Care

Source: Adapted from Lipsey et al 2001, Waldron et al, 2001, Dennis et al, 2004

NOTE: There is generally little or no differences in mean effect size between these brand names

Implementation is Essential (Reduction in Recidivism from .50 Control Group Rate)

The effect of a well implemented weak program is

as big as a strong program implemented poorly

The best is to have a strong

program implemented

well

Thus one should optimally pick the strongest intervention that one can

implement wellSource: Adapted from Lipsey, 1997, 2005

30

Percentage Change in Abstinence (6 mo-Intake) by level of Adolescent Community Reinforcement Approach (A-CRA) Quality Assurance

4%

24%36%

0%10%20%30%40%50%60%70%80%90%

100%

Training Only Training,Coaching,

Monitoring

Clinical TrialOnsite Protocol

Monitors

% P

oint

Cha

nge

in A

bsti

nenc

e

Source: CSAT 2008 SA Dataset subset to 6 Month Follow up (n=1,961)

Effects associated with intensity of quality

assurance and monitoring (OR=13.5)

31

Illustration of the Need for Proactive case Supervision of Individual: Prevalence of 12 problems

20%

41%

80%

48%

33%

63%

11%

24%

14%

34%

27%0% 10

%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Alcohol

Cannabis

Other drug disorder

Depression

Anxiety

Trauma

ADHD

CD

Suicide

Victimization

Violence/ illegal activity

Source: CSAT 2009 Summary Analytic Data Set (n=20,826)

32

The Number of Major Clinical Problems by Level of Care

41% 45%53%

65%

80%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Outpatient IntensiveOutpatient

Cont. CareOutpatient

Long TermResidential

Short TermResidential

None

One

Two

Three

Four

Five to Twelve


Significantly more likely to

have 5+ problems (OR=5.8)

33

46%

71%

15%0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Low (0) Moderate (1-3) High (4-15)

None

One

Two

Three

Four

Five to Twelve

The Number of Major Clinical Problemsis highly related to Victimization


Significantly more likely to have 5+

problems (OR=13.9)

But this is the issue staff least

like to ask about!

Overcoming Person or Staff Reluctance with the GAIN General Victimization Scale

40%

31%

6%10%

1%8%9%

26%

29%7%

57%32%

19%11%

35%

0%

10

%

20

%

30

%

40

%

50

%

60

%

70

%

80

%

90

%

10

0%

Ever attacked w/ gun, knife, other weapon

Ever hurt by striking/beating

Abused emotionally

Ever forced sex acts against your will/anyone

Age of 1st abuse < 18

Any with more than one person involved

Any several times or for long time

Was person family member/trusted one

Were you afraid for your life/injury

People you told not believe you/help you

Result in oral, vaginal, anal sex

Currently worried someone attack

Currently worried someone beat/hurt

Currently worried someone abuse emotionally

Currently worried someone force sex acts

Source: CSAT 2009 Summary Analytic Data Set (n=19,318) 34

The GAIN is ..

A family of instruments ranging from screening, to quick assessment to a full Biopsychosocial and monitoring tools

Designed to integrate clinical and research assessment

Designed to support clinical decision making at the individual client level

Designed to support evaluation and planning at program level

Designed to support secondary analyses and comparisons across individuals and programs

The GAIN is NOT an electronic health record (EHR), but a component that can interface with and support EHRs.

More in BZ, CA, CN, JP, MX

ID

ILMO

ND

VI

ME

OK

PR

SD

AR

KS

MS

MT

NM

WVIN

AL

AK

IA

MN

NJNV

RI

SC

UT

HI

LA

DENE

TN

PA

VT

VADC

MI

COKY

GA

OH

OR

MD

AZ

TX

NY

NH

WI

CA

NC

CT

FL

MA

WA

WY

No of GAIN Sites

None (Yet)

1 to 14

15 to 30

31 to 165

Global Appraisal of Individual Needs (GAIN) Network of Collaborators

State or Regional System

GAIN-Short Screener

GAIN-Quick

GAIN-Full

3/10 36

Some numbers as of June 2010

1,501 Licensed GAIN administrative units from 49 states (all by ND) and 7 countries

3,270 users in 396 Agencies using GAIN ABS

60,380 intake assessments (largest in field)

22,045 (88% w 1+ follow-up) from 278 CSAT grantees

22 states, 12 Federal, 6 Canadian provinces, 6 other countries, and 3 foundations mandate or strongly encourage its use

4 dozen researchers have published 179 GAIN-related research publications to date

37

Crosses a Continuum of Measurement (Common Measures)

Screening to Identify Who Needs to be “Assessed” (5-10 min)– Focus on brevity, simplicity for administration & scoring– Needs to be adequate for triage and referral– GAIN Short Screener for SUD, MH & Crime– ASSIST, AUDIT, CAGE, CRAFT, DAST, MAST for SUD– SCL, HSCL, BSI, CANS for Mental Health– LSI, MAYSI, YLS for Crime

Quick Assessment for Targeted Referral (20-30 min)– Assessment of who needs a feedback, brief intervention or referral for

more specialized assessment or treatment– Needs to be adequate for brief intervention– GAIN Quick – ADI, ASI, SASSI, T-ASI, MINI

Comprehensive Biopsychosocial (1-2 hours) – Used to identify common problems and how they are interrelated– Needs to be adequate for diagnosis, treatment planning and placement

of common problems– GAIN Initial (Clinical Core and Full)– CASI, A-CASI, MATE

Specialized Assessment (additional time per area)– Additional assessment by a specialist (e.g., psychiatrist, MD, nurse,

spec ed) may be needed to rule out a diagnosis or develop a treatment plan or individual education plan

– CIDI, DISC, KSADS, PDI, SCAN

Screener Quick C

omprehensive S

pecial

More E

xtensive / Longer/ E

xpensive

Longer assessments identify more areas to address in treatment planning

40%

69%

94%98%

22%

13%

3% 0%

22%

8%

1% 0%

9%8%

1% 1%3% 1% 1%7%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

GAIN SS GAIN Q(v2)

GAIN Q(v3 -Beta)

GAIN I

0 Reported

1 Prob.

2 Probs.

3 Probs.

4 Probs.

Source: Reclaiming Futures Portland, OR and Santa Cruz, CA sites (n=192)

Most substance users have multiple problems

39

5 min. 20 min 30 min 1-2 hr

40

Expected Factor Structure of Psychopathology and Psychopathy

Source: Dennis, Chan, and Funk (2006)

GAIN Short Screener (GAIN-SS)

Administration Time: A 5-minute screener Purpose: Used in general populations to

– identify or rule out clients who will be identified as having any behavioral health disorders on the 60-120 min versions of the GAIN

– triage area of problem– serve as a simple measure of change– ease administration and interpretation by staff with minimal training or direct

supervision Mode: Designed for self- or staff administration, with paper and pen, computer, or

on the web Languages: English, Spanish, French, Portuguese, Simple & Traditional Chinese

& 15 other languages Scales: Four screeners for Internalizing Disorders, Externalizing Disorders,

Substance Disorders, and Crime/Violence Disorders, and a Total Disorder Screener

Response Set: Recency of 20 problems rated past month (3), 2-12 months ago (2), more than a year ago (1), never (0)

Interpretation: Combined by cumulative time period as: − Past-month count (3s) to measure change− Past-year count (2s or 3s) to predict diagnosis− Lifetime count (1s, 2s, or 3s) as a measure of peak severity

– Can be classified within time period as low (0), moderate (1-2), or high (3)

– Can also be used to classify remission as − Early (lifetime but not past month)− Sustained (lifetime but not past year)

Reports: Narrative, tabular, and graphical reports built into web- based GAIN ABS or ASP application for local hosting

GAIN Short Screener (GAIN-SS) (continued)

Source: Dennis, Chan, and Funk (2006) www.chestnut.org/LI/gain/GAIN_SS

Screener items were selected using the Rasch Measurement Model

-1.89 -.8 -.32 +.28 +.71Items around key

decision pointSource: Riley et al 2007 43

Why do we use a cut point of 1 on the Substance Disorder Screener?

A cut point of 1 has 96% sensitivity and 73%

specificity (i.e., it gets most real cases but has

some false cases)

Source: Dennis et al 2006

A cut point of 1 has 68% sensitivity and 100%

specificity (i.e., it misses almost a third of real cases but has virtually no false

cases

Best Recommendation:

1+ on SDScr and

3+ on TDScr44

Construct Validity of GSS Internalizing Disorder Screener

0%10%20%30%40%50%60%70%80%90%

100%

% Days with MHproblem

Mod/High onEmotional Problem

Scale (EPS)

Mod/High onInternal MentalDistress Scale

(IMDS)

Internalizing Disorder Screener (IDScr)

Fu

ll G

AIN

mea

sure

0 1 2 3 4 5

Source: Dennis 2009, Education Service District 113 (n=979) and King County (n=1002) 45

Construct Validity of GSS Externalizing Disorder Screener

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

% Days withbehavioralproblems

Mod/High onEmotional Problem

Scale (EPS)

High on BehaviorComplexity Scale

(BCS)

Externalizing Disorder Screener (EDScr)

Fu

ll G

AIN

mea

sure

0 1 2 3 4 5


Construct Validity of GSS Substance Disorder Screener

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

% Days of AOD use

Past Year Abuse orDependence

Past YearDependence

Substance Disorder Screener (SDScr)

Fu

ll G

AIN

mea

sure

0 1 2 3 4 5


Construct Validity of GSS Crime/Violence Screener

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

% Days of illegalactivities

Mod/High onIllegal Activity

Scale (IAS)

High onCrime/Violence

Scale (CVS)

Crime and Violence Screener (CVScr)

Fu

ll G

AIN

mea

sure

0 1 2 3 4 5


Adolescent Rates of High (2+) Scores on Mental Health (MH) or Substance Abuse (SA) Screener by Setting

in Washington State

77% 86

%

73%

75%

61%67

%

83%

62%

75%

60%

57%

40% 46

%

12%

12%

47%

37%

35%

12%

11%

0%10%20%30%40%50%60%70%80%90%

100%

Substance AbuseTreatment(n=8,213)

Student AssistancePrograms(n=8,777)

Juvenile Justice(n=2,024)

Mental HealthTreatment (10,937)

Children'sAdministration

(n=239)

Either High on Mental Health High on Substance High on Both

Source: Lucenko et al. (2009). Report to the Legislature: Co-Occurring Disorders Among DSHS Clients. Olympia, WA: Department of Social and Health Services. Retrieved from http://publications.rda.dshs.wa.gov/1392/

Problems could be easily identified Comorbidity is common

http://publications.rda.dshs.wa.gov/1392/

35%

12%

11%

56%

34%

15%

9%

47%

0%10%20%30%40%50%60%70%80%90%

100%

Substance AbuseTreatment (n=8,213)

Juvenile Justice(n=2,024)

Mental HealthTreatment (10,937)

Children'sAdministration

(n=239)

GAIN Short Screener Clinical Indicators

Adolescent Client Validation of Hi Co-occurring from GAIN Short Screener vs Clinical Records

by Setting in Washington State

Two page measure closely approximated all found in the clinical record after the next two years



0 5,000 10,000 15,000 20,000 25,000

Any BehavioralHealth (n=22,879)

Mental Health(21,568)

Substance AbuseNeed (10,464)

Co-occurring(9,155)

Substance Abuse Treatment Student Assistance ProgramJuvenile Justice Mental Health TreatmentChildren's Administration

Where in the System Are the Adolescents with Mental Health, Substance Abuse, and Co-occurring?


2/3 of the teens with mental health issues are seen in

substance abuse treatment or student assistance programs

Student assistance programsrepresent 1/3 of the

behavioral health system


Adult Rates of High (2+) Scores on Mental Health (MH) or Substance Abuse (SA) Screener


81%

78%

65%

64% 69

%

18%

68% 73

%

43%

44%

69%

17%

69%

51%

53%

51%

17%

4%

56%

46%

31%

31%

17%

3%

0%10%20%30%40%50%60%70%80%90%

100%

SubstanceAbuse

Treatment(n=75,208)

Eastern StateHospital(n=422)

Corrections:Community(n=2,723)

Corrections:Prison

(n=7,881)

Mental HealthTreatment(55,847)

ChildrensAdministration

(n=1,238)

Either High on Mental Health High on Substance High on Both

Lower than expected rates of SA in mental health and children’s

admin



0 20,0

00

40,0

00

60,0

00

80,0

00

100,

000

120,

000

Any Behavioral Health (n=106,818)

Mental Health (n=94,832)

Substance Abuse (n=67,115)

Co-Occurring (n=55,128)

Substance Abuse Treatment Eastern State HospitalCorrections: Community Corrections: PrisonMental Health Treatment Childrens Administration

Where in the System Are the Adults with Mental Health, Substance Abuse, and Co-occurring?


More mental health treated in substance

abuse treatment


Adult Client Validation of Hi Co-occurring from GAIN Short Screener vs Clinical Records


17%

3%

59%

39%

22%

56%

0%

10%20%

30%40%

50%

60%70%

80%90%

100%

Substance Abuse Treatment(n=75,208)

Mental Health Treatment(55,847)

Childrens Administration(n=1,238)

GAIN Short Screener Clinical Indicators

Higher rate in clinical record in mental health and children’s administration. But that was based on -“any use” vs. “week use + abuse/dependence”

- and 2 years vs. past year



Other Validations of GAIN Short ScreenerSubstance Disorders: McDonnell and colleagues (2009) found that the 5-item GAIN SS Substance

Disorder Screener had 92% sensitivity and 85% correct classification relative to the Diagnostic Inventory Scale for Children (DISC) Predictive Scales (DPS; Lucas et al 2001) and 88% sensitivity and 88% correct classification relative to the CRAFFT (Knight et al 2001)

Internalizing Disorders: McDonnell and colleagues (2009) found that the 5-item GAIN SS Internalizing

Disorder Screener had 100% sensitivity and 75% correct classification relative to the Youth Self Report (YSR; Achenbach et al, 2001) and that the 5-item GAIN SS Externalizing Disorder Screener had 89% sensitivity and 65% correct classification to the YSR.

Riley and colleagues (2009) found that the 5-item GAIN SS’s Internalizing Disorder Screener had 92% sensitivity and 80% area under the curve relative to the Structured Clinical Interview for DSM (SCID) and was more efficient relative to 11 item Addiction Severity Index (ASI) psychiatric composite score (McLellan et al., 1992), 10 item K10 (Kessler et al., 2002) and the 87 item Psychiatric Diagnostic Screening Questionnaire (PDSQ; Zimmerman and Mattia, 2001)

55

GAIN SS can generate narrative and summaryreports

0%1%2%3%4%5%6%7%8%9%

10%11%

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Total Disorder Sceener (TDScr) Score

% w

ithi

n L

evel

of

Car

e

Residential (n=1,965)

OP/IOP (n=2,499)

Low

Mod. High ->

57

Total Disorder Screener Severity Predicts Level of Care: Adolescents

Source: SAPISP 2009 Data and Dennis et al 2006

Residential Median= 10.5(59% at 10+)

Outpatient Median=6.0(30% at 10+)

Few missed

(1/2-3%)

0%1%2%3%4%5%6%7%8%9%

10%11%12%

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Total Disorder Sceener (TDScr) Score

% w

ithi

n L

evel

of

Car

e

Residential (n=1,965)

OP/IOP (n=2,499)

Low

Mod. High ->

58

Total Disorder Screener Severity Predicts Level of Care: Adults

Source: SAPISP 2009 Data and Dennis et al 2006

Residential Median= 8.5(59% at 10+)

Outpatient Median=4.5(29% at 10+)

10% of adult OP missed)

GAIN SS Can Also be Used for Monitoring

109

11

910

8

32 2

0

4

8

12

16

20

Intake 3Mon

6Mon

9Mon

12Mon

15Mon

18Mon

21Mon

24Mon

Total Disorder Screener (TDScr)

12+ Mon.s ago (#1s)

2-12 Mon.s ago (#2s)

Past Month (#3s)

Lifetime (#1,2,or 3)

Track Gap Between Prior and current

Lifetime Problems to identify “under

reporting”

Track progress in reducing current

(past month) symptoms)

Monitor for Relapse

Use of a short common screener can

Provide immediate clinical feedback that is a good approximation of diagnosis and be used to guide placement and treatment planning

Can be used repeatedly to track change

Support evaluation and planning at program or state level (e.g., needs, case mix, services needed)

Provide practice based evidence to guide future clinical decision

Be incorporated into health risk/ wellness assessments and/or school surveys

GAIN Quick adds 20-30 minutes &

GAIN SS scales + similar scales for school, work, physical health, psychosocial stress, and HIV risks

Additional “days” items and scale for measuring behavioral change

Recency and past 90 day measures of service utilization in each area to aid in placement, track implementation and estimate quarterly costs to society

Reasons for change to support motivational interviewing in each area

Life Satisfaction Scale, Quality of Life and Interview quality documentation

GAIN I adds 60-90 minutes &

The GI has 9 sections (access to care, substance use, physical health, risk and protective behaviors, mental health, recovery environment, legal, vocational, and staff ratings) that include 103 long (alpha over .9) and short (alpha over .7) scales, summative indices, and over 3000 created variables to support clinical decision making and evaluation. It is also modularized to support customization

Designed to provide a standardized biopsychosocial for people presenting to a substance abuse treatment using DSM-IV for diagnosis, ASAM for placement, and needing to meet common (CARF, COA, JCAHO, insurance, CDS/TEDS, Medicaid, CSAT, NIDA) requirements for assessment, diagnosis, placement, treatment planning, accreditation, performance/outcome monitoring, economic analysis, program planning and to support referral/communications with other systems

References Dennis, M.L., Foss, M.A., & Scott, C.K (2007). An eight-year perspective on the relationship between the duration of abstinence and other aspects of recovery. Evaluation Review,

31(6), 585-612 Dennis, M. L., Scott, C. K. (2007). Managing Addiction as a Chronic Condition. Addiction Science & Clinical Practice , 4(1), 45-55. Dennis, M. L., Scott, C. K., Funk, R., & Foss, M. A. (2005). The duration and correlates of addiction and treatment careers. Journal of Substance Abuse Treatment, 28, S51-S62. Ettner, S.L., Huang, D., Evans, E., Ash, D.R., Hardy, M., Jourabchi, M., & Hser, Y.I. (2006). Benefit Cost in the California Treatment Outcome Project: Does Substance Abuse

Treatment Pay for Itself?. Health Services Research, 41(1), 192-213. Fowler JS, Volkow ND, Wolf AP, Dewey SL, Schlyer DJ, Macgregor RIR, Hitzemann R, Logan J, Bendreim B, Gatley ST. et al. (1989) Synapse, 4(4):371-377. French, M.T., Popovici, I., & Tapsell, L. (2008). The economic costs of substance abuse treatment: Updated estimates of cost bands for program assessment and reimbursement.

Journal of Substance Abuse Treatment, 35, 462-469 Lucenko et al (2009). Report to the Legislature: Co-Occurring Disorders Among DSHS Clients. Olympia, WA: Department of Social and Health Services. Retrieved from

http://publications.rda.dshs.wa.gov/1392/ Neumark, Y.D., Van Etten, M.L., & Anthony, J.C. (2000). Drug dependence and death: Survival analysis of the

Baltimore ECA sample from 1981 to 1995. Substance Use and Misuse, 35, 313-327. Office of Applied Studies (OAS; 2005). Treatment Episode Data Set (TEDS): 2002. Discharges from Substance Abuse Treatment Services, DASIS Series: S-25, DHHS

Publication No. (SMA) 04-3967, Rockville, MD: Substance Abuse and Mental Health Services Administration. Retrieved from http://wwwdasis.samhsa.gov/teds02/2002_teds_rpt_d.pdf and Office of Applied Studies 2006 Discharge – Treatment Episode Data Set (TEDS) http://www.samhsa.gov/oas/dasis.htm .

Office of Applied Studies (2006). Results from the 2005 National Survey on Drug Use and Health: National Findings Rockville, MD: Substance Abuse and Mental Health Services Administration. http://www.oas.samhsa.gov/NSDUH/2k5NSDUH/2k5results.htm#7.3.1

Riley, B.B.,, Scott, C.K, & Dennis, M.L. (2008). The effect of recovery management checkups on transitions from substance use to substance abuse treatment and from treatment to recovery. Poster presented at the UCLA Center for Advancing Longitudinal Drug Abuse Research Annual Conference, August 13-15, 2008, Los Angles, CA. www.caldar.org .

Scott, C. K., & Dennis, M. L. (2009). Results from Two Randomized Clinical Trials evaluating the impact of Quarterly Recovery Management Checkups with Adult Chronic Substance Users. Addiction, ??

Scott, C. K., Dennis, M. L., Simeone, R., & Funk R. (forthcoming). Predicting the likelihood of death of substance users over 9 years based on baseline risk, treatment and duration of abstinence. Chicago, IL: Chestnut Health Systems.

Scott, C. K., Foss, M. A., & Dennis, M. L. (2005). Pathways in the relapse, treatment, and recovery cycle over three years. Journal of Substance Abuse Treatment, 28, S61-S70.


http://www.samhsa.gov/oas/dasis.htm

http://www.oas.samhsa.gov/NSDUH/2k5NSDUH/2k5results.htm#7.3.1

http://www.caldar.org/

using routine data on needs and outcomes to improve clinical practice michael dennis, ph.d. chestnut...

Documents

treatment research

starting treatment

patientcentered treatment

research data

macro system slide

individual csat

doesnt lead

il presentation