Validation of self-administered Validation of self-administered single item screening questions single item screening questions
(SISQs) for unhealthy alcohol and (SISQs) for unhealthy alcohol and other drug use in primary care other drug use in primary care
patients at two sitespatients at two sites
Jennifer McNeely, Charles M. Cleland, Shiela M. Strauss, Joseph J. Palamar, John Rotrosen, Marc N. Gourevitch, Richard Saitz
No relevant financial relationships to disclose
ObjectivesObjectives
1. Describe the need for a self-administered approach to substance use screening
2. Single Item Screening Questions (SISQs) for alcohol and drug use
3. Present results of a validation study in primary care
4. Discuss feasibility and application
Screening for Screening for substance use in substance use in
primary careprimary care• Medical providers fail to identify
clinically relevant substance use• Barriers to screening:
o Timeo Workflowo Knowledge/Trainingo Discomforto Attitudes
D’Amico, Medical Care 2005
Sterling, Addict Med Clin Pract 2012Friedmann, J Gen Int Med 2000Friedmann, Arch Int Med 2001Anderson, Alcohol Alcoholism 2004McCormick, J Gen Int Med 2006
Self-administered screening Self-administered screening is a more feasible approach is a more feasible approach
Education Monitoring
Office-based counseling
Treatment
Screening
Assessment
Low Risk Moderate RiskHigh Risk or Dependence
+
Screening: SISQ-alcohol and SISQ-drug
Single Item Screening Single Item Screening
QuestionsQuestions
• SISQ-alcoholHow many times in the past year have you had X or
more drinks in a day?(X=5 for men, and X=4 for women)
• SISQ-drugHow many times in the past year have you used an
illegal drug or used a prescription medication for non-medical reasons (for example, because of the experience or feeling it caused?
Prior validation of SISQsPrior validation of SISQs
• Adult primary care patients (N=286)• Single site, urban safety net medical
center• Good sensitivity and specificity for
detection of unhealthy use • SISQ-alcohol: Sensitivity 82%, Specificity
79% • SISQ-drug: Sensitivity 85%, Specificity
96%Smith et al., JGIM 2009Smith et al., Arch Int Med 2010
Current Study Current Study
Screening (computer) Validation Measures (interviewer)
ReferralsIncentive
• Timeline Follow Back
• SIP-A and SIP-D• MINI-Plus
• REALM• Demographics
Saliva drug screen
Second Consent
• SUBS• SISQ-alcohol, SISQ-
drug
Reference standard Reference standard measuresmeasures
Timeline follow-back (30d)
SIP-A SIP-DMINI-Plus screening
MINI-Plus abuse or dep
Intercept oral fluid test*
Alcohol Unhealthy use
+ + +
Disorder +
DrugsUnhealthy use
+ + + +
Disorder +
* Collected at Site A only
Statistical AnalysisStatistical Analysis
1.Comparison of SISQs to composite reference standards
2.Examined site differences3.Calculate sensitivity, specificity, AUC:oUnhealthy useoSubstance use disorder
4. Subgroup analyses
Study Sites and Study Sites and RecruitmentRecruitment
• Adult primary care clinics• 2 urban safety net hospitals • Patients presenting for medical visits• Consecutive recruitment
Eligibility Criteria: • Age 21-65• Current clinic patient• Fluent in English• No disability preventing computer use
Participant Recruitment
Screened: N = 2131
Eligible: N = 915
1216 were excludedLanguage: 679Age: 306Not a patient: 168Other: 115
Completed interview: N = 459Site A: 265*Site B: 194
453 declinedNo time: 363Other: 90
1 lost data
*230 (87%) Site A participants agreed to saliva test
Characteristics of the 459 Characteristics of the 459
participantsparticipants
Age (years) Mean = 46, SD = 12Range = 21-65
Sex (%) MaleFemaleTransgender
48.451.20.4
Race/Ethnicity (%) Black/African American HispanicWhite/Caucasian Other
51.820.219.18.6
Country of Birth (%)
United StatesOutside of United States
64.635.3
Education and Health LiteracyEducation and Health Literacy
Highest Level of Education Health Literacy Level
Prevalence of substance usePrevalence of substance use
Substance Past year use (MINI)
N (%)
Past month use(TLFB)
N (%)
Alcohol 103 (22.3)a 89 (19.3)b
Drugs 114 (24.7)c 73 (15.8)c
Specific drug categoriesIllicit drugs 108 (23.4)
Marijuana -- 58 (12.6)Cocaine -- 12 (2.6)Heroin -- 10 (2.2)Hallucinogens -- 1
Prescription drugs (non-medical use)
21(4.6)
Opioids -- 5 Benzodiazepines -- 3 Stimulants -- 2
Unhealthy useUnhealthy use
Substance + on SISQ
N (%)
+ on Reference
N (%)
Sensitivity
%(95% CI)
Specificity
%(95% CI)
AUC
(95% CI)
Alcohol 155 (34) 146 (32) 73.3 (65.3, 80.3)
84.7 (80.2, 88.5)
0.79 (0.75, 0.83)
Drugs 107 (23) 122 (27) 71.3 (62.4, 79.1)
94.3 (91.3, 96.6)
0.83 (0.79, 0.87)
Oral fluid test results: 8 tested positive, all reported use on SISQ
No change to results
Substance use disorderSubstance use disorder
Substance + on SISQ
N (%)
+ on Reference
N (%)
Sensitivity
%(95% CI)
Specificity
%(95% CI)
AUC
(95% CI)
Alcohol 155 (34) 60 (13) 86.7 (75.4, 94.1)
74.2 (69.6, 78.4)
0.80 (0.76, 0.85)
Drugs 107 (23) 74 (16) 85.1 (75.0, 92.3)
88.6 (85.0, 91.6)
0.87 (0.83, 0.91)
Subgroup AnalysisSubgroup AnalysisSubgroups anticipated to have greater difficulty with self-administered screening:•Male•Age greater than 50•Hispanic/Latino•Primary language other than English•Born outside US•Education or health literacy lower than high school level
Subgroup AnalysisSubgroup Analysis• No differences for SISQ-alcohol• Lower sensitivity of SISQ-drug among:Primary language other than English (p<0.01)
Less than high school education (p<0.01)
Sensitivity Specificity
English 74.3 (65.1, 82.2) 94.4 (90.7, 96.9)
Non-English 46.2 (19.2, 74.9) 94.3 (87.1, 98.1)
Sensitivity Specificity
High school 79.0 (66.8, 88.3) 95.2 (91.0, 97.8)
< High school 63.3 (49.9, 75.4) 93.3 (88.0, 96.7)
LimitationsLimitations
• Safety net primary care populations• English speaking only• Tested in research context, with
assurance of confidentiality
ConclusionsConclusions• SISQs accurately identified unhealthy
substance us in primary care patients
• Lower sensitivity and specificity than interviewer-administered versions
• Efficiency, fidelity, and patient comfort may be advantages to self-administered approach
AcknowledgementsAcknowledgementsFunding:• K23 Career Development Award
NIDA K23 DA030395• NYU-HHC CTSI Pilot Grant
NIH/NCATS UL1 TR000038 • The MITRE Corporation (contract
from ONC and SAMHSA)
Staff and others:•Seville Meli•Jacqueline German•Ritika Batajoo•Catherine Federowicz•Marshall Gillette•Charlie Jose•Emily Maple•Keshia Toussaint•Julianne Cameron•Arianne Ramautar•Derek Nelsen•Linnea Russell•Study participants